Re: [lng-odp] [PATCH CATERPILLAR v1] [RFC] odp: pktio: added tcpdump like functionality

2018-02-09 Thread Github ODP bot
Mykyta Iziumtsev(MykytaI) replied on github web page:

platform/linux-generic/include/odp_pcapng.h
line 1
@@ -0,0 +1,56 @@
+/* Copyright (c) 2018, Linaro Limited


Comment:
Maybe consider moving code to separate .c file then if it has nothing to do 
with packet io ?

> Ilias Apalodimas(apalos) wrote:
> too much unrelated info to the packet io. I'd prefer this reside on a 
> different file


>> Mykyta Iziumtsev(MykytaI) wrote:
>> I believe sizeof(struct) directly in code is more informative.


>>> Mykyta Iziumtsev(MykytaI) wrote:
>>> I'm not sure event aggregation has any value here, but even if it does -- 
>>> we don't need buffer for 1024 events. 


 Ilias Apalodimas(apalos) wrote:
 The + 16 is just a reasonable guess. The event inotify returns might or 
 might not have an event->name of dynamic size. If the buffer is too small, 
 the system call returns zero. Since it allows event slurping i am trying 
 to read multiple events at once.
 
 i could change this and use FIONREAD ioctl to get the exact number of 
 bytes i can read from the inotify fd


> Ilias Apalodimas(apalos) wrote:
> will fix


>> Ilias Apalodimas(apalos) wrote:
>> will fix


>>> Mykyta Iziumtsev(MykytaI) wrote:
>>> Any justification for +16 and such a big buffer on a stack ? Do we 
>>> really need 1024 events at once ?


 Mykyta Iziumtsev(MykytaI) wrote:
 Shall be _t


> Mykyta Iziumtsev(MykytaI) wrote:
> In fact we do care. We can't tolerate partial packet writes, 
> otherwise pcapng file will be unreadable. The proper way to address 
> that is to see how much space is left in the pipe (buf_size - 
> TIOCOUTQ) and append packets to iovec as long as they fit in the 
> calculated 'budget'.
> 
> Another thing to consider -- is increasing pipe buffer size. The 
> linux default size is 64K, so 'dd' command you used doesn't make much 
> sense. Please consider setting it with fcntl to max size (1M 
> according to /proc/sys/fs/pipe-max-size).


>> Mykyta Iziumtsev(MykytaI) wrote:
>> I guess only  and  are needed here.


>>> Mykyta Iziumtsev(MykytaI) wrote:
>>> Is there any reason to put all of this into separate header file ? 
>>> From what I see only one .c file is using it, so why not move it 
>>> there ?


 Mykyta Iziumtsev(MykytaI) wrote:
 IMHO, only `int pcapng_fds[PKTIO_MAX_QUEUES]` belong here, the 
 rest is superfluous or belongs to global context. Why not 
 initialize pcapng_fds[0 .. ARRAY_SIZE - 1] = -1, open the pipes 
 from inotify thread and assign fds when the header is already 
 written out ?


> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
> @apalos No, start and stop are good times for this as well. An 
> application can stop a device, reconfigure it, and then restart 
> it, so having that be the trigger for capture file cleanup and 
> close makes sense.


>> Ilias Apalodimas(apalos) wrote:
>> wireshark re-defines them in every c file they want to use them. 
>> Keeping in mind this is the pcapng file format specification, 
>> these are high unlikely to change.
>> 
>> The only relevant library i found is 
>> ntar(https://www.winpcap.org/ntar/changelog.htm) but it seems 
>> abandoned for many many years.


>>> Ilias Apalodimas(apalos) wrote:
>>> @Bill-Fischofer-Linaro pcapng_destroy() is called from 
>>> _pktio_stop() which is called from odp_pktio_term_global().
>>> pcapng_prepare() on the other hand is called from 
>>> odp_pktio_start(). Want me to move it in 
>>> odp_pktio_init_global()?


 Ilias Apalodimas(apalos) wrote:
 This essentially "reads" the inotify triggers for 
 opening/closing the fifos and sets the appropriate variables 
 for starting/stopping the pcapng dump on fifos. Will try to 
 make it easier to read on the final PR


> Ilias Apalodimas(apalos) wrote:
> will fix


>> Ilias Apalodimas(apalos) wrote:
>> will fix


>>> Ilias Apalodimas(apalos) wrote:
>>> I'll check and update for the final PR


 Bill Fischofer(Bill-Fischofer-Linaro) wrote:
 General cleanup should be part of `odp_term_global()` 
 processing. Presumably `odp_pktio_close()` should also do 
 cleanup for specific pktios.


> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
> A few comments about what this routine is doing would be 
> very he

Re: [lng-odp] [PATCH CATERPILLAR v1] [RFC] odp: pktio: added tcpdump like functionality

2018-02-09 Thread Github ODP bot
Ilias Apalodimas(apalos) replied on github web page:

platform/linux-generic/include/odp_pcapng.h
line 1
@@ -0,0 +1,56 @@
+/* Copyright (c) 2018, Linaro Limited


Comment:
too much unrelated info to the packet io. I'd prefer this reside on a different 
file

> Mykyta Iziumtsev(MykytaI) wrote:
> I believe sizeof(struct) directly in code is more informative.


>> Mykyta Iziumtsev(MykytaI) wrote:
>> I'm not sure event aggregation has any value here, but even if it does -- we 
>> don't need buffer for 1024 events. 


>>> Ilias Apalodimas(apalos) wrote:
>>> The + 16 is just a reasonable guess. The event inotify returns might or 
>>> might not have an event->name of dynamic size. If the buffer is too small, 
>>> the system call returns zero. Since it allows event slurping i am trying to 
>>> read multiple events at once.
>>> 
>>> i could change this and use FIONREAD ioctl to get the exact number of bytes 
>>> i can read from the inotify fd


 Ilias Apalodimas(apalos) wrote:
 will fix


> Ilias Apalodimas(apalos) wrote:
> will fix


>> Mykyta Iziumtsev(MykytaI) wrote:
>> Any justification for +16 and such a big buffer on a stack ? Do we 
>> really need 1024 events at once ?


>>> Mykyta Iziumtsev(MykytaI) wrote:
>>> Shall be _t


 Mykyta Iziumtsev(MykytaI) wrote:
 In fact we do care. We can't tolerate partial packet writes, otherwise 
 pcapng file will be unreadable. The proper way to address that is to 
 see how much space is left in the pipe (buf_size - TIOCOUTQ) and 
 append packets to iovec as long as they fit in the calculated 'budget'.
 
 Another thing to consider -- is increasing pipe buffer size. The linux 
 default size is 64K, so 'dd' command you used doesn't make much sense. 
 Please consider setting it with fcntl to max size (1M according to 
 /proc/sys/fs/pipe-max-size).


> Mykyta Iziumtsev(MykytaI) wrote:
> I guess only  and  are needed here.


>> Mykyta Iziumtsev(MykytaI) wrote:
>> Is there any reason to put all of this into separate header file ? 
>> From what I see only one .c file is using it, so why not move it 
>> there ?


>>> Mykyta Iziumtsev(MykytaI) wrote:
>>> IMHO, only `int pcapng_fds[PKTIO_MAX_QUEUES]` belong here, the rest 
>>> is superfluous or belongs to global context. Why not initialize 
>>> pcapng_fds[0 .. ARRAY_SIZE - 1] = -1, open the pipes from inotify 
>>> thread and assign fds when the header is already written out ?


 Bill Fischofer(Bill-Fischofer-Linaro) wrote:
 @apalos No, start and stop are good times for this as well. An 
 application can stop a device, reconfigure it, and then restart 
 it, so having that be the trigger for capture file cleanup and 
 close makes sense.


> Ilias Apalodimas(apalos) wrote:
> wireshark re-defines them in every c file they want to use them. 
> Keeping in mind this is the pcapng file format specification, 
> these are high unlikely to change.
> 
> The only relevant library i found is 
> ntar(https://www.winpcap.org/ntar/changelog.htm) but it seems 
> abandoned for many many years.


>> Ilias Apalodimas(apalos) wrote:
>> @Bill-Fischofer-Linaro pcapng_destroy() is called from 
>> _pktio_stop() which is called from odp_pktio_term_global().
>> pcapng_prepare() on the other hand is called from 
>> odp_pktio_start(). Want me to move it in odp_pktio_init_global()?


>>> Ilias Apalodimas(apalos) wrote:
>>> This essentially "reads" the inotify triggers for 
>>> opening/closing the fifos and sets the appropriate variables 
>>> for starting/stopping the pcapng dump on fifos. Will try to 
>>> make it easier to read on the final PR


 Ilias Apalodimas(apalos) wrote:
 will fix


> Ilias Apalodimas(apalos) wrote:
> will fix


>> Ilias Apalodimas(apalos) wrote:
>> I'll check and update for the final PR


>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>>> General cleanup should be part of `odp_term_global()` 
>>> processing. Presumably `odp_pktio_close()` should also do 
>>> cleanup for specific pktios.


 Bill Fischofer(Bill-Fischofer-Linaro) wrote:
 A few comments about what this routine is doing would be 
 very helpful here. This is not easy code to follow. 
 Modularizing it into some  `static inline` helpers might 
 also make the logic simpler.


> Bill Fischofer(Bill-Fischofer-Linaro) wrote:

Re: [lng-odp] [PATCH CATERPILLAR v1] [RFC] odp: pktio: added tcpdump like functionality

2018-02-07 Thread Github ODP bot
Mykyta Iziumtsev(MykytaI) replied on github web page:

platform/linux-generic/odp_packet_io.c
line 105
@@ -89,6 +96,274 @@ int odp_pktio_init_local(void)
return odp_pktio_ops_init_local(true);
 }
 
+static int write_pcapng_hdr(pktio_entry_t *entry, int qidx)
+{
+   size_t len;
+   pcapng_section_hdr_block_t shb;
+   pcapng_interface_description_block_s idb;
+   int fd = entry->s.pcapng_info.pcapng_fd[qidx];
+
+   memset(&shb, 0, sizeof(shb));
+   memset(&idb, 0, sizeof(idb));
+
+   shb.block_type = PCAPNG_BLOCK_TYPE_SHB;
+   shb.block_total_length = sizeof(shb);
+   shb.block_total_length2 = sizeof(shb);
+   shb.magic = PCAPNG_ENDIAN_MAGIC;
+   shb.version_major = 0x1;
+   shb.version_minor = 0x0;
+   shb.section_len = -1;
+
+   len = write(fd, &shb, sizeof(shb));
+   fsync(fd);
+   /* fail to write shb/idb means the pcapng is unreadable */
+   if (!len) {
+   ODP_ERR("Failed to write pcapng section hdr\n");
+   close(fd);
+   return -1;
+   }
+
+   idb.block_type = PCAPNG_BLOCK_TYPE_IDB;
+   idb.block_total_length = sizeof(idb);
+   idb.block_total_length2 = sizeof(idb);
+   idb.linktype = 0x1; /* LINKTYPE_ETHERNET */
+   idb.snaplen = 0x0; /* unlimited */
+   len = write(fd, &idb, sizeof(idb));
+   if (!len) {
+   ODP_ERR("Failed to write pcapng interface description\n");
+   close(fd);
+   return -1;
+   }
+   fsync(fd);
+
+   entry->s.pcapng_info.state[qidx] = PCAPNG_WR_HDR;
+
+   return 0;
+}
+
+static int write_pcapng_pkts(pktio_entry_t *entry, int qidx,
+const odp_packet_t packets[], int num)
+{
+   int i = 0;
+   struct iovec packet_iov[3 * num];
+   pcapng_enhanced_packet_block_t epb;
+   int iovcnt = 0;
+   ssize_t len = 0;
+   int fd = entry->s.pcapng_info.pcapng_fd[qidx];
+
+   for (i = 0; i < num; i++) {
+   odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(packets[i]);
+   uint32_t pkt_len = _odp_packet_len(packets[i]);
+   char *buf = (char *)odp_packet_data(packets[i]);
+
+   epb.block_type = PCAPNG_BLOCK_TYPE_EPB;
+   epb.block_total_length = sizeof(epb) +
+   ROUNDUP_ALIGN(pkt_len, PCAP_DATA_ALIGN) +
+   PCAP_DATA_ALIGN;
+   epb.interface_idx = 0;
+   epb.timestamp_high = (uint32_t)(pkt_hdr->timestamp.u64 >> 32);
+   epb.timestamp_low = (uint32_t)(pkt_hdr->timestamp.u64);
+   epb.captured_len = pkt_len;
+   epb.packet_len = pkt_len;
+
+   /* epb */
+   packet_iov[iovcnt].iov_base = &epb;
+   packet_iov[iovcnt].iov_len = sizeof(epb);
+   iovcnt++;
+
+   /* data */
+   packet_iov[iovcnt].iov_base = buf;
+   packet_iov[iovcnt].iov_len =
+   ROUNDUP_ALIGN(pkt_len, PCAP_DATA_ALIGN);
+   iovcnt++;
+
+   /* trailing */
+   packet_iov[iovcnt].iov_base = &epb.block_total_length;
+   packet_iov[iovcnt].iov_len = sizeof(uint32_t);
+   iovcnt++;
+   }
+
+   /* we don't really care if we manage to write *all* data */


Comment:
In fact we do care. We can't tolerate partial packet writes, otherwise pcapng 
file will be unreadable. The proper way to address that is to see how much 
space is left in the pipe (buf_size - TIOCOUTQ) and append packets to iovec as 
long as they fit in the calculated 'budget'.

Another thing to consider -- is increasing pipe buffer size. The linux default 
size is 64K, so 'dd' command you used doesn't make much sense. Please consider 
setting it with fcntl to max size (1M according to /proc/sys/fs/pipe-max-size).

> Mykyta Iziumtsev(MykytaI) wrote:
> I guess only  and  are needed here.


>> Mykyta Iziumtsev(MykytaI) wrote:
>> Is there any reason to put all of this into separate header file ? From what 
>> I see only one .c file is using it, so why not move it there ?


>>> Mykyta Iziumtsev(MykytaI) wrote:
>>> IMHO, only `int pcapng_fds[PKTIO_MAX_QUEUES]` belong here, the rest is 
>>> superfluous or belongs to global context. Why not initialize pcapng_fds[0 
>>> .. ARRAY_SIZE - 1] = -1, open the pipes from inotify thread and assign fds 
>>> when the header is already written out ?


 Bill Fischofer(Bill-Fischofer-Linaro) wrote:
 @apalos No, start and stop are good times for this as well. An application 
 can stop a device, reconfigure it, and then restart it, so having that be 
 the trigger for capture file cleanup and close makes sense.


> Ilias Apalodimas(apalos) wrote:
> wireshark re-defines them in every c file they want to use them. Keeping 
> in mind this is the pcapng file format specification, these are high 
> unlikely to change.
> 
> The only relevant library i

Re: [lng-odp] [PATCH CATERPILLAR v1] [RFC] odp: pktio: added tcpdump like functionality

2018-02-07 Thread Github ODP bot
Mykyta Iziumtsev(MykytaI) replied on github web page:

platform/linux-generic/include/odp_pcapng.h
line 23
@@ -0,0 +1,56 @@
+/* Copyright (c) 2018, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#define PCAPNG_BLOCK_TYPE_EPB 0x0006UL
+#define PCAPNG_BLOCK_TYPE_SHB 0x0A0D0D0AUL
+#define PCAPNG_BLOCK_TYPE_IDB 0x0001UL
+#define PCAPNG_ENDIAN_MAGIC 0x1A2B3C4DUL
+#define PCAP_DATA_ALIGN (4)
+
+/* inotify */
+#define INOTIFY_EVENT_SIZE  (sizeof(struct inotify_event))


Comment:
I believe sizeof(struct) directly in code is more informative.

> Mykyta Iziumtsev(MykytaI) wrote:
> I'm not sure event aggregation has any value here, but even if it does -- we 
> don't need buffer for 1024 events. 


>> Ilias Apalodimas(apalos) wrote:
>> The + 16 is just a reasonable guess. The event inotify returns might or 
>> might not have an event->name of dynamic size. If the buffer is too small, 
>> the system call returns zero. Since it allows event slurping i am trying to 
>> read multiple events at once.
>> 
>> i could change this and use FIONREAD ioctl to get the exact number of bytes 
>> i can read from the inotify fd


>>> Ilias Apalodimas(apalos) wrote:
>>> will fix


 Ilias Apalodimas(apalos) wrote:
 will fix


> Mykyta Iziumtsev(MykytaI) wrote:
> Any justification for +16 and such a big buffer on a stack ? Do we really 
> need 1024 events at once ?


>> Mykyta Iziumtsev(MykytaI) wrote:
>> Shall be _t


>>> Mykyta Iziumtsev(MykytaI) wrote:
>>> In fact we do care. We can't tolerate partial packet writes, otherwise 
>>> pcapng file will be unreadable. The proper way to address that is to 
>>> see how much space is left in the pipe (buf_size - TIOCOUTQ) and append 
>>> packets to iovec as long as they fit in the calculated 'budget'.
>>> 
>>> Another thing to consider -- is increasing pipe buffer size. The linux 
>>> default size is 64K, so 'dd' command you used doesn't make much sense. 
>>> Please consider setting it with fcntl to max size (1M according to 
>>> /proc/sys/fs/pipe-max-size).


 Mykyta Iziumtsev(MykytaI) wrote:
 I guess only  and  are needed here.


> Mykyta Iziumtsev(MykytaI) wrote:
> Is there any reason to put all of this into separate header file ? 
> From what I see only one .c file is using it, so why not move it 
> there ?


>> Mykyta Iziumtsev(MykytaI) wrote:
>> IMHO, only `int pcapng_fds[PKTIO_MAX_QUEUES]` belong here, the rest 
>> is superfluous or belongs to global context. Why not initialize 
>> pcapng_fds[0 .. ARRAY_SIZE - 1] = -1, open the pipes from inotify 
>> thread and assign fds when the header is already written out ?


>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>>> @apalos No, start and stop are good times for this as well. An 
>>> application can stop a device, reconfigure it, and then restart it, 
>>> so having that be the trigger for capture file cleanup and close 
>>> makes sense.


 Ilias Apalodimas(apalos) wrote:
 wireshark re-defines them in every c file they want to use them. 
 Keeping in mind this is the pcapng file format specification, 
 these are high unlikely to change.
 
 The only relevant library i found is 
 ntar(https://www.winpcap.org/ntar/changelog.htm) but it seems 
 abandoned for many many years.


> Ilias Apalodimas(apalos) wrote:
> @Bill-Fischofer-Linaro pcapng_destroy() is called from 
> _pktio_stop() which is called from odp_pktio_term_global().
> pcapng_prepare() on the other hand is called from 
> odp_pktio_start(). Want me to move it in odp_pktio_init_global()?


>> Ilias Apalodimas(apalos) wrote:
>> This essentially "reads" the inotify triggers for 
>> opening/closing the fifos and sets the appropriate variables for 
>> starting/stopping the pcapng dump on fifos. Will try to make it 
>> easier to read on the final PR


>>> Ilias Apalodimas(apalos) wrote:
>>> will fix


 Ilias Apalodimas(apalos) wrote:
 will fix


> Ilias Apalodimas(apalos) wrote:
> I'll check and update for the final PR


>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>> General cleanup should be part of `odp_term_global()` 
>> processing. Presumably `odp_pktio_close()` should also do 
>> cleanup for specific pktios.


>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>>> A few comments about what this routine is doing would be 
>>> very helpfu

Re: [lng-odp] [PATCH CATERPILLAR v1] [RFC] odp: pktio: added tcpdump like functionality

2018-02-07 Thread Github ODP bot
Mykyta Iziumtsev(MykytaI) replied on github web page:

platform/linux-generic/include/odp_pcapng.h
line 24
@@ -0,0 +1,56 @@
+/* Copyright (c) 2018, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#define PCAPNG_BLOCK_TYPE_EPB 0x0006UL
+#define PCAPNG_BLOCK_TYPE_SHB 0x0A0D0D0AUL
+#define PCAPNG_BLOCK_TYPE_IDB 0x0001UL
+#define PCAPNG_ENDIAN_MAGIC 0x1A2B3C4DUL
+#define PCAP_DATA_ALIGN (4)
+
+/* inotify */
+#define INOTIFY_EVENT_SIZE  (sizeof(struct inotify_event))
+#define INOTIFY_BUF_LEN (1024 * (INOTIFY_EVENT_SIZE + 16))


Comment:
I'm not sure event aggregation has any value here, but even if it does -- we 
don't need buffer for 1024 events. 

> Ilias Apalodimas(apalos) wrote:
> The + 16 is just a reasonable guess. The event inotify returns might or might 
> not have an event->name of dynamic size. If the buffer is too small, the 
> system call returns zero. Since it allows event slurping i am trying to read 
> multiple events at once.
> 
> i could change this and use FIONREAD ioctl to get the exact number of bytes i 
> can read from the inotify fd


>> Ilias Apalodimas(apalos) wrote:
>> will fix


>>> Ilias Apalodimas(apalos) wrote:
>>> will fix


 Mykyta Iziumtsev(MykytaI) wrote:
 Any justification for +16 and such a big buffer on a stack ? Do we really 
 need 1024 events at once ?


> Mykyta Iziumtsev(MykytaI) wrote:
> Shall be _t


>> Mykyta Iziumtsev(MykytaI) wrote:
>> In fact we do care. We can't tolerate partial packet writes, otherwise 
>> pcapng file will be unreadable. The proper way to address that is to see 
>> how much space is left in the pipe (buf_size - TIOCOUTQ) and append 
>> packets to iovec as long as they fit in the calculated 'budget'.
>> 
>> Another thing to consider -- is increasing pipe buffer size. The linux 
>> default size is 64K, so 'dd' command you used doesn't make much sense. 
>> Please consider setting it with fcntl to max size (1M according to 
>> /proc/sys/fs/pipe-max-size).


>>> Mykyta Iziumtsev(MykytaI) wrote:
>>> I guess only  and  are needed here.


 Mykyta Iziumtsev(MykytaI) wrote:
 Is there any reason to put all of this into separate header file ? 
 From what I see only one .c file is using it, so why not move it there 
 ?


> Mykyta Iziumtsev(MykytaI) wrote:
> IMHO, only `int pcapng_fds[PKTIO_MAX_QUEUES]` belong here, the rest 
> is superfluous or belongs to global context. Why not initialize 
> pcapng_fds[0 .. ARRAY_SIZE - 1] = -1, open the pipes from inotify 
> thread and assign fds when the header is already written out ?


>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>> @apalos No, start and stop are good times for this as well. An 
>> application can stop a device, reconfigure it, and then restart it, 
>> so having that be the trigger for capture file cleanup and close 
>> makes sense.


>>> Ilias Apalodimas(apalos) wrote:
>>> wireshark re-defines them in every c file they want to use them. 
>>> Keeping in mind this is the pcapng file format specification, these 
>>> are high unlikely to change.
>>> 
>>> The only relevant library i found is 
>>> ntar(https://www.winpcap.org/ntar/changelog.htm) but it seems 
>>> abandoned for many many years.


 Ilias Apalodimas(apalos) wrote:
 @Bill-Fischofer-Linaro pcapng_destroy() is called from 
 _pktio_stop() which is called from odp_pktio_term_global().
 pcapng_prepare() on the other hand is called from 
 odp_pktio_start(). Want me to move it in odp_pktio_init_global()?


> Ilias Apalodimas(apalos) wrote:
> This essentially "reads" the inotify triggers for opening/closing 
> the fifos and sets the appropriate variables for 
> starting/stopping the pcapng dump on fifos. Will try to make it 
> easier to read on the final PR


>> Ilias Apalodimas(apalos) wrote:
>> will fix


>>> Ilias Apalodimas(apalos) wrote:
>>> will fix


 Ilias Apalodimas(apalos) wrote:
 I'll check and update for the final PR


> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
> General cleanup should be part of `odp_term_global()` 
> processing. Presumably `odp_pktio_close()` should also do 
> cleanup for specific pktios.


>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>> A few comments about what this routine is doing would be 
>> very helpful here. This is not easy code to follow. 
>> Modularizing it into some  `static inline` helpers mi

Re: [lng-odp] [PATCH CATERPILLAR v1] [RFC] odp: pktio: added tcpdump like functionality

2018-02-07 Thread Github ODP bot
Ilias Apalodimas(apalos) replied on github web page:

platform/linux-generic/include/odp_packet_io_internal.h
line 18
@@ -97,6 +97,20 @@ struct pktio_entry {
odp_queue_tqueue;
odp_pktout_queue_t pktout;
} out_queue[PKTIO_MAX_QUEUES];
+
+   /**< inotify instance for pcapng fifos */
+   struct {
+   int inotify_pcapng_fd;
+   int inotify_watch_fd;
+   pthread_t inotify_thread;
+   enum {
+   PCAPNG_WR_INVALID = 0,
+   PCAPNG_WR_STOP,
+   PCAPNG_WR_HDR,
+   PCAPNG_WR_PKT,
+   } state[PKTIO_MAX_QUEUES];
+   int pcapng_fd[PKTIO_MAX_QUEUES];
+   } pcapng_info;
 };


Comment:
will fix

> Ilias Apalodimas(apalos) wrote:
> will fix


>> Mykyta Iziumtsev(MykytaI) wrote:
>> Any justification for +16 and such a big buffer on a stack ? Do we really 
>> need 1024 events at once ?


>>> Mykyta Iziumtsev(MykytaI) wrote:
>>> Shall be _t


 Mykyta Iziumtsev(MykytaI) wrote:
 In fact we do care. We can't tolerate partial packet writes, otherwise 
 pcapng file will be unreadable. The proper way to address that is to see 
 how much space is left in the pipe (buf_size - TIOCOUTQ) and append 
 packets to iovec as long as they fit in the calculated 'budget'.
 
 Another thing to consider -- is increasing pipe buffer size. The linux 
 default size is 64K, so 'dd' command you used doesn't make much sense. 
 Please consider setting it with fcntl to max size (1M according to 
 /proc/sys/fs/pipe-max-size).


> Mykyta Iziumtsev(MykytaI) wrote:
> I guess only  and  are needed here.


>> Mykyta Iziumtsev(MykytaI) wrote:
>> Is there any reason to put all of this into separate header file ? From 
>> what I see only one .c file is using it, so why not move it there ?


>>> Mykyta Iziumtsev(MykytaI) wrote:
>>> IMHO, only `int pcapng_fds[PKTIO_MAX_QUEUES]` belong here, the rest is 
>>> superfluous or belongs to global context. Why not initialize 
>>> pcapng_fds[0 .. ARRAY_SIZE - 1] = -1, open the pipes from inotify 
>>> thread and assign fds when the header is already written out ?


 Bill Fischofer(Bill-Fischofer-Linaro) wrote:
 @apalos No, start and stop are good times for this as well. An 
 application can stop a device, reconfigure it, and then restart it, so 
 having that be the trigger for capture file cleanup and close makes 
 sense.


> Ilias Apalodimas(apalos) wrote:
> wireshark re-defines them in every c file they want to use them. 
> Keeping in mind this is the pcapng file format specification, these 
> are high unlikely to change.
> 
> The only relevant library i found is 
> ntar(https://www.winpcap.org/ntar/changelog.htm) but it seems 
> abandoned for many many years.


>> Ilias Apalodimas(apalos) wrote:
>> @Bill-Fischofer-Linaro pcapng_destroy() is called from _pktio_stop() 
>> which is called from odp_pktio_term_global().
>> pcapng_prepare() on the other hand is called from odp_pktio_start(). 
>> Want me to move it in odp_pktio_init_global()?


>>> Ilias Apalodimas(apalos) wrote:
>>> This essentially "reads" the inotify triggers for opening/closing 
>>> the fifos and sets the appropriate variables for starting/stopping 
>>> the pcapng dump on fifos. Will try to make it easier to read on the 
>>> final PR


 Ilias Apalodimas(apalos) wrote:
 will fix


> Ilias Apalodimas(apalos) wrote:
> will fix


>> Ilias Apalodimas(apalos) wrote:
>> I'll check and update for the final PR


>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>>> General cleanup should be part of `odp_term_global()` 
>>> processing. Presumably `odp_pktio_close()` should also do 
>>> cleanup for specific pktios.


 Bill Fischofer(Bill-Fischofer-Linaro) wrote:
 A few comments about what this routine is doing would be very 
 helpful here. This is not easy code to follow. Modularizing it 
 into some  `static inline` helpers might also make the logic 
 simpler.


> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
> This is redundant. If `len > 0` then by definition `len != 
> -1`.


>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>> This fails for multi-segment packets. `odp_packet_data(pkt)` 
>> is shorthand for `odp_packet_offset(pkt, 0, NULL, NULL)` so 
>> you don't know how long the first segment really is. First 
>> cut would be to only dump the first segment, so correct code 
>

Re: [lng-odp] [PATCH CATERPILLAR v1] [RFC] odp: pktio: added tcpdump like functionality

2018-02-07 Thread Github ODP bot
Ilias Apalodimas(apalos) replied on github web page:

platform/linux-generic/include/odp_pcapng.h
line 24
@@ -0,0 +1,56 @@
+/* Copyright (c) 2018, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#define PCAPNG_BLOCK_TYPE_EPB 0x0006UL
+#define PCAPNG_BLOCK_TYPE_SHB 0x0A0D0D0AUL
+#define PCAPNG_BLOCK_TYPE_IDB 0x0001UL
+#define PCAPNG_ENDIAN_MAGIC 0x1A2B3C4DUL
+#define PCAP_DATA_ALIGN (4)
+
+/* inotify */
+#define INOTIFY_EVENT_SIZE  (sizeof(struct inotify_event))
+#define INOTIFY_BUF_LEN (1024 * (INOTIFY_EVENT_SIZE + 16))


Comment:
The + 16 is just a reasonable guess. The event inotify returns might or might 
not have an event->name of dynamic size. If the buffer is too small, the system 
call returns zero. Since it allows event slurping i am trying to read multiple 
events at once.

i could change this and use FIONREAD ioctl to get the exact number of bytes i 
can read from the inotify fd

> Ilias Apalodimas(apalos) wrote:
> will fix


>> Ilias Apalodimas(apalos) wrote:
>> will fix


>>> Mykyta Iziumtsev(MykytaI) wrote:
>>> Any justification for +16 and such a big buffer on a stack ? Do we really 
>>> need 1024 events at once ?


 Mykyta Iziumtsev(MykytaI) wrote:
 Shall be _t


> Mykyta Iziumtsev(MykytaI) wrote:
> In fact we do care. We can't tolerate partial packet writes, otherwise 
> pcapng file will be unreadable. The proper way to address that is to see 
> how much space is left in the pipe (buf_size - TIOCOUTQ) and append 
> packets to iovec as long as they fit in the calculated 'budget'.
> 
> Another thing to consider -- is increasing pipe buffer size. The linux 
> default size is 64K, so 'dd' command you used doesn't make much sense. 
> Please consider setting it with fcntl to max size (1M according to 
> /proc/sys/fs/pipe-max-size).


>> Mykyta Iziumtsev(MykytaI) wrote:
>> I guess only  and  are needed here.


>>> Mykyta Iziumtsev(MykytaI) wrote:
>>> Is there any reason to put all of this into separate header file ? From 
>>> what I see only one .c file is using it, so why not move it there ?


 Mykyta Iziumtsev(MykytaI) wrote:
 IMHO, only `int pcapng_fds[PKTIO_MAX_QUEUES]` belong here, the rest is 
 superfluous or belongs to global context. Why not initialize 
 pcapng_fds[0 .. ARRAY_SIZE - 1] = -1, open the pipes from inotify 
 thread and assign fds when the header is already written out ?


> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
> @apalos No, start and stop are good times for this as well. An 
> application can stop a device, reconfigure it, and then restart it, 
> so having that be the trigger for capture file cleanup and close 
> makes sense.


>> Ilias Apalodimas(apalos) wrote:
>> wireshark re-defines them in every c file they want to use them. 
>> Keeping in mind this is the pcapng file format specification, these 
>> are high unlikely to change.
>> 
>> The only relevant library i found is 
>> ntar(https://www.winpcap.org/ntar/changelog.htm) but it seems 
>> abandoned for many many years.


>>> Ilias Apalodimas(apalos) wrote:
>>> @Bill-Fischofer-Linaro pcapng_destroy() is called from 
>>> _pktio_stop() which is called from odp_pktio_term_global().
>>> pcapng_prepare() on the other hand is called from 
>>> odp_pktio_start(). Want me to move it in odp_pktio_init_global()?


 Ilias Apalodimas(apalos) wrote:
 This essentially "reads" the inotify triggers for opening/closing 
 the fifos and sets the appropriate variables for starting/stopping 
 the pcapng dump on fifos. Will try to make it easier to read on 
 the final PR


> Ilias Apalodimas(apalos) wrote:
> will fix


>> Ilias Apalodimas(apalos) wrote:
>> will fix


>>> Ilias Apalodimas(apalos) wrote:
>>> I'll check and update for the final PR


 Bill Fischofer(Bill-Fischofer-Linaro) wrote:
 General cleanup should be part of `odp_term_global()` 
 processing. Presumably `odp_pktio_close()` should also do 
 cleanup for specific pktios.


> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
> A few comments about what this routine is doing would be very 
> helpful here. This is not easy code to follow. Modularizing 
> it into some  `static inline` helpers might also make the 
> logic simpler.


>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>> This is redundant. If `len > 0` then by definition `len != 
>> -1`.


>>

Re: [lng-odp] [PATCH CATERPILLAR v1] [RFC] odp: pktio: added tcpdump like functionality

2018-02-07 Thread Github ODP bot
Mykyta Iziumtsev(MykytaI) replied on github web page:

platform/linux-generic/include/odp_pcapng.h
line 45
@@ -0,0 +1,56 @@
+/* Copyright (c) 2018, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#define PCAPNG_BLOCK_TYPE_EPB 0x0006UL
+#define PCAPNG_BLOCK_TYPE_SHB 0x0A0D0D0AUL
+#define PCAPNG_BLOCK_TYPE_IDB 0x0001UL
+#define PCAPNG_ENDIAN_MAGIC 0x1A2B3C4DUL
+#define PCAP_DATA_ALIGN (4)
+
+/* inotify */
+#define INOTIFY_EVENT_SIZE  (sizeof(struct inotify_event))
+#define INOTIFY_BUF_LEN (1024 * (INOTIFY_EVENT_SIZE + 16))
+#define PCAPNG_WATCH_DIR "/var/run/odp/"
+
+/* pcapng: enhanced packet block file encoding */
+typedef struct ODP_PACKED pcapng_section_hdr_block_s {
+   uint32_t block_type;
+   uint32_t block_total_length;
+   uint32_t magic;
+   uint16_t version_major;
+   uint16_t version_minor;
+   int64_t section_len;
+   uint32_t block_total_length2;
+} pcapng_section_hdr_block_t;
+
+typedef struct pcapng_interface_description_block {
+   uint32_t block_type;
+   uint32_t block_total_length;
+   uint16_t linktype;
+   uint16_t reserved;
+   uint32_t snaplen;
+   uint32_t block_total_length2;
+} pcapng_interface_description_block_s;


Comment:
Shall be _t

> Mykyta Iziumtsev(MykytaI) wrote:
> In fact we do care. We can't tolerate partial packet writes, otherwise pcapng 
> file will be unreadable. The proper way to address that is to see how much 
> space is left in the pipe (buf_size - TIOCOUTQ) and append packets to iovec 
> as long as they fit in the calculated 'budget'.
> 
> Another thing to consider -- is increasing pipe buffer size. The linux 
> default size is 64K, so 'dd' command you used doesn't make much sense. Please 
> consider setting it with fcntl to max size (1M according to 
> /proc/sys/fs/pipe-max-size).


>> Mykyta Iziumtsev(MykytaI) wrote:
>> I guess only  and  are needed here.


>>> Mykyta Iziumtsev(MykytaI) wrote:
>>> Is there any reason to put all of this into separate header file ? From 
>>> what I see only one .c file is using it, so why not move it there ?


 Mykyta Iziumtsev(MykytaI) wrote:
 IMHO, only `int pcapng_fds[PKTIO_MAX_QUEUES]` belong here, the rest is 
 superfluous or belongs to global context. Why not initialize pcapng_fds[0 
 .. ARRAY_SIZE - 1] = -1, open the pipes from inotify thread and assign fds 
 when the header is already written out ?


> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
> @apalos No, start and stop are good times for this as well. An 
> application can stop a device, reconfigure it, and then restart it, so 
> having that be the trigger for capture file cleanup and close makes sense.


>> Ilias Apalodimas(apalos) wrote:
>> wireshark re-defines them in every c file they want to use them. Keeping 
>> in mind this is the pcapng file format specification, these are high 
>> unlikely to change.
>> 
>> The only relevant library i found is 
>> ntar(https://www.winpcap.org/ntar/changelog.htm) but it seems abandoned 
>> for many many years.


>>> Ilias Apalodimas(apalos) wrote:
>>> @Bill-Fischofer-Linaro pcapng_destroy() is called from _pktio_stop() 
>>> which is called from odp_pktio_term_global().
>>> pcapng_prepare() on the other hand is called from odp_pktio_start(). 
>>> Want me to move it in odp_pktio_init_global()?


 Ilias Apalodimas(apalos) wrote:
 This essentially "reads" the inotify triggers for opening/closing the 
 fifos and sets the appropriate variables for starting/stopping the 
 pcapng dump on fifos. Will try to make it easier to read on the final 
 PR


> Ilias Apalodimas(apalos) wrote:
> will fix


>> Ilias Apalodimas(apalos) wrote:
>> will fix


>>> Ilias Apalodimas(apalos) wrote:
>>> I'll check and update for the final PR


 Bill Fischofer(Bill-Fischofer-Linaro) wrote:
 General cleanup should be part of `odp_term_global()` processing. 
 Presumably `odp_pktio_close()` should also do cleanup for specific 
 pktios.


> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
> A few comments about what this routine is doing would be very 
> helpful here. This is not easy code to follow. Modularizing it 
> into some  `static inline` helpers might also make the logic 
> simpler.


>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>> This is redundant. If `len > 0` then by definition `len != -1`.


>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>>> This fails for multi-segment packets. `odp_packet_data(pkt)` is 
>>> shorthand for `odp_packet_offset(pkt, 0, NULL, NULL)` so you 
>>> don't know how long 

Re: [lng-odp] [PATCH CATERPILLAR v1] [RFC] odp: pktio: added tcpdump like functionality

2018-02-07 Thread Github ODP bot
Ilias Apalodimas(apalos) replied on github web page:

platform/linux-generic/include/odp_pcapng.h
line 45
@@ -0,0 +1,56 @@
+/* Copyright (c) 2018, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#define PCAPNG_BLOCK_TYPE_EPB 0x0006UL
+#define PCAPNG_BLOCK_TYPE_SHB 0x0A0D0D0AUL
+#define PCAPNG_BLOCK_TYPE_IDB 0x0001UL
+#define PCAPNG_ENDIAN_MAGIC 0x1A2B3C4DUL
+#define PCAP_DATA_ALIGN (4)
+
+/* inotify */
+#define INOTIFY_EVENT_SIZE  (sizeof(struct inotify_event))
+#define INOTIFY_BUF_LEN (1024 * (INOTIFY_EVENT_SIZE + 16))
+#define PCAPNG_WATCH_DIR "/var/run/odp/"
+
+/* pcapng: enhanced packet block file encoding */
+typedef struct ODP_PACKED pcapng_section_hdr_block_s {
+   uint32_t block_type;
+   uint32_t block_total_length;
+   uint32_t magic;
+   uint16_t version_major;
+   uint16_t version_minor;
+   int64_t section_len;
+   uint32_t block_total_length2;
+} pcapng_section_hdr_block_t;
+
+typedef struct pcapng_interface_description_block {
+   uint32_t block_type;
+   uint32_t block_total_length;
+   uint16_t linktype;
+   uint16_t reserved;
+   uint32_t snaplen;
+   uint32_t block_total_length2;
+} pcapng_interface_description_block_s;


Comment:
will fix

> Mykyta Iziumtsev(MykytaI) wrote:
> Any justification for +16 and such a big buffer on a stack ? Do we really 
> need 1024 events at once ?


>> Mykyta Iziumtsev(MykytaI) wrote:
>> Shall be _t


>>> Mykyta Iziumtsev(MykytaI) wrote:
>>> In fact we do care. We can't tolerate partial packet writes, otherwise 
>>> pcapng file will be unreadable. The proper way to address that is to see 
>>> how much space is left in the pipe (buf_size - TIOCOUTQ) and append packets 
>>> to iovec as long as they fit in the calculated 'budget'.
>>> 
>>> Another thing to consider -- is increasing pipe buffer size. The linux 
>>> default size is 64K, so 'dd' command you used doesn't make much sense. 
>>> Please consider setting it with fcntl to max size (1M according to 
>>> /proc/sys/fs/pipe-max-size).


 Mykyta Iziumtsev(MykytaI) wrote:
 I guess only  and  are needed here.


> Mykyta Iziumtsev(MykytaI) wrote:
> Is there any reason to put all of this into separate header file ? From 
> what I see only one .c file is using it, so why not move it there ?


>> Mykyta Iziumtsev(MykytaI) wrote:
>> IMHO, only `int pcapng_fds[PKTIO_MAX_QUEUES]` belong here, the rest is 
>> superfluous or belongs to global context. Why not initialize 
>> pcapng_fds[0 .. ARRAY_SIZE - 1] = -1, open the pipes from inotify thread 
>> and assign fds when the header is already written out ?


>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>>> @apalos No, start and stop are good times for this as well. An 
>>> application can stop a device, reconfigure it, and then restart it, so 
>>> having that be the trigger for capture file cleanup and close makes 
>>> sense.


 Ilias Apalodimas(apalos) wrote:
 wireshark re-defines them in every c file they want to use them. 
 Keeping in mind this is the pcapng file format specification, these 
 are high unlikely to change.
 
 The only relevant library i found is 
 ntar(https://www.winpcap.org/ntar/changelog.htm) but it seems 
 abandoned for many many years.


> Ilias Apalodimas(apalos) wrote:
> @Bill-Fischofer-Linaro pcapng_destroy() is called from _pktio_stop() 
> which is called from odp_pktio_term_global().
> pcapng_prepare() on the other hand is called from odp_pktio_start(). 
> Want me to move it in odp_pktio_init_global()?


>> Ilias Apalodimas(apalos) wrote:
>> This essentially "reads" the inotify triggers for opening/closing 
>> the fifos and sets the appropriate variables for starting/stopping 
>> the pcapng dump on fifos. Will try to make it easier to read on the 
>> final PR


>>> Ilias Apalodimas(apalos) wrote:
>>> will fix


 Ilias Apalodimas(apalos) wrote:
 will fix


> Ilias Apalodimas(apalos) wrote:
> I'll check and update for the final PR


>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>> General cleanup should be part of `odp_term_global()` 
>> processing. Presumably `odp_pktio_close()` should also do 
>> cleanup for specific pktios.


>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>>> A few comments about what this routine is doing would be very 
>>> helpful here. This is not easy code to follow. Modularizing it 
>>> into some  `static inline` helpers might also make the logic 
>>> simpler.


 Bill Fischofer(Bill-Fischofer-Linaro) wrote:
 Thi

Re: [lng-odp] [PATCH CATERPILLAR v1] [RFC] odp: pktio: added tcpdump like functionality

2018-02-07 Thread Github ODP bot
Mykyta Iziumtsev(MykytaI) replied on github web page:

platform/linux-generic/include/odp_pcapng.h
line 24
@@ -0,0 +1,56 @@
+/* Copyright (c) 2018, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#define PCAPNG_BLOCK_TYPE_EPB 0x0006UL
+#define PCAPNG_BLOCK_TYPE_SHB 0x0A0D0D0AUL
+#define PCAPNG_BLOCK_TYPE_IDB 0x0001UL
+#define PCAPNG_ENDIAN_MAGIC 0x1A2B3C4DUL
+#define PCAP_DATA_ALIGN (4)
+
+/* inotify */
+#define INOTIFY_EVENT_SIZE  (sizeof(struct inotify_event))
+#define INOTIFY_BUF_LEN (1024 * (INOTIFY_EVENT_SIZE + 16))


Comment:
Any justification for +16 and such a big buffer on a stack ? Do we really need 
1024 events at once ?

> Mykyta Iziumtsev(MykytaI) wrote:
> Shall be _t


>> Mykyta Iziumtsev(MykytaI) wrote:
>> In fact we do care. We can't tolerate partial packet writes, otherwise 
>> pcapng file will be unreadable. The proper way to address that is to see how 
>> much space is left in the pipe (buf_size - TIOCOUTQ) and append packets to 
>> iovec as long as they fit in the calculated 'budget'.
>> 
>> Another thing to consider -- is increasing pipe buffer size. The linux 
>> default size is 64K, so 'dd' command you used doesn't make much sense. 
>> Please consider setting it with fcntl to max size (1M according to 
>> /proc/sys/fs/pipe-max-size).


>>> Mykyta Iziumtsev(MykytaI) wrote:
>>> I guess only  and  are needed here.


 Mykyta Iziumtsev(MykytaI) wrote:
 Is there any reason to put all of this into separate header file ? From 
 what I see only one .c file is using it, so why not move it there ?


> Mykyta Iziumtsev(MykytaI) wrote:
> IMHO, only `int pcapng_fds[PKTIO_MAX_QUEUES]` belong here, the rest is 
> superfluous or belongs to global context. Why not initialize pcapng_fds[0 
> .. ARRAY_SIZE - 1] = -1, open the pipes from inotify thread and assign 
> fds when the header is already written out ?


>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>> @apalos No, start and stop are good times for this as well. An 
>> application can stop a device, reconfigure it, and then restart it, so 
>> having that be the trigger for capture file cleanup and close makes 
>> sense.


>>> Ilias Apalodimas(apalos) wrote:
>>> wireshark re-defines them in every c file they want to use them. 
>>> Keeping in mind this is the pcapng file format specification, these are 
>>> high unlikely to change.
>>> 
>>> The only relevant library i found is 
>>> ntar(https://www.winpcap.org/ntar/changelog.htm) but it seems abandoned 
>>> for many many years.


 Ilias Apalodimas(apalos) wrote:
 @Bill-Fischofer-Linaro pcapng_destroy() is called from _pktio_stop() 
 which is called from odp_pktio_term_global().
 pcapng_prepare() on the other hand is called from odp_pktio_start(). 
 Want me to move it in odp_pktio_init_global()?


> Ilias Apalodimas(apalos) wrote:
> This essentially "reads" the inotify triggers for opening/closing the 
> fifos and sets the appropriate variables for starting/stopping the 
> pcapng dump on fifos. Will try to make it easier to read on the final 
> PR


>> Ilias Apalodimas(apalos) wrote:
>> will fix


>>> Ilias Apalodimas(apalos) wrote:
>>> will fix


 Ilias Apalodimas(apalos) wrote:
 I'll check and update for the final PR


> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
> General cleanup should be part of `odp_term_global()` processing. 
> Presumably `odp_pktio_close()` should also do cleanup for 
> specific pktios.


>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>> A few comments about what this routine is doing would be very 
>> helpful here. This is not easy code to follow. Modularizing it 
>> into some  `static inline` helpers might also make the logic 
>> simpler.


>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>>> This is redundant. If `len > 0` then by definition `len != -1`.


 Bill Fischofer(Bill-Fischofer-Linaro) wrote:
 This fails for multi-segment packets. `odp_packet_data(pkt)` 
 is shorthand for `odp_packet_offset(pkt, 0, NULL, NULL)` so 
 you don't know how long the first segment really is. First cut 
 would be to only dump the first segment, so correct code is:
 ```
 uint32_t seg_len;
 char *buf = (char *)odp_packet_offset(packets[i], 0, &seg_len, 
 NULL);
 ```
 And use `seg_len` rather than `pkt_len` in this routine.


> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
> Does pcapng no

Re: [lng-odp] [PATCH CATERPILLAR v1] [RFC] odp: pktio: added tcpdump like functionality

2018-02-07 Thread Github ODP bot
Mykyta Iziumtsev(MykytaI) replied on github web page:

platform/linux-generic/include/odp_pcapng.h
line 14
@@ -0,0 +1,56 @@
+/* Copyright (c) 2018, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 


Comment:
I guess only  and  are needed here.

> Mykyta Iziumtsev(MykytaI) wrote:
> Is there any reason to put all of this into separate header file ? From what 
> I see only one .c file is using it, so why not move it there ?


>> Mykyta Iziumtsev(MykytaI) wrote:
>> IMHO, only `int pcapng_fds[PKTIO_MAX_QUEUES]` belong here, the rest is 
>> superfluous or belongs to global context. Why not initialize pcapng_fds[0 .. 
>> ARRAY_SIZE - 1] = -1, open the pipes from inotify thread and assign fds when 
>> the header is already written out ?


>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>>> @apalos No, start and stop are good times for this as well. An application 
>>> can stop a device, reconfigure it, and then restart it, so having that be 
>>> the trigger for capture file cleanup and close makes sense.


 Ilias Apalodimas(apalos) wrote:
 wireshark re-defines them in every c file they want to use them. Keeping 
 in mind this is the pcapng file format specification, these are high 
 unlikely to change.
 
 The only relevant library i found is 
 ntar(https://www.winpcap.org/ntar/changelog.htm) but it seems abandoned 
 for many many years.


> Ilias Apalodimas(apalos) wrote:
> @Bill-Fischofer-Linaro pcapng_destroy() is called from _pktio_stop() 
> which is called from odp_pktio_term_global().
> pcapng_prepare() on the other hand is called from odp_pktio_start(). Want 
> me to move it in odp_pktio_init_global()?


>> Ilias Apalodimas(apalos) wrote:
>> This essentially "reads" the inotify triggers for opening/closing the 
>> fifos and sets the appropriate variables for starting/stopping the 
>> pcapng dump on fifos. Will try to make it easier to read on the final PR


>>> Ilias Apalodimas(apalos) wrote:
>>> will fix


 Ilias Apalodimas(apalos) wrote:
 will fix


> Ilias Apalodimas(apalos) wrote:
> I'll check and update for the final PR


>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>> General cleanup should be part of `odp_term_global()` processing. 
>> Presumably `odp_pktio_close()` should also do cleanup for specific 
>> pktios.


>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>>> A few comments about what this routine is doing would be very 
>>> helpful here. This is not easy code to follow. Modularizing it into 
>>> some  `static inline` helpers might also make the logic simpler.


 Bill Fischofer(Bill-Fischofer-Linaro) wrote:
 This is redundant. If `len > 0` then by definition `len != -1`.


> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
> This fails for multi-segment packets. `odp_packet_data(pkt)` is 
> shorthand for `odp_packet_offset(pkt, 0, NULL, NULL)` so you 
> don't know how long the first segment really is. First cut would 
> be to only dump the first segment, so correct code is:
> ```
> uint32_t seg_len;
> char *buf = (char *)odp_packet_offset(packets[i], 0, &seg_len, 
> NULL);
> ```
> And use `seg_len` rather than `pkt_len` in this routine.


>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>> Does pcapng not supply a `.h` file for these "magic numbers" and 
>> associated structs?


>>> Ilias Apalodimas(apalos) wrote:
>>> Will fix. This will also remove the requirement of manually 
>>> deleting the fifos


 muvarov wrote
 odp_global_data_s->main_pid needs to be here.


https://github.com/Linaro/odp/pull/435#discussion_r166566828
updated_at 2018-02-07 10:31:08


Re: [lng-odp] [PATCH CATERPILLAR v1] [RFC] odp: pktio: added tcpdump like functionality

2018-02-07 Thread Github ODP bot
Mykyta Iziumtsev(MykytaI) replied on github web page:

platform/linux-generic/include/odp_pcapng.h
line 1
@@ -0,0 +1,56 @@
+/* Copyright (c) 2018, Linaro Limited


Comment:
Is there any reason to put all of this into separate header file ? From what I 
see only one .c file is using it, so why not move it there ?

> Mykyta Iziumtsev(MykytaI) wrote:
> IMHO, only `int pcapng_fds[PKTIO_MAX_QUEUES]` belong here, the rest is 
> superfluous or belongs to global context. Why not initialize pcapng_fds[0 .. 
> ARRAY_SIZE - 1] = -1, open the pipes from inotify thread and assign fds when 
> the header is already written out ?


>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>> @apalos No, start and stop are good times for this as well. An application 
>> can stop a device, reconfigure it, and then restart it, so having that be 
>> the trigger for capture file cleanup and close makes sense.


>>> Ilias Apalodimas(apalos) wrote:
>>> wireshark re-defines them in every c file they want to use them. Keeping in 
>>> mind this is the pcapng file format specification, these are high unlikely 
>>> to change.
>>> 
>>> The only relevant library i found is 
>>> ntar(https://www.winpcap.org/ntar/changelog.htm) but it seems abandoned for 
>>> many many years.


 Ilias Apalodimas(apalos) wrote:
 @Bill-Fischofer-Linaro pcapng_destroy() is called from _pktio_stop() which 
 is called from odp_pktio_term_global().
 pcapng_prepare() on the other hand is called from odp_pktio_start(). Want 
 me to move it in odp_pktio_init_global()?


> Ilias Apalodimas(apalos) wrote:
> This essentially "reads" the inotify triggers for opening/closing the 
> fifos and sets the appropriate variables for starting/stopping the pcapng 
> dump on fifos. Will try to make it easier to read on the final PR


>> Ilias Apalodimas(apalos) wrote:
>> will fix


>>> Ilias Apalodimas(apalos) wrote:
>>> will fix


 Ilias Apalodimas(apalos) wrote:
 I'll check and update for the final PR


> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
> General cleanup should be part of `odp_term_global()` processing. 
> Presumably `odp_pktio_close()` should also do cleanup for specific 
> pktios.


>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>> A few comments about what this routine is doing would be very 
>> helpful here. This is not easy code to follow. Modularizing it into 
>> some  `static inline` helpers might also make the logic simpler.


>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>>> This is redundant. If `len > 0` then by definition `len != -1`.


 Bill Fischofer(Bill-Fischofer-Linaro) wrote:
 This fails for multi-segment packets. `odp_packet_data(pkt)` is 
 shorthand for `odp_packet_offset(pkt, 0, NULL, NULL)` so you don't 
 know how long the first segment really is. First cut would be to 
 only dump the first segment, so correct code is:
 ```
 uint32_t seg_len;
 char *buf = (char *)odp_packet_offset(packets[i], 0, &seg_len, 
 NULL);
 ```
 And use `seg_len` rather than `pkt_len` in this routine.


> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
> Does pcapng not supply a `.h` file for these "magic numbers" and 
> associated structs?


>> Ilias Apalodimas(apalos) wrote:
>> Will fix. This will also remove the requirement of manually 
>> deleting the fifos


>>> muvarov wrote
>>> odp_global_data_s->main_pid needs to be here.


https://github.com/Linaro/odp/pull/435#discussion_r166565709
updated_at 2018-02-07 10:31:08


Re: [lng-odp] [PATCH CATERPILLAR v1] [RFC] odp: pktio: added tcpdump like functionality

2018-02-07 Thread Github ODP bot
Mykyta Iziumtsev(MykytaI) replied on github web page:

platform/linux-generic/include/odp_packet_io_internal.h
line 18
@@ -97,6 +97,20 @@ struct pktio_entry {
odp_queue_tqueue;
odp_pktout_queue_t pktout;
} out_queue[PKTIO_MAX_QUEUES];
+
+   /**< inotify instance for pcapng fifos */
+   struct {
+   int inotify_pcapng_fd;
+   int inotify_watch_fd;
+   pthread_t inotify_thread;
+   enum {
+   PCAPNG_WR_INVALID = 0,
+   PCAPNG_WR_STOP,
+   PCAPNG_WR_HDR,
+   PCAPNG_WR_PKT,
+   } state[PKTIO_MAX_QUEUES];
+   int pcapng_fd[PKTIO_MAX_QUEUES];
+   } pcapng_info;
 };


Comment:
IMHO, only `int pcapng_fds[PKTIO_MAX_QUEUES]` belong here, the rest is 
superfluous or belongs to global context. Why not initialize pcapng_fds[0 .. 
ARRAY_SIZE - 1] = -1, open the pipes from inotify thread and assign fds when 
the header is already written out ?


> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
> @apalos No, start and stop are good times for this as well. An application 
> can stop a device, reconfigure it, and then restart it, so having that be the 
> trigger for capture file cleanup and close makes sense.


>> Ilias Apalodimas(apalos) wrote:
>> wireshark re-defines them in every c file they want to use them. Keeping in 
>> mind this is the pcapng file format specification, these are high unlikely 
>> to change.
>> 
>> The only relevant library i found is 
>> ntar(https://www.winpcap.org/ntar/changelog.htm) but it seems abandoned for 
>> many many years.


>>> Ilias Apalodimas(apalos) wrote:
>>> @Bill-Fischofer-Linaro pcapng_destroy() is called from _pktio_stop() which 
>>> is called from odp_pktio_term_global().
>>> pcapng_prepare() on the other hand is called from odp_pktio_start(). Want 
>>> me to move it in odp_pktio_init_global()?


 Ilias Apalodimas(apalos) wrote:
 This essentially "reads" the inotify triggers for opening/closing the 
 fifos and sets the appropriate variables for starting/stopping the pcapng 
 dump on fifos. Will try to make it easier to read on the final PR


> Ilias Apalodimas(apalos) wrote:
> will fix


>> Ilias Apalodimas(apalos) wrote:
>> will fix


>>> Ilias Apalodimas(apalos) wrote:
>>> I'll check and update for the final PR


 Bill Fischofer(Bill-Fischofer-Linaro) wrote:
 General cleanup should be part of `odp_term_global()` processing. 
 Presumably `odp_pktio_close()` should also do cleanup for specific 
 pktios.


> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
> A few comments about what this routine is doing would be very helpful 
> here. This is not easy code to follow. Modularizing it into some  
> `static inline` helpers might also make the logic simpler.


>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>> This is redundant. If `len > 0` then by definition `len != -1`.


>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>>> This fails for multi-segment packets. `odp_packet_data(pkt)` is 
>>> shorthand for `odp_packet_offset(pkt, 0, NULL, NULL)` so you don't 
>>> know how long the first segment really is. First cut would be to 
>>> only dump the first segment, so correct code is:
>>> ```
>>> uint32_t seg_len;
>>> char *buf = (char *)odp_packet_offset(packets[i], 0, &seg_len, 
>>> NULL);
>>> ```
>>> And use `seg_len` rather than `pkt_len` in this routine.


 Bill Fischofer(Bill-Fischofer-Linaro) wrote:
 Does pcapng not supply a `.h` file for these "magic numbers" and 
 associated structs?


> Ilias Apalodimas(apalos) wrote:
> Will fix. This will also remove the requirement of manually 
> deleting the fifos


>> muvarov wrote
>> odp_global_data_s->main_pid needs to be here.


https://github.com/Linaro/odp/pull/435#discussion_r166565206
updated_at 2018-02-07 10:31:08


Re: [lng-odp] [PATCH CATERPILLAR v1] [RFC] odp: pktio: added tcpdump like functionality

2018-02-05 Thread Github ODP bot
Bill Fischofer(Bill-Fischofer-Linaro) replied on github web page:

platform/linux-generic/odp_packet_io.c
line 237
@@ -89,6 +96,274 @@ int odp_pktio_init_local(void)
return odp_pktio_ops_init_local(true);
 }
 
+static int write_pcapng_hdr(pktio_entry_t *entry, int qidx)
+{
+   size_t len;
+   pcapng_section_hdr_block_t shb;
+   pcapng_interface_description_block_s idb;
+   int fd = entry->s.pcapng_info.pcapng_fd[qidx];
+
+   memset(&shb, 0, sizeof(shb));
+   memset(&idb, 0, sizeof(idb));
+
+   shb.block_type = PCAPNG_BLOCK_TYPE_SHB;
+   shb.block_total_length = sizeof(shb);
+   shb.block_total_length2 = sizeof(shb);
+   shb.magic = PCAPNG_ENDIAN_MAGIC;
+   shb.version_major = 0x1;
+   shb.version_minor = 0x0;
+   shb.section_len = -1;
+
+   len = write(fd, &shb, sizeof(shb));
+   fsync(fd);
+   /* fail to write shb/idb means the pcapng is unreadable */
+   if (!len) {
+   ODP_ERR("Failed to write pcapng section hdr\n");
+   close(fd);
+   return -1;
+   }
+
+   idb.block_type = PCAPNG_BLOCK_TYPE_IDB;
+   idb.block_total_length = sizeof(idb);
+   idb.block_total_length2 = sizeof(idb);
+   idb.linktype = 0x1; /* LINKTYPE_ETHERNET */
+   idb.snaplen = 0x0; /* unlimited */
+   len = write(fd, &idb, sizeof(idb));
+   if (!len) {
+   ODP_ERR("Failed to write pcapng interface description\n");
+   close(fd);
+   return -1;
+   }
+   fsync(fd);
+
+   entry->s.pcapng_info.state[qidx] = PCAPNG_WR_HDR;
+
+   return 0;
+}
+
+static int write_pcapng_pkts(pktio_entry_t *entry, int qidx,
+const odp_packet_t packets[], int num)
+{
+   int i = 0;
+   struct iovec packet_iov[3 * num];
+   pcapng_enhanced_packet_block_t epb;
+   int iovcnt = 0;
+   ssize_t len = 0;
+   int fd = entry->s.pcapng_info.pcapng_fd[qidx];
+
+   for (i = 0; i < num; i++) {
+   odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(packets[i]);
+   uint32_t pkt_len = _odp_packet_len(packets[i]);
+   char *buf = (char *)odp_packet_data(packets[i]);
+
+   epb.block_type = PCAPNG_BLOCK_TYPE_EPB;
+   epb.block_total_length = sizeof(epb) +
+   ROUNDUP_ALIGN(pkt_len, PCAP_DATA_ALIGN) +
+   PCAP_DATA_ALIGN;
+   epb.interface_idx = 0;
+   epb.timestamp_high = (uint32_t)(pkt_hdr->timestamp.u64 >> 32);
+   epb.timestamp_low = (uint32_t)(pkt_hdr->timestamp.u64);
+   epb.captured_len = pkt_len;
+   epb.packet_len = pkt_len;
+
+   /* epb */
+   packet_iov[iovcnt].iov_base = &epb;
+   packet_iov[iovcnt].iov_len = sizeof(epb);
+   iovcnt++;
+
+   /* data */
+   packet_iov[iovcnt].iov_base = buf;
+   packet_iov[iovcnt].iov_len =
+   ROUNDUP_ALIGN(pkt_len, PCAP_DATA_ALIGN);
+   iovcnt++;
+
+   /* trailing */
+   packet_iov[iovcnt].iov_base = &epb.block_total_length;
+   packet_iov[iovcnt].iov_len = sizeof(uint32_t);
+   iovcnt++;
+   }
+
+   /* we don't really care if we manage to write *all* data */
+   len = writev(fd, packet_iov, iovcnt);
+   if (!len)
+   ODP_ERR("Failed to write pcapng data\n");
+   fsync(fd);
+
+   return len;
+}
+
+static void pcapng_drain_fifo(int fd)
+{
+   char c;
+   ssize_t len;
+
+   do {
+   len = read(fd, &c, sizeof(c));
+   } while (len > 0 && len != -1);
+   ODP_DBG("Drain pcap fifo %d\n", len);
+}
+
+static void *inotify_update(void *arg)
+{
+   pktio_entry_t *entry = (pktio_entry_t *)arg;
+   struct timeval time;
+   int ret;
+   ssize_t rdlen;
+   int i;
+   char buffer[INOTIFY_BUF_LEN];
+   unsigned int max_queue =
+   MAX(entry->s.num_in_queue, entry->s.num_out_queue);
+   fd_set rfds;
+
+   while (1) {
+   i = 0;
+   FD_ZERO(&rfds);
+   FD_SET(entry->s.pcapng_info.inotify_pcapng_fd, &rfds);
+   time.tv_sec = 2;
+   time.tv_usec = 0;
+   select(entry->s.pcapng_info.inotify_pcapng_fd + 1, &rfds, NULL,
+  NULL, &time);
+   if (FD_ISSET(entry->s.pcapng_info.inotify_pcapng_fd, &rfds)) {
+   rdlen = read(entry->s.pcapng_info.inotify_pcapng_fd,
+buffer, INOTIFY_BUF_LEN);
+   while (i < rdlen) {
+   char *e;
+   unsigned int qidx;
+   struct inotify_event *event =
+   (struct inotify_event *)(void *)
+&buffer[i];
+
+

Re: [lng-odp] [PATCH CATERPILLAR v1] [RFC] odp: pktio: added tcpdump like functionality

2018-02-02 Thread Github ODP bot
Ilias Apalodimas(apalos) replied on github web page:

platform/linux-generic/odp_packet_io.c
line 237
@@ -89,6 +96,274 @@ int odp_pktio_init_local(void)
return odp_pktio_ops_init_local(true);
 }
 
+static int write_pcapng_hdr(pktio_entry_t *entry, int qidx)
+{
+   size_t len;
+   pcapng_section_hdr_block_t shb;
+   pcapng_interface_description_block_s idb;
+   int fd = entry->s.pcapng_info.pcapng_fd[qidx];
+
+   memset(&shb, 0, sizeof(shb));
+   memset(&idb, 0, sizeof(idb));
+
+   shb.block_type = PCAPNG_BLOCK_TYPE_SHB;
+   shb.block_total_length = sizeof(shb);
+   shb.block_total_length2 = sizeof(shb);
+   shb.magic = PCAPNG_ENDIAN_MAGIC;
+   shb.version_major = 0x1;
+   shb.version_minor = 0x0;
+   shb.section_len = -1;
+
+   len = write(fd, &shb, sizeof(shb));
+   fsync(fd);
+   /* fail to write shb/idb means the pcapng is unreadable */
+   if (!len) {
+   ODP_ERR("Failed to write pcapng section hdr\n");
+   close(fd);
+   return -1;
+   }
+
+   idb.block_type = PCAPNG_BLOCK_TYPE_IDB;
+   idb.block_total_length = sizeof(idb);
+   idb.block_total_length2 = sizeof(idb);
+   idb.linktype = 0x1; /* LINKTYPE_ETHERNET */
+   idb.snaplen = 0x0; /* unlimited */
+   len = write(fd, &idb, sizeof(idb));
+   if (!len) {
+   ODP_ERR("Failed to write pcapng interface description\n");
+   close(fd);
+   return -1;
+   }
+   fsync(fd);
+
+   entry->s.pcapng_info.state[qidx] = PCAPNG_WR_HDR;
+
+   return 0;
+}
+
+static int write_pcapng_pkts(pktio_entry_t *entry, int qidx,
+const odp_packet_t packets[], int num)
+{
+   int i = 0;
+   struct iovec packet_iov[3 * num];
+   pcapng_enhanced_packet_block_t epb;
+   int iovcnt = 0;
+   ssize_t len = 0;
+   int fd = entry->s.pcapng_info.pcapng_fd[qidx];
+
+   for (i = 0; i < num; i++) {
+   odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(packets[i]);
+   uint32_t pkt_len = _odp_packet_len(packets[i]);
+   char *buf = (char *)odp_packet_data(packets[i]);
+
+   epb.block_type = PCAPNG_BLOCK_TYPE_EPB;
+   epb.block_total_length = sizeof(epb) +
+   ROUNDUP_ALIGN(pkt_len, PCAP_DATA_ALIGN) +
+   PCAP_DATA_ALIGN;
+   epb.interface_idx = 0;
+   epb.timestamp_high = (uint32_t)(pkt_hdr->timestamp.u64 >> 32);
+   epb.timestamp_low = (uint32_t)(pkt_hdr->timestamp.u64);
+   epb.captured_len = pkt_len;
+   epb.packet_len = pkt_len;
+
+   /* epb */
+   packet_iov[iovcnt].iov_base = &epb;
+   packet_iov[iovcnt].iov_len = sizeof(epb);
+   iovcnt++;
+
+   /* data */
+   packet_iov[iovcnt].iov_base = buf;
+   packet_iov[iovcnt].iov_len =
+   ROUNDUP_ALIGN(pkt_len, PCAP_DATA_ALIGN);
+   iovcnt++;
+
+   /* trailing */
+   packet_iov[iovcnt].iov_base = &epb.block_total_length;
+   packet_iov[iovcnt].iov_len = sizeof(uint32_t);
+   iovcnt++;
+   }
+
+   /* we don't really care if we manage to write *all* data */
+   len = writev(fd, packet_iov, iovcnt);
+   if (!len)
+   ODP_ERR("Failed to write pcapng data\n");
+   fsync(fd);
+
+   return len;
+}
+
+static void pcapng_drain_fifo(int fd)
+{
+   char c;
+   ssize_t len;
+
+   do {
+   len = read(fd, &c, sizeof(c));
+   } while (len > 0 && len != -1);
+   ODP_DBG("Drain pcap fifo %d\n", len);
+}
+
+static void *inotify_update(void *arg)
+{
+   pktio_entry_t *entry = (pktio_entry_t *)arg;
+   struct timeval time;
+   int ret;
+   ssize_t rdlen;
+   int i;
+   char buffer[INOTIFY_BUF_LEN];
+   unsigned int max_queue =
+   MAX(entry->s.num_in_queue, entry->s.num_out_queue);
+   fd_set rfds;
+
+   while (1) {
+   i = 0;
+   FD_ZERO(&rfds);
+   FD_SET(entry->s.pcapng_info.inotify_pcapng_fd, &rfds);
+   time.tv_sec = 2;
+   time.tv_usec = 0;
+   select(entry->s.pcapng_info.inotify_pcapng_fd + 1, &rfds, NULL,
+  NULL, &time);
+   if (FD_ISSET(entry->s.pcapng_info.inotify_pcapng_fd, &rfds)) {
+   rdlen = read(entry->s.pcapng_info.inotify_pcapng_fd,
+buffer, INOTIFY_BUF_LEN);
+   while (i < rdlen) {
+   char *e;
+   unsigned int qidx;
+   struct inotify_event *event =
+   (struct inotify_event *)(void *)
+&buffer[i];
+
+   e 

Re: [lng-odp] [PATCH CATERPILLAR v1] [RFC] odp: pktio: added tcpdump like functionality

2018-02-02 Thread Github ODP bot
Ilias Apalodimas(apalos) replied on github web page:

platform/linux-generic/include/odp_pcapng.h
line 20
@@ -0,0 +1,56 @@
+/* Copyright (c) 2018, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#define PCAPNG_BLOCK_TYPE_EPB 0x0006UL
+#define PCAPNG_BLOCK_TYPE_SHB 0x0A0D0D0AUL
+#define PCAPNG_BLOCK_TYPE_IDB 0x0001UL
+#define PCAPNG_ENDIAN_MAGIC 0x1A2B3C4DUL
+#define PCAP_DATA_ALIGN (4)


Comment:
wireshark re-defines them in every c file they want to use them. Keeping in 
mind this is the pcapng file format specification, these are high unlikely to 
change.

The only relevant library i found is 
ntar(https://www.winpcap.org/ntar/changelog.htm) but it seems abandoned for 
many many years.

> Ilias Apalodimas(apalos) wrote:
> will update


>> Ilias Apalodimas(apalos) wrote:
>> This essentially "reads" the inotify triggers for opening/closing the fifos 
>> and sets the appropriate variables for starting/stopping the pcapng dump on 
>> fifos. Will try to make it easier to read on the final PR


>>> Ilias Apalodimas(apalos) wrote:
>>> will fix


 Ilias Apalodimas(apalos) wrote:
 will fix


> Ilias Apalodimas(apalos) wrote:
> I'll check and update for the final PR


>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>> General cleanup should be part of `odp_term_global()` processing. 
>> Presumably `odp_pktio_close()` should also do cleanup for specific 
>> pktios.


>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>>> A few comments about what this routine is doing would be very helpful 
>>> here. This is not easy code to follow. Modularizing it into some  
>>> `static inline` helpers might also make the logic simpler.


 Bill Fischofer(Bill-Fischofer-Linaro) wrote:
 This is redundant. If `len > 0` then by definition `len != -1`.


> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
> This fails for multi-segment packets. `odp_packet_data(pkt)` is 
> shorthand for `odp_packet_offset(pkt, 0, NULL, NULL)` so you don't 
> know how long the first segment really is. First cut would be to only 
> dump the first segment, so correct code is:
> ```
> uint32_t seg_len;
> char *buf = (char *)odp_packet_offset(packets[i], 0, &seg_len, NULL);
> ```
> And use `seg_len` rather than `pkt_len` in this routine.


>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>> Does pcapng not supply a `.h` file for these "magic numbers" and 
>> associated structs?


>>> Ilias Apalodimas(apalos) wrote:
>>> Will fix. This will also remove the requirement of manually 
>>> deleting the fifos


 muvarov wrote
 odp_global_data_s->main_pid needs to be here.


https://github.com/Linaro/odp/pull/435#discussion_r165633289
updated_at 2018-02-02 12:46:49


Re: [lng-odp] [PATCH CATERPILLAR v1] [RFC] odp: pktio: added tcpdump like functionality

2018-02-02 Thread Github ODP bot
Ilias Apalodimas(apalos) replied on github web page:

platform/linux-generic/odp_packet_io.c
line 137
@@ -89,6 +96,274 @@ int odp_pktio_init_local(void)
return odp_pktio_ops_init_local(true);
 }
 
+static int write_pcapng_hdr(pktio_entry_t *entry, int qidx)
+{
+   size_t len;
+   pcapng_section_hdr_block_t shb;
+   pcapng_interface_description_block_s idb;
+   int fd = entry->s.pcapng_info.pcapng_fd[qidx];
+
+   memset(&shb, 0, sizeof(shb));
+   memset(&idb, 0, sizeof(idb));
+
+   shb.block_type = PCAPNG_BLOCK_TYPE_SHB;
+   shb.block_total_length = sizeof(shb);
+   shb.block_total_length2 = sizeof(shb);
+   shb.magic = PCAPNG_ENDIAN_MAGIC;
+   shb.version_major = 0x1;
+   shb.version_minor = 0x0;
+   shb.section_len = -1;
+
+   len = write(fd, &shb, sizeof(shb));
+   fsync(fd);
+   /* fail to write shb/idb means the pcapng is unreadable */
+   if (!len) {
+   ODP_ERR("Failed to write pcapng section hdr\n");
+   close(fd);
+   return -1;
+   }
+
+   idb.block_type = PCAPNG_BLOCK_TYPE_IDB;
+   idb.block_total_length = sizeof(idb);
+   idb.block_total_length2 = sizeof(idb);
+   idb.linktype = 0x1; /* LINKTYPE_ETHERNET */
+   idb.snaplen = 0x0; /* unlimited */
+   len = write(fd, &idb, sizeof(idb));
+   if (!len) {
+   ODP_ERR("Failed to write pcapng interface description\n");
+   close(fd);
+   return -1;
+   }
+   fsync(fd);
+
+   entry->s.pcapng_info.state[qidx] = PCAPNG_WR_HDR;
+
+   return 0;
+}
+
+static int write_pcapng_pkts(pktio_entry_t *entry, int qidx,
+const odp_packet_t packets[], int num)
+{
+   int i = 0;
+   struct iovec packet_iov[3 * num];
+   pcapng_enhanced_packet_block_t epb;
+   int iovcnt = 0;
+   ssize_t len = 0;
+   int fd = entry->s.pcapng_info.pcapng_fd[qidx];
+
+   for (i = 0; i < num; i++) {
+   odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(packets[i]);
+   uint32_t pkt_len = _odp_packet_len(packets[i]);
+   char *buf = (char *)odp_packet_data(packets[i]);
+
+   epb.block_type = PCAPNG_BLOCK_TYPE_EPB;
+   epb.block_total_length = sizeof(epb) +
+   ROUNDUP_ALIGN(pkt_len, PCAP_DATA_ALIGN) +
+   PCAP_DATA_ALIGN;
+   epb.interface_idx = 0;
+   epb.timestamp_high = (uint32_t)(pkt_hdr->timestamp.u64 >> 32);
+   epb.timestamp_low = (uint32_t)(pkt_hdr->timestamp.u64);
+   epb.captured_len = pkt_len;
+   epb.packet_len = pkt_len;
+
+   /* epb */
+   packet_iov[iovcnt].iov_base = &epb;
+   packet_iov[iovcnt].iov_len = sizeof(epb);
+   iovcnt++;
+
+   /* data */
+   packet_iov[iovcnt].iov_base = buf;
+   packet_iov[iovcnt].iov_len =
+   ROUNDUP_ALIGN(pkt_len, PCAP_DATA_ALIGN);
+   iovcnt++;
+
+   /* trailing */
+   packet_iov[iovcnt].iov_base = &epb.block_total_length;
+   packet_iov[iovcnt].iov_len = sizeof(uint32_t);
+   iovcnt++;
+   }
+
+   /* we don't really care if we manage to write *all* data */
+   len = writev(fd, packet_iov, iovcnt);
+   if (!len)
+   ODP_ERR("Failed to write pcapng data\n");
+   fsync(fd);
+
+   return len;
+}
+
+static void pcapng_drain_fifo(int fd)
+{
+   char c;
+   ssize_t len;
+
+   do {
+   len = read(fd, &c, sizeof(c));
+   } while (len > 0 && len != -1);
+   ODP_DBG("Drain pcap fifo %d\n", len);
+}
+
+static void *inotify_update(void *arg)
+{
+   pktio_entry_t *entry = (pktio_entry_t *)arg;
+   struct timeval time;
+   int ret;
+   ssize_t rdlen;
+   int i;
+   char buffer[INOTIFY_BUF_LEN];
+   unsigned int max_queue =
+   MAX(entry->s.num_in_queue, entry->s.num_out_queue);
+   fd_set rfds;
+
+   while (1) {


Comment:
This essentially "reads" the inotify triggers for opening/closing the fifos and 
sets the appropriate variables for starting/stopping the pcapng dump on fifos. 
Will try to make it easier to read on the final PR

> Ilias Apalodimas(apalos) wrote:
> will fix


>> Ilias Apalodimas(apalos) wrote:
>> will fix


>>> Ilias Apalodimas(apalos) wrote:
>>> I'll check and update for the final PR


 Bill Fischofer(Bill-Fischofer-Linaro) wrote:
 General cleanup should be part of `odp_term_global()` processing. 
 Presumably `odp_pktio_close()` should also do cleanup for specific pktios.


> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
> A few comments about what this routine is doing would be very helpful 
> here. This is not easy code to follow. Modularizing it into some  `static 
> inline` helpers might also make the logic simpler.


>> Bill Fisc

Re: [lng-odp] [PATCH CATERPILLAR v1] [RFC] odp: pktio: added tcpdump like functionality

2018-02-02 Thread Github ODP bot
Ilias Apalodimas(apalos) replied on github web page:

platform/linux-generic/odp_packet_io.c
line 121
@@ -89,6 +96,274 @@ int odp_pktio_init_local(void)
return odp_pktio_ops_init_local(true);
 }
 
+static int write_pcapng_hdr(pktio_entry_t *entry, int qidx)
+{
+   size_t len;
+   pcapng_section_hdr_block_t shb;
+   pcapng_interface_description_block_s idb;
+   int fd = entry->s.pcapng_info.pcapng_fd[qidx];
+
+   memset(&shb, 0, sizeof(shb));
+   memset(&idb, 0, sizeof(idb));
+
+   shb.block_type = PCAPNG_BLOCK_TYPE_SHB;
+   shb.block_total_length = sizeof(shb);
+   shb.block_total_length2 = sizeof(shb);
+   shb.magic = PCAPNG_ENDIAN_MAGIC;
+   shb.version_major = 0x1;
+   shb.version_minor = 0x0;
+   shb.section_len = -1;
+
+   len = write(fd, &shb, sizeof(shb));
+   fsync(fd);
+   /* fail to write shb/idb means the pcapng is unreadable */
+   if (!len) {
+   ODP_ERR("Failed to write pcapng section hdr\n");
+   close(fd);
+   return -1;
+   }
+
+   idb.block_type = PCAPNG_BLOCK_TYPE_IDB;
+   idb.block_total_length = sizeof(idb);
+   idb.block_total_length2 = sizeof(idb);
+   idb.linktype = 0x1; /* LINKTYPE_ETHERNET */
+   idb.snaplen = 0x0; /* unlimited */
+   len = write(fd, &idb, sizeof(idb));
+   if (!len) {
+   ODP_ERR("Failed to write pcapng interface description\n");
+   close(fd);
+   return -1;
+   }
+   fsync(fd);
+
+   entry->s.pcapng_info.state[qidx] = PCAPNG_WR_HDR;
+
+   return 0;
+}
+
+static int write_pcapng_pkts(pktio_entry_t *entry, int qidx,
+const odp_packet_t packets[], int num)
+{
+   int i = 0;
+   struct iovec packet_iov[3 * num];
+   pcapng_enhanced_packet_block_t epb;
+   int iovcnt = 0;
+   ssize_t len = 0;
+   int fd = entry->s.pcapng_info.pcapng_fd[qidx];
+
+   for (i = 0; i < num; i++) {
+   odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(packets[i]);
+   uint32_t pkt_len = _odp_packet_len(packets[i]);
+   char *buf = (char *)odp_packet_data(packets[i]);
+
+   epb.block_type = PCAPNG_BLOCK_TYPE_EPB;
+   epb.block_total_length = sizeof(epb) +
+   ROUNDUP_ALIGN(pkt_len, PCAP_DATA_ALIGN) +
+   PCAP_DATA_ALIGN;
+   epb.interface_idx = 0;
+   epb.timestamp_high = (uint32_t)(pkt_hdr->timestamp.u64 >> 32);
+   epb.timestamp_low = (uint32_t)(pkt_hdr->timestamp.u64);
+   epb.captured_len = pkt_len;
+   epb.packet_len = pkt_len;
+
+   /* epb */
+   packet_iov[iovcnt].iov_base = &epb;
+   packet_iov[iovcnt].iov_len = sizeof(epb);
+   iovcnt++;
+
+   /* data */
+   packet_iov[iovcnt].iov_base = buf;
+   packet_iov[iovcnt].iov_len =
+   ROUNDUP_ALIGN(pkt_len, PCAP_DATA_ALIGN);
+   iovcnt++;
+
+   /* trailing */
+   packet_iov[iovcnt].iov_base = &epb.block_total_length;
+   packet_iov[iovcnt].iov_len = sizeof(uint32_t);
+   iovcnt++;
+   }
+
+   /* we don't really care if we manage to write *all* data */
+   len = writev(fd, packet_iov, iovcnt);
+   if (!len)
+   ODP_ERR("Failed to write pcapng data\n");
+   fsync(fd);
+
+   return len;
+}
+
+static void pcapng_drain_fifo(int fd)
+{
+   char c;
+   ssize_t len;
+
+   do {
+   len = read(fd, &c, sizeof(c));
+   } while (len > 0 && len != -1);


Comment:
will fix

> Ilias Apalodimas(apalos) wrote:
> will fix


>> Ilias Apalodimas(apalos) wrote:
>> I'll check and update for the final PR


>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>>> General cleanup should be part of `odp_term_global()` processing. 
>>> Presumably `odp_pktio_close()` should also do cleanup for specific pktios.


 Bill Fischofer(Bill-Fischofer-Linaro) wrote:
 A few comments about what this routine is doing would be very helpful 
 here. This is not easy code to follow. Modularizing it into some  `static 
 inline` helpers might also make the logic simpler.


> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
> This is redundant. If `len > 0` then by definition `len != -1`.


>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>> This fails for multi-segment packets. `odp_packet_data(pkt)` is 
>> shorthand for `odp_packet_offset(pkt, 0, NULL, NULL)` so you don't know 
>> how long the first segment really is. First cut would be to only dump 
>> the first segment, so correct code is:
>> ```
>> uint32_t seg_len;
>> char *buf = (char *)odp_packet_offset(packets[i], 0, &seg_len, NULL);
>> ```
>> And use `seg_len` rather than `pkt_len` in this routine.


>>> Bill Fischofer(Bill-Fischofer-Linaro) wro

Re: [lng-odp] [PATCH CATERPILLAR v1] [RFC] odp: pktio: added tcpdump like functionality

2018-02-02 Thread Github ODP bot
Ilias Apalodimas(apalos) replied on github web page:

platform/linux-generic/odp_packet_io.c
line 76
@@ -89,6 +96,274 @@ int odp_pktio_init_local(void)
return odp_pktio_ops_init_local(true);
 }
 
+static int write_pcapng_hdr(pktio_entry_t *entry, int qidx)
+{
+   size_t len;
+   pcapng_section_hdr_block_t shb;
+   pcapng_interface_description_block_s idb;
+   int fd = entry->s.pcapng_info.pcapng_fd[qidx];
+
+   memset(&shb, 0, sizeof(shb));
+   memset(&idb, 0, sizeof(idb));
+
+   shb.block_type = PCAPNG_BLOCK_TYPE_SHB;
+   shb.block_total_length = sizeof(shb);
+   shb.block_total_length2 = sizeof(shb);
+   shb.magic = PCAPNG_ENDIAN_MAGIC;
+   shb.version_major = 0x1;
+   shb.version_minor = 0x0;
+   shb.section_len = -1;
+
+   len = write(fd, &shb, sizeof(shb));
+   fsync(fd);
+   /* fail to write shb/idb means the pcapng is unreadable */
+   if (!len) {
+   ODP_ERR("Failed to write pcapng section hdr\n");
+   close(fd);
+   return -1;
+   }
+
+   idb.block_type = PCAPNG_BLOCK_TYPE_IDB;
+   idb.block_total_length = sizeof(idb);
+   idb.block_total_length2 = sizeof(idb);
+   idb.linktype = 0x1; /* LINKTYPE_ETHERNET */
+   idb.snaplen = 0x0; /* unlimited */
+   len = write(fd, &idb, sizeof(idb));
+   if (!len) {
+   ODP_ERR("Failed to write pcapng interface description\n");
+   close(fd);
+   return -1;
+   }
+   fsync(fd);
+
+   entry->s.pcapng_info.state[qidx] = PCAPNG_WR_HDR;
+
+   return 0;
+}
+
+static int write_pcapng_pkts(pktio_entry_t *entry, int qidx,
+const odp_packet_t packets[], int num)
+{
+   int i = 0;
+   struct iovec packet_iov[3 * num];
+   pcapng_enhanced_packet_block_t epb;
+   int iovcnt = 0;
+   ssize_t len = 0;
+   int fd = entry->s.pcapng_info.pcapng_fd[qidx];
+
+   for (i = 0; i < num; i++) {
+   odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(packets[i]);
+   uint32_t pkt_len = _odp_packet_len(packets[i]);
+   char *buf = (char *)odp_packet_data(packets[i]);


Comment:
will fix

> Ilias Apalodimas(apalos) wrote:
> I'll check and update for the final PR


>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>> General cleanup should be part of `odp_term_global()` processing. Presumably 
>> `odp_pktio_close()` should also do cleanup for specific pktios.


>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>>> A few comments about what this routine is doing would be very helpful here. 
>>> This is not easy code to follow. Modularizing it into some  `static inline` 
>>> helpers might also make the logic simpler.


 Bill Fischofer(Bill-Fischofer-Linaro) wrote:
 This is redundant. If `len > 0` then by definition `len != -1`.


> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
> This fails for multi-segment packets. `odp_packet_data(pkt)` is shorthand 
> for `odp_packet_offset(pkt, 0, NULL, NULL)` so you don't know how long 
> the first segment really is. First cut would be to only dump the first 
> segment, so correct code is:
> ```
> uint32_t seg_len;
> char *buf = (char *)odp_packet_offset(packets[i], 0, &seg_len, NULL);
> ```
> And use `seg_len` rather than `pkt_len` in this routine.


>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>> Does pcapng not supply a `.h` file for these "magic numbers" and 
>> associated structs?


>>> Ilias Apalodimas(apalos) wrote:
>>> Will fix. This will also remove the requirement of manually deleting 
>>> the fifos


 muvarov wrote
 odp_global_data_s->main_pid needs to be here.


https://github.com/Linaro/odp/pull/435#discussion_r165571963
updated_at 2018-02-02 07:13:58


Re: [lng-odp] [PATCH CATERPILLAR v1] [RFC] odp: pktio: added tcpdump like functionality

2018-02-02 Thread Github ODP bot
Ilias Apalodimas(apalos) replied on github web page:

platform/linux-generic/include/odp_pcapng.h
line 20
@@ -0,0 +1,56 @@
+/* Copyright (c) 2018, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#define PCAPNG_BLOCK_TYPE_EPB 0x0006UL
+#define PCAPNG_BLOCK_TYPE_SHB 0x0A0D0D0AUL
+#define PCAPNG_BLOCK_TYPE_IDB 0x0001UL
+#define PCAPNG_ENDIAN_MAGIC 0x1A2B3C4DUL
+#define PCAP_DATA_ALIGN (4)


Comment:
I'll check and update for the final PR

> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
> General cleanup should be part of `odp_term_global()` processing. Presumably 
> `odp_pktio_close()` should also do cleanup for specific pktios.


>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>> A few comments about what this routine is doing would be very helpful here. 
>> This is not easy code to follow. Modularizing it into some  `static inline` 
>> helpers might also make the logic simpler.


>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>>> This is redundant. If `len > 0` then by definition `len != -1`.


 Bill Fischofer(Bill-Fischofer-Linaro) wrote:
 This fails for multi-segment packets. `odp_packet_data(pkt)` is shorthand 
 for `odp_packet_offset(pkt, 0, NULL, NULL)` so you don't know how long the 
 first segment really is. First cut would be to only dump the first 
 segment, so correct code is:
 ```
 uint32_t seg_len;
 char *buf = (char *)odp_packet_offset(packets[i], 0, &seg_len, NULL);
 ```
 And use `seg_len` rather than `pkt_len` in this routine.


> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
> Does pcapng not supply a `.h` file for these "magic numbers" and 
> associated structs?


>> Ilias Apalodimas(apalos) wrote:
>> Will fix. This will also remove the requirement of manually deleting the 
>> fifos


>>> muvarov wrote
>>> odp_global_data_s->main_pid needs to be here.


https://github.com/Linaro/odp/pull/435#discussion_r165571920
updated_at 2018-02-02 07:13:35


Re: [lng-odp] [PATCH CATERPILLAR v1] [RFC] odp: pktio: added tcpdump like functionality

2018-02-02 Thread Github ODP bot
Bill Fischofer(Bill-Fischofer-Linaro) replied on github web page:

platform/linux-generic/odp_packet_io.c
line 237
@@ -89,6 +96,274 @@ int odp_pktio_init_local(void)
return odp_pktio_ops_init_local(true);
 }
 
+static int write_pcapng_hdr(pktio_entry_t *entry, int qidx)
+{
+   size_t len;
+   pcapng_section_hdr_block_t shb;
+   pcapng_interface_description_block_s idb;
+   int fd = entry->s.pcapng_info.pcapng_fd[qidx];
+
+   memset(&shb, 0, sizeof(shb));
+   memset(&idb, 0, sizeof(idb));
+
+   shb.block_type = PCAPNG_BLOCK_TYPE_SHB;
+   shb.block_total_length = sizeof(shb);
+   shb.block_total_length2 = sizeof(shb);
+   shb.magic = PCAPNG_ENDIAN_MAGIC;
+   shb.version_major = 0x1;
+   shb.version_minor = 0x0;
+   shb.section_len = -1;
+
+   len = write(fd, &shb, sizeof(shb));
+   fsync(fd);
+   /* fail to write shb/idb means the pcapng is unreadable */
+   if (!len) {
+   ODP_ERR("Failed to write pcapng section hdr\n");
+   close(fd);
+   return -1;
+   }
+
+   idb.block_type = PCAPNG_BLOCK_TYPE_IDB;
+   idb.block_total_length = sizeof(idb);
+   idb.block_total_length2 = sizeof(idb);
+   idb.linktype = 0x1; /* LINKTYPE_ETHERNET */
+   idb.snaplen = 0x0; /* unlimited */
+   len = write(fd, &idb, sizeof(idb));
+   if (!len) {
+   ODP_ERR("Failed to write pcapng interface description\n");
+   close(fd);
+   return -1;
+   }
+   fsync(fd);
+
+   entry->s.pcapng_info.state[qidx] = PCAPNG_WR_HDR;
+
+   return 0;
+}
+
+static int write_pcapng_pkts(pktio_entry_t *entry, int qidx,
+const odp_packet_t packets[], int num)
+{
+   int i = 0;
+   struct iovec packet_iov[3 * num];
+   pcapng_enhanced_packet_block_t epb;
+   int iovcnt = 0;
+   ssize_t len = 0;
+   int fd = entry->s.pcapng_info.pcapng_fd[qidx];
+
+   for (i = 0; i < num; i++) {
+   odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(packets[i]);
+   uint32_t pkt_len = _odp_packet_len(packets[i]);
+   char *buf = (char *)odp_packet_data(packets[i]);
+
+   epb.block_type = PCAPNG_BLOCK_TYPE_EPB;
+   epb.block_total_length = sizeof(epb) +
+   ROUNDUP_ALIGN(pkt_len, PCAP_DATA_ALIGN) +
+   PCAP_DATA_ALIGN;
+   epb.interface_idx = 0;
+   epb.timestamp_high = (uint32_t)(pkt_hdr->timestamp.u64 >> 32);
+   epb.timestamp_low = (uint32_t)(pkt_hdr->timestamp.u64);
+   epb.captured_len = pkt_len;
+   epb.packet_len = pkt_len;
+
+   /* epb */
+   packet_iov[iovcnt].iov_base = &epb;
+   packet_iov[iovcnt].iov_len = sizeof(epb);
+   iovcnt++;
+
+   /* data */
+   packet_iov[iovcnt].iov_base = buf;
+   packet_iov[iovcnt].iov_len =
+   ROUNDUP_ALIGN(pkt_len, PCAP_DATA_ALIGN);
+   iovcnt++;
+
+   /* trailing */
+   packet_iov[iovcnt].iov_base = &epb.block_total_length;
+   packet_iov[iovcnt].iov_len = sizeof(uint32_t);
+   iovcnt++;
+   }
+
+   /* we don't really care if we manage to write *all* data */
+   len = writev(fd, packet_iov, iovcnt);
+   if (!len)
+   ODP_ERR("Failed to write pcapng data\n");
+   fsync(fd);
+
+   return len;
+}
+
+static void pcapng_drain_fifo(int fd)
+{
+   char c;
+   ssize_t len;
+
+   do {
+   len = read(fd, &c, sizeof(c));
+   } while (len > 0 && len != -1);
+   ODP_DBG("Drain pcap fifo %d\n", len);
+}
+
+static void *inotify_update(void *arg)
+{
+   pktio_entry_t *entry = (pktio_entry_t *)arg;
+   struct timeval time;
+   int ret;
+   ssize_t rdlen;
+   int i;
+   char buffer[INOTIFY_BUF_LEN];
+   unsigned int max_queue =
+   MAX(entry->s.num_in_queue, entry->s.num_out_queue);
+   fd_set rfds;
+
+   while (1) {
+   i = 0;
+   FD_ZERO(&rfds);
+   FD_SET(entry->s.pcapng_info.inotify_pcapng_fd, &rfds);
+   time.tv_sec = 2;
+   time.tv_usec = 0;
+   select(entry->s.pcapng_info.inotify_pcapng_fd + 1, &rfds, NULL,
+  NULL, &time);
+   if (FD_ISSET(entry->s.pcapng_info.inotify_pcapng_fd, &rfds)) {
+   rdlen = read(entry->s.pcapng_info.inotify_pcapng_fd,
+buffer, INOTIFY_BUF_LEN);
+   while (i < rdlen) {
+   char *e;
+   unsigned int qidx;
+   struct inotify_event *event =
+   (struct inotify_event *)(void *)
+&buffer[i];
+
+

Re: [lng-odp] [PATCH CATERPILLAR v1] [RFC] odp: pktio: added tcpdump like functionality

2018-02-02 Thread Github ODP bot
Bill Fischofer(Bill-Fischofer-Linaro) replied on github web page:

platform/linux-generic/odp_packet_io.c
line 121
@@ -89,6 +96,274 @@ int odp_pktio_init_local(void)
return odp_pktio_ops_init_local(true);
 }
 
+static int write_pcapng_hdr(pktio_entry_t *entry, int qidx)
+{
+   size_t len;
+   pcapng_section_hdr_block_t shb;
+   pcapng_interface_description_block_s idb;
+   int fd = entry->s.pcapng_info.pcapng_fd[qidx];
+
+   memset(&shb, 0, sizeof(shb));
+   memset(&idb, 0, sizeof(idb));
+
+   shb.block_type = PCAPNG_BLOCK_TYPE_SHB;
+   shb.block_total_length = sizeof(shb);
+   shb.block_total_length2 = sizeof(shb);
+   shb.magic = PCAPNG_ENDIAN_MAGIC;
+   shb.version_major = 0x1;
+   shb.version_minor = 0x0;
+   shb.section_len = -1;
+
+   len = write(fd, &shb, sizeof(shb));
+   fsync(fd);
+   /* fail to write shb/idb means the pcapng is unreadable */
+   if (!len) {
+   ODP_ERR("Failed to write pcapng section hdr\n");
+   close(fd);
+   return -1;
+   }
+
+   idb.block_type = PCAPNG_BLOCK_TYPE_IDB;
+   idb.block_total_length = sizeof(idb);
+   idb.block_total_length2 = sizeof(idb);
+   idb.linktype = 0x1; /* LINKTYPE_ETHERNET */
+   idb.snaplen = 0x0; /* unlimited */
+   len = write(fd, &idb, sizeof(idb));
+   if (!len) {
+   ODP_ERR("Failed to write pcapng interface description\n");
+   close(fd);
+   return -1;
+   }
+   fsync(fd);
+
+   entry->s.pcapng_info.state[qidx] = PCAPNG_WR_HDR;
+
+   return 0;
+}
+
+static int write_pcapng_pkts(pktio_entry_t *entry, int qidx,
+const odp_packet_t packets[], int num)
+{
+   int i = 0;
+   struct iovec packet_iov[3 * num];
+   pcapng_enhanced_packet_block_t epb;
+   int iovcnt = 0;
+   ssize_t len = 0;
+   int fd = entry->s.pcapng_info.pcapng_fd[qidx];
+
+   for (i = 0; i < num; i++) {
+   odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(packets[i]);
+   uint32_t pkt_len = _odp_packet_len(packets[i]);
+   char *buf = (char *)odp_packet_data(packets[i]);
+
+   epb.block_type = PCAPNG_BLOCK_TYPE_EPB;
+   epb.block_total_length = sizeof(epb) +
+   ROUNDUP_ALIGN(pkt_len, PCAP_DATA_ALIGN) +
+   PCAP_DATA_ALIGN;
+   epb.interface_idx = 0;
+   epb.timestamp_high = (uint32_t)(pkt_hdr->timestamp.u64 >> 32);
+   epb.timestamp_low = (uint32_t)(pkt_hdr->timestamp.u64);
+   epb.captured_len = pkt_len;
+   epb.packet_len = pkt_len;
+
+   /* epb */
+   packet_iov[iovcnt].iov_base = &epb;
+   packet_iov[iovcnt].iov_len = sizeof(epb);
+   iovcnt++;
+
+   /* data */
+   packet_iov[iovcnt].iov_base = buf;
+   packet_iov[iovcnt].iov_len =
+   ROUNDUP_ALIGN(pkt_len, PCAP_DATA_ALIGN);
+   iovcnt++;
+
+   /* trailing */
+   packet_iov[iovcnt].iov_base = &epb.block_total_length;
+   packet_iov[iovcnt].iov_len = sizeof(uint32_t);
+   iovcnt++;
+   }
+
+   /* we don't really care if we manage to write *all* data */
+   len = writev(fd, packet_iov, iovcnt);
+   if (!len)
+   ODP_ERR("Failed to write pcapng data\n");
+   fsync(fd);
+
+   return len;
+}
+
+static void pcapng_drain_fifo(int fd)
+{
+   char c;
+   ssize_t len;
+
+   do {
+   len = read(fd, &c, sizeof(c));
+   } while (len > 0 && len != -1);


Comment:
This is redundant. If `len > 0` then by definition `len != -1`.

> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
> This fails for multi-segment packets. `odp_packet_data(pkt)` is shorthand for 
> `odp_packet_offset(pkt, 0, NULL, NULL)` so you don't know how long the first 
> segment really is. First cut would be to only dump the first segment, so 
> correct code is:
> ```
> uint32_t seg_len;
> char *buf = (char *)odp_packet_offset(packets[i], 0, &seg_len, NULL);
> ```
> And use `seg_len` rather than `pkt_len` in this routine.


>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>> Does pcapng not supply a `.h` file for these "magic numbers" and associated 
>> structs?


>>> Ilias Apalodimas(apalos) wrote:
>>> Will fix. This will also remove the requirement of manually deleting the 
>>> fifos


 muvarov wrote
 odp_global_data_s->main_pid needs to be here.


https://github.com/Linaro/odp/pull/435#discussion_r165548254
updated_at 2018-02-02 03:09:47


Re: [lng-odp] [PATCH CATERPILLAR v1] [RFC] odp: pktio: added tcpdump like functionality

2018-02-02 Thread Github ODP bot
Bill Fischofer(Bill-Fischofer-Linaro) replied on github web page:

platform/linux-generic/odp_packet_io.c
line 137
@@ -89,6 +96,274 @@ int odp_pktio_init_local(void)
return odp_pktio_ops_init_local(true);
 }
 
+static int write_pcapng_hdr(pktio_entry_t *entry, int qidx)
+{
+   size_t len;
+   pcapng_section_hdr_block_t shb;
+   pcapng_interface_description_block_s idb;
+   int fd = entry->s.pcapng_info.pcapng_fd[qidx];
+
+   memset(&shb, 0, sizeof(shb));
+   memset(&idb, 0, sizeof(idb));
+
+   shb.block_type = PCAPNG_BLOCK_TYPE_SHB;
+   shb.block_total_length = sizeof(shb);
+   shb.block_total_length2 = sizeof(shb);
+   shb.magic = PCAPNG_ENDIAN_MAGIC;
+   shb.version_major = 0x1;
+   shb.version_minor = 0x0;
+   shb.section_len = -1;
+
+   len = write(fd, &shb, sizeof(shb));
+   fsync(fd);
+   /* fail to write shb/idb means the pcapng is unreadable */
+   if (!len) {
+   ODP_ERR("Failed to write pcapng section hdr\n");
+   close(fd);
+   return -1;
+   }
+
+   idb.block_type = PCAPNG_BLOCK_TYPE_IDB;
+   idb.block_total_length = sizeof(idb);
+   idb.block_total_length2 = sizeof(idb);
+   idb.linktype = 0x1; /* LINKTYPE_ETHERNET */
+   idb.snaplen = 0x0; /* unlimited */
+   len = write(fd, &idb, sizeof(idb));
+   if (!len) {
+   ODP_ERR("Failed to write pcapng interface description\n");
+   close(fd);
+   return -1;
+   }
+   fsync(fd);
+
+   entry->s.pcapng_info.state[qidx] = PCAPNG_WR_HDR;
+
+   return 0;
+}
+
+static int write_pcapng_pkts(pktio_entry_t *entry, int qidx,
+const odp_packet_t packets[], int num)
+{
+   int i = 0;
+   struct iovec packet_iov[3 * num];
+   pcapng_enhanced_packet_block_t epb;
+   int iovcnt = 0;
+   ssize_t len = 0;
+   int fd = entry->s.pcapng_info.pcapng_fd[qidx];
+
+   for (i = 0; i < num; i++) {
+   odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(packets[i]);
+   uint32_t pkt_len = _odp_packet_len(packets[i]);
+   char *buf = (char *)odp_packet_data(packets[i]);
+
+   epb.block_type = PCAPNG_BLOCK_TYPE_EPB;
+   epb.block_total_length = sizeof(epb) +
+   ROUNDUP_ALIGN(pkt_len, PCAP_DATA_ALIGN) +
+   PCAP_DATA_ALIGN;
+   epb.interface_idx = 0;
+   epb.timestamp_high = (uint32_t)(pkt_hdr->timestamp.u64 >> 32);
+   epb.timestamp_low = (uint32_t)(pkt_hdr->timestamp.u64);
+   epb.captured_len = pkt_len;
+   epb.packet_len = pkt_len;
+
+   /* epb */
+   packet_iov[iovcnt].iov_base = &epb;
+   packet_iov[iovcnt].iov_len = sizeof(epb);
+   iovcnt++;
+
+   /* data */
+   packet_iov[iovcnt].iov_base = buf;
+   packet_iov[iovcnt].iov_len =
+   ROUNDUP_ALIGN(pkt_len, PCAP_DATA_ALIGN);
+   iovcnt++;
+
+   /* trailing */
+   packet_iov[iovcnt].iov_base = &epb.block_total_length;
+   packet_iov[iovcnt].iov_len = sizeof(uint32_t);
+   iovcnt++;
+   }
+
+   /* we don't really care if we manage to write *all* data */
+   len = writev(fd, packet_iov, iovcnt);
+   if (!len)
+   ODP_ERR("Failed to write pcapng data\n");
+   fsync(fd);
+
+   return len;
+}
+
+static void pcapng_drain_fifo(int fd)
+{
+   char c;
+   ssize_t len;
+
+   do {
+   len = read(fd, &c, sizeof(c));
+   } while (len > 0 && len != -1);
+   ODP_DBG("Drain pcap fifo %d\n", len);
+}
+
+static void *inotify_update(void *arg)
+{
+   pktio_entry_t *entry = (pktio_entry_t *)arg;
+   struct timeval time;
+   int ret;
+   ssize_t rdlen;
+   int i;
+   char buffer[INOTIFY_BUF_LEN];
+   unsigned int max_queue =
+   MAX(entry->s.num_in_queue, entry->s.num_out_queue);
+   fd_set rfds;
+
+   while (1) {


Comment:
A few comments about what this routine is doing would be very helpful here. 
This is not easy code to follow. Modularizing it into some  `static inline` 
helpers might also make the logic simpler.

> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
> This is redundant. If `len > 0` then by definition `len != -1`.


>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>> This fails for multi-segment packets. `odp_packet_data(pkt)` is shorthand 
>> for `odp_packet_offset(pkt, 0, NULL, NULL)` so you don't know how long the 
>> first segment really is. First cut would be to only dump the first segment, 
>> so correct code is:
>> ```
>> uint32_t seg_len;
>> char *buf = (char *)odp_packet_offset(packets[i], 0, &seg_len, NULL);
>> ```
>> And use `seg_len` rather than `pkt_len` in this routine.


>>> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
>>> Does pcapng not suppl

Re: [lng-odp] [PATCH CATERPILLAR v1] [RFC] odp: pktio: added tcpdump like functionality

2018-02-02 Thread Github ODP bot
Bill Fischofer(Bill-Fischofer-Linaro) replied on github web page:

platform/linux-generic/odp_packet_io.c
line 76
@@ -89,6 +96,274 @@ int odp_pktio_init_local(void)
return odp_pktio_ops_init_local(true);
 }
 
+static int write_pcapng_hdr(pktio_entry_t *entry, int qidx)
+{
+   size_t len;
+   pcapng_section_hdr_block_t shb;
+   pcapng_interface_description_block_s idb;
+   int fd = entry->s.pcapng_info.pcapng_fd[qidx];
+
+   memset(&shb, 0, sizeof(shb));
+   memset(&idb, 0, sizeof(idb));
+
+   shb.block_type = PCAPNG_BLOCK_TYPE_SHB;
+   shb.block_total_length = sizeof(shb);
+   shb.block_total_length2 = sizeof(shb);
+   shb.magic = PCAPNG_ENDIAN_MAGIC;
+   shb.version_major = 0x1;
+   shb.version_minor = 0x0;
+   shb.section_len = -1;
+
+   len = write(fd, &shb, sizeof(shb));
+   fsync(fd);
+   /* fail to write shb/idb means the pcapng is unreadable */
+   if (!len) {
+   ODP_ERR("Failed to write pcapng section hdr\n");
+   close(fd);
+   return -1;
+   }
+
+   idb.block_type = PCAPNG_BLOCK_TYPE_IDB;
+   idb.block_total_length = sizeof(idb);
+   idb.block_total_length2 = sizeof(idb);
+   idb.linktype = 0x1; /* LINKTYPE_ETHERNET */
+   idb.snaplen = 0x0; /* unlimited */
+   len = write(fd, &idb, sizeof(idb));
+   if (!len) {
+   ODP_ERR("Failed to write pcapng interface description\n");
+   close(fd);
+   return -1;
+   }
+   fsync(fd);
+
+   entry->s.pcapng_info.state[qidx] = PCAPNG_WR_HDR;
+
+   return 0;
+}
+
+static int write_pcapng_pkts(pktio_entry_t *entry, int qidx,
+const odp_packet_t packets[], int num)
+{
+   int i = 0;
+   struct iovec packet_iov[3 * num];
+   pcapng_enhanced_packet_block_t epb;
+   int iovcnt = 0;
+   ssize_t len = 0;
+   int fd = entry->s.pcapng_info.pcapng_fd[qidx];
+
+   for (i = 0; i < num; i++) {
+   odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(packets[i]);
+   uint32_t pkt_len = _odp_packet_len(packets[i]);
+   char *buf = (char *)odp_packet_data(packets[i]);


Comment:
This fails for multi-segment packets. `odp_packet_data(pkt)` is shorthand for 
`odp_packet_offset(pkt, 0, NULL, NULL)` so you don't know how long the first 
segment really is. First cut would be to only dump the first segment, so 
correct code is:
```
uint32_t seg_len;
char *buf = (char *)odp_packet_offset(packets[i], 0, &seg_len, NULL);
```
And use `seg_len` rather than `pkt_len` in this routine.

> Bill Fischofer(Bill-Fischofer-Linaro) wrote:
> Does pcapng not supply a `.h` file for these "magic numbers" and associated 
> structs?


>> Ilias Apalodimas(apalos) wrote:
>> Will fix. This will also remove the requirement of manually deleting the 
>> fifos


>>> muvarov wrote
>>> odp_global_data_s->main_pid needs to be here.


https://github.com/Linaro/odp/pull/435#discussion_r165547875
updated_at 2018-02-02 03:09:47


Re: [lng-odp] [PATCH CATERPILLAR v1] [RFC] odp: pktio: added tcpdump like functionality

2018-02-02 Thread Github ODP bot
Bill Fischofer(Bill-Fischofer-Linaro) replied on github web page:

platform/linux-generic/include/odp_pcapng.h
line 20
@@ -0,0 +1,56 @@
+/* Copyright (c) 2018, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#define PCAPNG_BLOCK_TYPE_EPB 0x0006UL
+#define PCAPNG_BLOCK_TYPE_SHB 0x0A0D0D0AUL
+#define PCAPNG_BLOCK_TYPE_IDB 0x0001UL
+#define PCAPNG_ENDIAN_MAGIC 0x1A2B3C4DUL
+#define PCAP_DATA_ALIGN (4)


Comment:
Does pcapng not supply a `.h` file for these "magic numbers" and associated 
structs?

> Ilias Apalodimas(apalos) wrote:
> Will fix. This will also remove the requirement of manually deleting the fifos


>> muvarov wrote
>> odp_global_data_s->main_pid needs to be here.


https://github.com/Linaro/odp/pull/435#discussion_r165547037
updated_at 2018-02-02 03:09:47


Re: [lng-odp] [PATCH CATERPILLAR v1] [RFC] odp: pktio: added tcpdump like functionality

2018-01-31 Thread Github ODP bot
Ilias Apalodimas(apalos) replied on github web page:

platform/linux-generic/odp_packet_io.c
line 237
@@ -89,6 +96,274 @@ int odp_pktio_init_local(void)
return odp_pktio_ops_init_local(true);
 }
 
+static int write_pcapng_hdr(pktio_entry_t *entry, int qidx)
+{
+   size_t len;
+   pcapng_section_hdr_block_t shb;
+   pcapng_interface_description_block_s idb;
+   int fd = entry->s.pcapng_info.pcapng_fd[qidx];
+
+   memset(&shb, 0, sizeof(shb));
+   memset(&idb, 0, sizeof(idb));
+
+   shb.block_type = PCAPNG_BLOCK_TYPE_SHB;
+   shb.block_total_length = sizeof(shb);
+   shb.block_total_length2 = sizeof(shb);
+   shb.magic = PCAPNG_ENDIAN_MAGIC;
+   shb.version_major = 0x1;
+   shb.version_minor = 0x0;
+   shb.section_len = -1;
+
+   len = write(fd, &shb, sizeof(shb));
+   fsync(fd);
+   /* fail to write shb/idb means the pcapng is unreadable */
+   if (!len) {
+   ODP_ERR("Failed to write pcapng section hdr\n");
+   close(fd);
+   return -1;
+   }
+
+   idb.block_type = PCAPNG_BLOCK_TYPE_IDB;
+   idb.block_total_length = sizeof(idb);
+   idb.block_total_length2 = sizeof(idb);
+   idb.linktype = 0x1; /* LINKTYPE_ETHERNET */
+   idb.snaplen = 0x0; /* unlimited */
+   len = write(fd, &idb, sizeof(idb));
+   if (!len) {
+   ODP_ERR("Failed to write pcapng interface description\n");
+   close(fd);
+   return -1;
+   }
+   fsync(fd);
+
+   entry->s.pcapng_info.state[qidx] = PCAPNG_WR_HDR;
+
+   return 0;
+}
+
+static int write_pcapng_pkts(pktio_entry_t *entry, int qidx,
+const odp_packet_t packets[], int num)
+{
+   int i = 0;
+   struct iovec packet_iov[3 * num];
+   pcapng_enhanced_packet_block_t epb;
+   int iovcnt = 0;
+   ssize_t len = 0;
+   int fd = entry->s.pcapng_info.pcapng_fd[qidx];
+
+   for (i = 0; i < num; i++) {
+   odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(packets[i]);
+   uint32_t pkt_len = _odp_packet_len(packets[i]);
+   char *buf = (char *)odp_packet_data(packets[i]);
+
+   epb.block_type = PCAPNG_BLOCK_TYPE_EPB;
+   epb.block_total_length = sizeof(epb) +
+   ROUNDUP_ALIGN(pkt_len, PCAP_DATA_ALIGN) +
+   PCAP_DATA_ALIGN;
+   epb.interface_idx = 0;
+   epb.timestamp_high = (uint32_t)(pkt_hdr->timestamp.u64 >> 32);
+   epb.timestamp_low = (uint32_t)(pkt_hdr->timestamp.u64);
+   epb.captured_len = pkt_len;
+   epb.packet_len = pkt_len;
+
+   /* epb */
+   packet_iov[iovcnt].iov_base = &epb;
+   packet_iov[iovcnt].iov_len = sizeof(epb);
+   iovcnt++;
+
+   /* data */
+   packet_iov[iovcnt].iov_base = buf;
+   packet_iov[iovcnt].iov_len =
+   ROUNDUP_ALIGN(pkt_len, PCAP_DATA_ALIGN);
+   iovcnt++;
+
+   /* trailing */
+   packet_iov[iovcnt].iov_base = &epb.block_total_length;
+   packet_iov[iovcnt].iov_len = sizeof(uint32_t);
+   iovcnt++;
+   }
+
+   /* we don't really care if we manage to write *all* data */
+   len = writev(fd, packet_iov, iovcnt);
+   if (!len)
+   ODP_ERR("Failed to write pcapng data\n");
+   fsync(fd);
+
+   return len;
+}
+
+static void pcapng_drain_fifo(int fd)
+{
+   char c;
+   ssize_t len;
+
+   do {
+   len = read(fd, &c, sizeof(c));
+   } while (len > 0 && len != -1);
+   ODP_DBG("Drain pcap fifo %d\n", len);
+}
+
+static void *inotify_update(void *arg)
+{
+   pktio_entry_t *entry = (pktio_entry_t *)arg;
+   struct timeval time;
+   int ret;
+   ssize_t rdlen;
+   int i;
+   char buffer[INOTIFY_BUF_LEN];
+   unsigned int max_queue =
+   MAX(entry->s.num_in_queue, entry->s.num_out_queue);
+   fd_set rfds;
+
+   while (1) {
+   i = 0;
+   FD_ZERO(&rfds);
+   FD_SET(entry->s.pcapng_info.inotify_pcapng_fd, &rfds);
+   time.tv_sec = 2;
+   time.tv_usec = 0;
+   select(entry->s.pcapng_info.inotify_pcapng_fd + 1, &rfds, NULL,
+  NULL, &time);
+   if (FD_ISSET(entry->s.pcapng_info.inotify_pcapng_fd, &rfds)) {
+   rdlen = read(entry->s.pcapng_info.inotify_pcapng_fd,
+buffer, INOTIFY_BUF_LEN);
+   while (i < rdlen) {
+   char *e;
+   unsigned int qidx;
+   struct inotify_event *event =
+   (struct inotify_event *)(void *)
+&buffer[i];
+
+   e 

Re: [lng-odp] [PATCH CATERPILLAR v1] [RFC] odp: pktio: added tcpdump like functionality

2018-01-29 Thread Github ODP bot
muvarov replied on github web page:

platform/linux-generic/odp_packet_io.c
line 237
@@ -89,6 +96,274 @@ int odp_pktio_init_local(void)
return odp_pktio_ops_init_local(true);
 }
 
+static int write_pcapng_hdr(pktio_entry_t *entry, int qidx)
+{
+   size_t len;
+   pcapng_section_hdr_block_t shb;
+   pcapng_interface_description_block_s idb;
+   int fd = entry->s.pcapng_info.pcapng_fd[qidx];
+
+   memset(&shb, 0, sizeof(shb));
+   memset(&idb, 0, sizeof(idb));
+
+   shb.block_type = PCAPNG_BLOCK_TYPE_SHB;
+   shb.block_total_length = sizeof(shb);
+   shb.block_total_length2 = sizeof(shb);
+   shb.magic = PCAPNG_ENDIAN_MAGIC;
+   shb.version_major = 0x1;
+   shb.version_minor = 0x0;
+   shb.section_len = -1;
+
+   len = write(fd, &shb, sizeof(shb));
+   fsync(fd);
+   /* fail to write shb/idb means the pcapng is unreadable */
+   if (!len) {
+   ODP_ERR("Failed to write pcapng section hdr\n");
+   close(fd);
+   return -1;
+   }
+
+   idb.block_type = PCAPNG_BLOCK_TYPE_IDB;
+   idb.block_total_length = sizeof(idb);
+   idb.block_total_length2 = sizeof(idb);
+   idb.linktype = 0x1; /* LINKTYPE_ETHERNET */
+   idb.snaplen = 0x0; /* unlimited */
+   len = write(fd, &idb, sizeof(idb));
+   if (!len) {
+   ODP_ERR("Failed to write pcapng interface description\n");
+   close(fd);
+   return -1;
+   }
+   fsync(fd);
+
+   entry->s.pcapng_info.state[qidx] = PCAPNG_WR_HDR;
+
+   return 0;
+}
+
+static int write_pcapng_pkts(pktio_entry_t *entry, int qidx,
+const odp_packet_t packets[], int num)
+{
+   int i = 0;
+   struct iovec packet_iov[3 * num];
+   pcapng_enhanced_packet_block_t epb;
+   int iovcnt = 0;
+   ssize_t len = 0;
+   int fd = entry->s.pcapng_info.pcapng_fd[qidx];
+
+   for (i = 0; i < num; i++) {
+   odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(packets[i]);
+   uint32_t pkt_len = _odp_packet_len(packets[i]);
+   char *buf = (char *)odp_packet_data(packets[i]);
+
+   epb.block_type = PCAPNG_BLOCK_TYPE_EPB;
+   epb.block_total_length = sizeof(epb) +
+   ROUNDUP_ALIGN(pkt_len, PCAP_DATA_ALIGN) +
+   PCAP_DATA_ALIGN;
+   epb.interface_idx = 0;
+   epb.timestamp_high = (uint32_t)(pkt_hdr->timestamp.u64 >> 32);
+   epb.timestamp_low = (uint32_t)(pkt_hdr->timestamp.u64);
+   epb.captured_len = pkt_len;
+   epb.packet_len = pkt_len;
+
+   /* epb */
+   packet_iov[iovcnt].iov_base = &epb;
+   packet_iov[iovcnt].iov_len = sizeof(epb);
+   iovcnt++;
+
+   /* data */
+   packet_iov[iovcnt].iov_base = buf;
+   packet_iov[iovcnt].iov_len =
+   ROUNDUP_ALIGN(pkt_len, PCAP_DATA_ALIGN);
+   iovcnt++;
+
+   /* trailing */
+   packet_iov[iovcnt].iov_base = &epb.block_total_length;
+   packet_iov[iovcnt].iov_len = sizeof(uint32_t);
+   iovcnt++;
+   }
+
+   /* we don't really care if we manage to write *all* data */
+   len = writev(fd, packet_iov, iovcnt);
+   if (!len)
+   ODP_ERR("Failed to write pcapng data\n");
+   fsync(fd);
+
+   return len;
+}
+
+static void pcapng_drain_fifo(int fd)
+{
+   char c;
+   ssize_t len;
+
+   do {
+   len = read(fd, &c, sizeof(c));
+   } while (len > 0 && len != -1);
+   ODP_DBG("Drain pcap fifo %d\n", len);
+}
+
+static void *inotify_update(void *arg)
+{
+   pktio_entry_t *entry = (pktio_entry_t *)arg;
+   struct timeval time;
+   int ret;
+   ssize_t rdlen;
+   int i;
+   char buffer[INOTIFY_BUF_LEN];
+   unsigned int max_queue =
+   MAX(entry->s.num_in_queue, entry->s.num_out_queue);
+   fd_set rfds;
+
+   while (1) {
+   i = 0;
+   FD_ZERO(&rfds);
+   FD_SET(entry->s.pcapng_info.inotify_pcapng_fd, &rfds);
+   time.tv_sec = 2;
+   time.tv_usec = 0;
+   select(entry->s.pcapng_info.inotify_pcapng_fd + 1, &rfds, NULL,
+  NULL, &time);
+   if (FD_ISSET(entry->s.pcapng_info.inotify_pcapng_fd, &rfds)) {
+   rdlen = read(entry->s.pcapng_info.inotify_pcapng_fd,
+buffer, INOTIFY_BUF_LEN);
+   while (i < rdlen) {
+   char *e;
+   unsigned int qidx;
+   struct inotify_event *event =
+   (struct inotify_event *)(void *)
+&buffer[i];
+
+   e = strrchr(event->