[netsniff-ng] Re: flowtop: ping for fixes ...
On 07/13/2015 10:24 AM, Vadim Kochan wrote: Hi, After 1 week I decided to ping for some flowtop related fixes: http://article.gmane.org/gmane.linux.network.netsniff-ng/973 http://article.gmane.org/gmane.linux.network.netsniff-ng/974 Sorry for the long delay. I've applied both fixes and am going over the traffic counter series now. Thanks for your patience! and series about traffic counters: http://article.gmane.org/gmane.linux.network.netsniff-ng/978 Regards, -- You received this message because you are subscribed to the Google Groups netsniff-ng group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [netsniff-ng] [PATCH] flowtop: Fix flows disappearing
On 07/04/2015 09:18 PM, Vadim Kochan wrote: From: Vadim Kochan vadi...@gmail.com While removing flow which is pointed by 'head' then head is set to NULL and all the list disappears, so fixed by set removing flow next entry to list 'head'. Signed-off-by: Vadim Kochan vadi...@gmail.com Applied, thanks! -- You received this message because you are subscribed to the Google Groups netsniff-ng group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [netsniff-ng] [PATCH 0/5] flowtop: Add traffic accounting dump
On Mon, Jul 13, 2015 at 11:28:49AM +0200, Daniel Borkmann wrote: On 07/08/2015 12:20 PM, Vadim Kochan wrote: From: Vadim Kochan vadi...@gmail.com Added periodic (0.5s) dump of existing and visible flow to update packets bytes counters. Needs to clone nf_conntrack object for new added flow entry to update counters for this particular flow, instead of dump entire conntrack table. Counters are showed in human readable format in SI units. Also added showing count of existing valid flows. Added new sysctl module with helpers to easy set/get value from /proc/sys. I just tried out the result and I like it. With regards to the 1st sysctl patch, it's true that we have /proc/foo handling code spread across various sub-projects inside netsniff-ng. For your series, could you drop the 1st patch for now and integrate this directly? If later on your plan is to move *all* /proc/foo handling into a generic library code, I'm totally fine with that. In this series, it would not address other places however, and thus be a bit inconsistent. If I understood correctly, you asked to do not use separated sysctl module unless it will not be used in other places ? But these funcs can be used statically in flowtop.c so they then can be used later while sysctl refactoring ? Thanks a lot, Daniel -- You received this message because you are subscribed to the Google Groups netsniff-ng group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [netsniff-ng] [PATCH 0/5] flowtop: Add traffic accounting dump
On Mon, Jul 13, 2015 at 12:31:39PM +0300, Vadim Kochan wrote: On Mon, Jul 13, 2015 at 11:28:49AM +0200, Daniel Borkmann wrote: On 07/08/2015 12:20 PM, Vadim Kochan wrote: From: Vadim Kochan vadi...@gmail.com Added periodic (0.5s) dump of existing and visible flow to update packets bytes counters. Needs to clone nf_conntrack object for new added flow entry to update counters for this particular flow, instead of dump entire conntrack table. Counters are showed in human readable format in SI units. Also added showing count of existing valid flows. Added new sysctl module with helpers to easy set/get value from /proc/sys. I just tried out the result and I like it. With regards to the 1st sysctl patch, it's true that we have /proc/foo handling code spread across various sub-projects inside netsniff-ng. For your series, could you drop the 1st patch for now and integrate this directly? If later on your plan is to move *all* /proc/foo handling into a generic library code, I'm totally fine with that. In this series, it would not address other places however, and thus be a bit inconsistent. If I understood correctly, you asked to do not use separated sysctl module unless it will not be used in other places ? But these funcs can be used statically in flowtop.c so they then can be used later while sysctl refactoring ? Thanks a lot, Daniel OK I squashed sysctl commit with traffic counters dump commit and will resend series v2. Thanks for review, Vadim Kochan -- You received this message because you are subscribed to the Google Groups netsniff-ng group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [netsniff-ng] [PATCH 0/5] flowtop: Add traffic accounting dump
On 07/13/2015 12:09 PM, Vadim Kochan wrote: On Mon, Jul 13, 2015 at 12:31:39PM +0300, Vadim Kochan wrote: ... If I understood correctly, you asked to do not use separated sysctl module unless it will not be used in other places ? But these funcs can be used statically in flowtop.c so they then can be used later while sysctl refactoring ? Yes, would be good if we introduce such library bits, to also make consistent use of it across the project. OK I squashed sysctl commit with traffic counters dump commit and will resend series v2. Great, thanks! Btw, I think it would also be useful to have flowtop dump/list the current data set to the test console (like conntrack -L), so that an admin could do a one time snapshot dump and do some more analysis on the data afterwards. Thoughts? ;) Cheers, Daniel -- You received this message because you are subscribed to the Google Groups netsniff-ng group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[netsniff-ng] wrong number of packets filtered in
Hi I'm using netsniff-ng for sniffing UDP packets on a network. For testign the performance I tried to send packets using iperf with bandwidth 100M and length 250. When the capture was completed iperf says it send 28000 packets sent but netsniff-ng shows a lesser value and it also shows 0% packet dropped. Could you please help me solve this confusion on whats happening. I'm more intereted in checking if the sniffer can handle upto 100Mbits/s data rate on the network Regards Jishnu -- You received this message because you are subscribed to the Google Groups netsniff-ng group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[netsniff-ng] [PATCH v2 0/4] flowtop: Add traffic accounting dump
From: Vadim Kochan vadi...@gmail.com Added periodic dump of existing and visible flow to update packets bytes counters. Needs to clone nf_conntrack object for new added flow entry to update counters for this particular flow, instead of dump entire conntrack table. Counters are showed in human readable format in SI units. Also added showing count of existing valid flows. v2: 1) Get rid of separated sysctl.c module, sysctl helpers moved into flowtop.c 2) Decreased sleep before poll update counters to 300ms. 3) Increased sleep after refresh presenter to 200ms to give collector more time to update each flow. 4) Renamed collector_refresh_ct - collector_refresh_flows Vadim Kochan (4): flowtop: Refactor walking for each flow node by presenter flowtop: Add connection traffic accounting flowtop: Show total numbers of flows flowtop: Show flow bytes in human readable format flowtop.c | 408 -- 1 file changed, 290 insertions(+), 118 deletions(-) -- 2.4.2 -- You received this message because you are subscribed to the Google Groups netsniff-ng group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[netsniff-ng] [PATCH v2 2/4] flowtop: Add connection traffic accounting
Mark each flow if it is visible on the screen to know
if it is needed update traffic acct info.
Changed to use non blocking recv of nf conntrack events to update
traffic accounting.
Now nf_conntrack is cloned when new flow entry is added to send dump
request which is used to update traffic accounting info (packet, bytes).
Signed-off-by: Vadim Kochan vadi...@gmail.com
---
flowtop.c | 234 ++
1 file changed, 205 insertions(+), 29 deletions(-)
diff --git a/flowtop.c b/flowtop.c
index 5f24c71..55de258 100644
--- a/flowtop.c
+++ b/flowtop.c
@@ -23,6 +23,8 @@
#include urcu.h
#include libgen.h
#include inttypes.h
+#include poll.h
+#include fcntl.h
#include die.h
#include xmalloc.h
@@ -53,6 +55,8 @@ struct flow_entry {
struct flow_entry *next;
int inode;
unsigned int procnum;
+ bool is_visible;
+ struct nf_conntrack *ct;
};
struct flow_list {
@@ -81,6 +85,7 @@ static volatile sig_atomic_t sigint = 0;
static int what = INCLUDE_IPV4 | INCLUDE_IPV6 | INCLUDE_TCP, show_src = 0;
static struct flow_list flow_list;
static struct condlock collector_ready;
+static int nfct_acct_val = -1;
static const char *short_options = vhTUsDIS46u;
static const struct option long_options[] = {
@@ -217,6 +222,62 @@ static const struct nfct_filter_ipv6 filter_ipv6 = {
.mask = { 0x, 0x, 0x, 0x },
};
+#define SYS_PATH /proc/sys/
+
+static int sysctl_set_int(char *file, int value)
+{
+ char path[PATH_MAX];
+ char str[64];
+ ssize_t ret;
+ int fd;
+
+ path[0] = '\0';
+ strcat(path, SYS_PATH);
+ strncat(path, file, PATH_MAX - sizeof(SYS_PATH) - 1);
+
+ fd = open(path, O_WRONLY);
+ if (unlikely(fd 0))
+ return -1;
+
+ ret = snprintf(str, 63, %d, value);
+ if (ret 0) {
+ close(fd);
+ return -1;
+ }
+
+ ret = write(fd, str, strlen(str));
+
+ close(fd);
+ return ret = 0 ? -1 : 0;
+}
+
+static int sysctl_get_int(char *file, int *value)
+{
+ char path[PATH_MAX];
+ char str[64];
+ ssize_t ret;
+ int fd;
+
+ path[0] = '\0';
+ strcat(path, SYS_PATH);
+ strncat(path, file, PATH_MAX - sizeof(SYS_PATH) - 1);
+
+ fd = open(path, O_RDONLY);
+ if (fd 0)
+ return -1;
+
+ ret = read(fd, str, sizeof(str));
+ if (ret 0) {
+ *value = atoi(str);
+ ret = 0;
+ } else {
+ ret = -1;
+ }
+
+ close(fd);
+ return ret;
+}
+
static void signal_handler(int number)
{
switch (number) {
@@ -279,6 +340,9 @@ static inline struct flow_entry *flow_entry_xalloc(void)
static inline void flow_entry_xfree(struct flow_entry *n)
{
+ if (n-ct)
+ nfct_destroy(n-ct);
+
xfree(n);
}
@@ -292,6 +356,8 @@ static void flow_list_new_entry(struct flow_list *fl,
struct nf_conntrack *ct)
{
struct flow_entry *n = flow_entry_xalloc();
+ n-ct = nfct_clone(ct);
+
flow_entry_from_ct(n, ct);
flow_entry_get_extended(n);
@@ -335,22 +401,15 @@ static struct flow_entry *flow_list_find_prev_id(struct
flow_list *fl,
static void flow_list_update_entry(struct flow_list *fl,
struct nf_conntrack *ct)
{
- int do_ext = 0;
struct flow_entry *n;
n = flow_list_find_id(fl, nfct_get_attr_u32(ct, ATTR_ID));
if (n == NULL) {
- n = flow_entry_xalloc();
- do_ext = 1;
+ flow_list_new_entry(fl, ct);
+ return;
}
flow_entry_from_ct(n, ct);
- if (do_ext) {
- flow_entry_get_extended(n);
-
- rcu_assign_pointer(n-next, fl-head);
- rcu_assign_pointer(fl-head, n);
- }
}
static void flow_list_destroy_entry(struct flow_list *fl,
@@ -925,23 +984,31 @@ static void presenter_screen_update(WINDOW *screen,
struct flow_list *fl,
mvwprintw(screen, line, 2, (No active sessions!
Is netfilter running?));
- for (; n maxy 0; n = rcu_dereference(n-next)) {
+ for (; n; n = rcu_dereference(n-next)) {
+
+ if (maxy = 0)
+ goto skip;
if (presenter_get_port(n-port_src, n-port_dst, 0) == 53)
- continue;
+ goto skip;
if (presenter_flow_wrong_state(n))
- continue;
+ goto skip;
if (skip_lines 0) {
skip_lines--;
- continue;
+ goto skip;
}
presenter_screen_do_line(screen, n, line);
line++;
maxy -= (2 + 1 * show_src);
+ n-is_visible = true;
+
[netsniff-ng] [PATCH v2 1/4] flowtop: Refactor walking for each flow node by presenter
Change code to walk each flow by presenter to look more understandable.
Signed-off-by: Vadim Kochan vadi...@gmail.com
---
flowtop.c | 138 +-
1 file changed, 56 insertions(+), 82 deletions(-)
diff --git a/flowtop.c b/flowtop.c
index 9d1991a..5f24c71 100644
--- a/flowtop.c
+++ b/flowtop.c
@@ -146,17 +146,17 @@ static const char *const tcp_state2str[TCP_CONNTRACK_MAX]
= {
[TCP_CONNTRACK_SYN_SENT2] = SYN_SENT2,
};
-static const uint8_t tcp_states[] = {
- TCP_CONNTRACK_SYN_SENT,
- TCP_CONNTRACK_SYN_RECV,
- TCP_CONNTRACK_ESTABLISHED,
- TCP_CONNTRACK_FIN_WAIT,
- TCP_CONNTRACK_CLOSE_WAIT,
- TCP_CONNTRACK_LAST_ACK,
- TCP_CONNTRACK_TIME_WAIT,
- TCP_CONNTRACK_CLOSE,
- TCP_CONNTRACK_SYN_SENT2,
- TCP_CONNTRACK_NONE,
+static const bool tcp_states_show[TCP_CONNTRACK_MAX] = {
+ [TCP_CONNTRACK_SYN_SENT] = true,
+ [TCP_CONNTRACK_SYN_RECV] = true,
+ [TCP_CONNTRACK_ESTABLISHED] = true,
+ [TCP_CONNTRACK_FIN_WAIT] = true,
+ [TCP_CONNTRACK_CLOSE_WAIT] = true,
+ [TCP_CONNTRACK_LAST_ACK] = true,
+ [TCP_CONNTRACK_TIME_WAIT] = true,
+ [TCP_CONNTRACK_CLOSE] = true,
+ [TCP_CONNTRACK_SYN_SENT2] = true,
+ [TCP_CONNTRACK_NONE] = true,
};
static const char *const dccp_state2str[DCCP_CONNTRACK_MAX] = {
@@ -172,17 +172,17 @@ static const char *const
dccp_state2str[DCCP_CONNTRACK_MAX] = {
[DCCP_CONNTRACK_INVALID]= INVALID,
};
-static const uint8_t dccp_states[] = {
- DCCP_CONNTRACK_NONE,
- DCCP_CONNTRACK_REQUEST,
- DCCP_CONNTRACK_RESPOND,
- DCCP_CONNTRACK_PARTOPEN,
- DCCP_CONNTRACK_OPEN,
- DCCP_CONNTRACK_CLOSEREQ,
- DCCP_CONNTRACK_CLOSING,
- DCCP_CONNTRACK_TIMEWAIT,
- DCCP_CONNTRACK_IGNORE,
- DCCP_CONNTRACK_INVALID,
+static const uint8_t dccp_states_show[DCCP_CONNTRACK_MAX] = {
+ [DCCP_CONNTRACK_NONE] = true,
+ [DCCP_CONNTRACK_REQUEST] = true,
+ [DCCP_CONNTRACK_RESPOND] = true,
+ [DCCP_CONNTRACK_PARTOPEN] = true,
+ [DCCP_CONNTRACK_OPEN] = true,
+ [DCCP_CONNTRACK_CLOSEREQ] = true,
+ [DCCP_CONNTRACK_CLOSING] = true,
+ [DCCP_CONNTRACK_TIMEWAIT] = true,
+ [DCCP_CONNTRACK_IGNORE] = true,
+ [DCCP_CONNTRACK_INVALID] = true,
};
static const char *const sctp_state2str[SCTP_CONNTRACK_MAX] = {
@@ -196,15 +196,15 @@ static const char *const
sctp_state2str[SCTP_CONNTRACK_MAX] = {
[SCTP_CONNTRACK_SHUTDOWN_ACK_SENT] = SHUTDOWN_ACK_SENT,
};
-static const uint8_t sctp_states[] = {
- SCTP_CONNTRACK_NONE,
- SCTP_CONNTRACK_CLOSED,
- SCTP_CONNTRACK_COOKIE_WAIT,
- SCTP_CONNTRACK_COOKIE_ECHOED,
- SCTP_CONNTRACK_ESTABLISHED,
- SCTP_CONNTRACK_SHUTDOWN_SENT,
- SCTP_CONNTRACK_SHUTDOWN_RECD,
- SCTP_CONNTRACK_SHUTDOWN_ACK_SENT,
+static const uint8_t sctp_states_show[SCTP_CONNTRACK_MAX] = {
+ [SCTP_CONNTRACK_NONE] = true,
+ [SCTP_CONNTRACK_CLOSED] = true,
+ [SCTP_CONNTRACK_COOKIE_WAIT] = true,
+ [SCTP_CONNTRACK_COOKIE_ECHOED] = true,
+ [SCTP_CONNTRACK_ESTABLISHED] = true,
+ [SCTP_CONNTRACK_SHUTDOWN_SENT] = true,
+ [SCTP_CONNTRACK_SHUTDOWN_RECD] = true,
+ [SCTP_CONNTRACK_SHUTDOWN_ACK_SENT] = true,
};
static const struct nfct_filter_ipv4 filter_ipv4 = {
@@ -860,21 +860,21 @@ static void presenter_screen_do_line(WINDOW *screen,
struct flow_entry *n,
}
}
-static inline int presenter_flow_wrong_state(struct flow_entry *n, int state)
+static inline int presenter_flow_wrong_state(struct flow_entry *n)
{
int ret = 1;
switch (n-l4_proto) {
case IPPROTO_TCP:
- if (n-tcp_state == state)
+ if (tcp_states_show[n-tcp_state])
ret = 0;
break;
case IPPROTO_SCTP:
- if (n-sctp_state == state)
+ if (sctp_states_show[n-sctp_state])
ret = 0;
break;
case IPPROTO_DCCP:
- if (n-dccp_state == state)
+ if (dccp_states_show[n-dccp_state])
ret = 0;
break;
case IPPROTO_UDP:
@@ -891,28 +891,9 @@ static inline int presenter_flow_wrong_state(struct
flow_entry *n, int state)
static void presenter_screen_update(WINDOW *screen, struct flow_list *fl,
int skip_lines)
{
- int maxy;
- size_t i, j;
unsigned int line = 3;
struct flow_entry *n;
- uint8_t protocols[] = {
- IPPROTO_TCP,
- IPPROTO_DCCP,
- IPPROTO_SCTP,
- IPPROTO_UDP,
- IPPROTO_UDPLITE,
- IPPROTO_ICMP,
- IPPROTO_ICMPV6,
- };
- size_t protocol_state_size[] = {
- [IPPROTO_TCP] = array_size(tcp_states),
- [IPPROTO_DCCP] =
[netsniff-ng] [PATCH v2 3/4] flowtop: Show total numbers of flows
Count flows which might be showed and show this number
on the top status line.
Signed-off-by: Vadim Kochan vadi...@gmail.com
---
flowtop.c | 34 +++---
1 file changed, 19 insertions(+), 15 deletions(-)
diff --git a/flowtop.c b/flowtop.c
index 55de258..64fd824 100644
--- a/flowtop.c
+++ b/flowtop.c
@@ -950,6 +950,8 @@ static inline int presenter_flow_wrong_state(struct
flow_entry *n)
static void presenter_screen_update(WINDOW *screen, struct flow_list *fl,
int skip_lines)
{
+ int skip_left = skip_lines;
+ unsigned int flows = 0;
unsigned int line = 3;
struct flow_entry *n;
int maxy;
@@ -968,15 +970,6 @@ static void presenter_screen_update(WINDOW *screen, struct
flow_list *fl,
wclear(screen);
clear();
- mvwprintw(screen, 1, 2, Kernel netfilter flows for %s%s%s%s%s%s
- [+%d], what INCLUDE_TCP ? TCP, : ,
- what INCLUDE_UDP ? UDP, : ,
- what INCLUDE_SCTP ? SCTP, : ,
- what INCLUDE_DCCP ? DCCP, : ,
- what INCLUDE_ICMP what INCLUDE_IPV4 ? ICMP, : ,
- what INCLUDE_ICMP what INCLUDE_IPV6 ? ICMP6, : ,
- skip_lines);
-
rcu_read_lock();
n = rcu_dereference(fl-head);
@@ -985,18 +978,20 @@ static void presenter_screen_update(WINDOW *screen,
struct flow_list *fl,
Is netfilter running?));
for (; n; n = rcu_dereference(n-next)) {
-
- if (maxy = 0)
- goto skip;
-
if (presenter_get_port(n-port_src, n-port_dst, 0) == 53)
goto skip;
if (presenter_flow_wrong_state(n))
goto skip;
- if (skip_lines 0) {
- skip_lines--;
+ /* count only flows which might be showed */
+ flows++;
+
+ if (maxy = 0)
+ goto skip;
+
+ if (skip_left 0) {
+ skip_left--;
goto skip;
}
@@ -1011,6 +1006,15 @@ skip:
continue;
}
+ mvwprintw(screen, 1, 2, Kernel netfilter flows(%u) for %s%s%s%s%s%s
+ [+%d], flows, what INCLUDE_TCP ? TCP, : ,
+ what INCLUDE_UDP ? UDP, : ,
+ what INCLUDE_SCTP ? SCTP, : ,
+ what INCLUDE_DCCP ? DCCP, : ,
+ what INCLUDE_ICMP what INCLUDE_IPV4 ? ICMP, : ,
+ what INCLUDE_ICMP what INCLUDE_IPV6 ? ICMP6, : ,
+ skip_lines);
+
rcu_read_unlock();
wrefresh(screen);
--
2.4.2
--
You received this message because you are subscribed to the Google Groups
netsniff-ng group.
To unsubscribe from this group and stop receiving emails from it, send an email
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [netsniff-ng] [PATCH 0/5] flowtop: Add traffic accounting dump
On Mon, Jul 13, 2015 at 12:41:10PM +0200, Daniel Borkmann wrote: On 07/13/2015 12:09 PM, Vadim Kochan wrote: On Mon, Jul 13, 2015 at 12:31:39PM +0300, Vadim Kochan wrote: ... If I understood correctly, you asked to do not use separated sysctl module unless it will not be used in other places ? But these funcs can be used statically in flowtop.c so they then can be used later while sysctl refactoring ? Yes, would be good if we introduce such library bits, to also make consistent use of it across the project. OK I squashed sysctl commit with traffic counters dump commit and will resend series v2. Great, thanks! Btw, I think it would also be useful to have flowtop dump/list the current data set to the test console (like conntrack -L), so that an admin could do a one time snapshot dump and do some more analysis on the data afterwards. Yes, but in that case NFCT_Q_FLUSH should not be used. Thoughts? ;) There are a lot of thoughts ... like: 1) Print counters in separated column and highlight flow entry depend on odd order number. 2) Print traffic rate if possible (needs to look into timestamp info). 3) Sort by maximum rate/traffic amount/process name. 4) Group counters by process name/pid as one entry. 5) Additional move like: - jump over 1 page (Ctlr-U/Ctrl-D like in VIM ?). - jump to the top (gg ?). Cheers, Daniel -- You received this message because you are subscribed to the Google Groups netsniff-ng group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[netsniff-ng] flowtop: ping for fixes ...
Hi, After 1 week I decided to ping for some flowtop related fixes: http://article.gmane.org/gmane.linux.network.netsniff-ng/973 http://article.gmane.org/gmane.linux.network.netsniff-ng/974 and series about traffic counters: http://article.gmane.org/gmane.linux.network.netsniff-ng/978 Regards, -- You received this message because you are subscribed to the Google Groups netsniff-ng group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
