Josh, After thinking about this, I did have issues with reading the FLOW files generated by NTOP with flow tools.
Here's the error: flow-cat: ftiheader_read(): Warning, bad magic number flow-cat: ftiheader_read(1270982473.flow): Failed, ignoring file. Does this look familiar? Below is information on my build of NTOP. I'll be happy to provide more information or work on solving this puzzle, if there's someone with a more in depth knowledge of how this works. I ended up using a Cisco monitor port and using tcpdump to grab 40Gig of raw data from a 24 hour period, and then culling it with Python. Kind Regards, -Chris Welcome to ntop v.3.4-pre2 (32 bit) [Configured on Mar 4 2010 19:48:49, built on Mar 4 2010 19:49:28] Copyright 1998-2010 by Luca Deri <[email protected]>. Get the freshest ntop from http://www.ntop.org/ *config.log:* build_os='linux-gnu' build_vendor='suse' datadir='${datarootdir}' datarootdir='/usr/local/share' docdir='${datarootdir}/doc/${PACKAGE_TARNAME}' dvidir='${docdir}' exec_prefix='${prefix}' host='i686-suse-linux-gnu' host_alias='' host_cpu='i686' host_os='linux-gnu' host_vendor='suse' htmldir='${docdir}' includedir='${prefix}/include' infodir='${datarootdir}/info' install_sh='$(SHELL) /home/cleonardos/ntop-3.4-pre2/install-sh' libdir='${exec_prefix}/lib' libexecdir='${exec_prefix}/libexec' localedir='${datarootdir}/locale' localstatedir='${prefix}/var' lt_ECHO='echo' mandir='${datarootdir}/man' mkdir_p='/bin/mkdir -p' oldincludedir='/usr/include' pdfdir='${docdir}' prefix='/usr/local' program_transform_name='s,x,x,' psdir='${docdir}' sbindir='${exec_prefix}/sbin' sharedstatedir='${prefix}/com' sysconfdir='${prefix}/etc' target='i686-suse-linux-gnu' target_alias='' target_cpu='i686' target_os='linux-gnu' target_vendor='suse' ## ----------- ## ## confdefs.h. ## ## ----------- ## #define PACKAGE_NAME "ntop" #define PACKAGE_TARNAME "ntop" #define PACKAGE_VERSION "3.4-pre2" #define PACKAGE_STRING "ntop 3.4-pre2" #define PACKAGE_BUGREPORT "" #define PACKAGE "ntop" #define VERSION "3.4-pre2" #define STDC_HEADERS 1 #define HAVE_SYS_TYPES_H 1 #define HAVE_SYS_STAT_H 1 #define HAVE_STDLIB_H 1 #define HAVE_STRING_H 1 #define HAVE_MEMORY_H 1 #define HAVE_STRINGS_H 1 #define HAVE_INTTYPES_H 1 #define HAVE_STDINT_H 1 #define HAVE_UNISTD_H 1 #define HAVE_DLFCN_H 1 #define LT_OBJDIR ".libs/" #define HAVE_PCAP_NEXT_EX 1 #define HAVE_GDBM_H 1 #define HAVE_LIBGDBM 1 #define HAVE_LONG_DOUBLE_WIDER 1 #define HAVE_LONG_DOUBLE 1 #define CFG_LITTLE_ENDIAN 1 #define STDC_HEADERS 1 #define HAVE_DIRENT_H 1 #define HAVE_SYS_WAIT_H 1 #define TIME_WITH_SYS_TIME 1 #define HAVE_FLOAT_H 1 #define HAVE_STDDEF_H 1 #define HAVE_STDLIB_H 1 #define HAVE_STRING_H 1 #define HAVE_ERRNO_H 1 #define HAVE_FCNTL_H 1 #define HAVE_LIMITS_H 1 #define HAVE_MATH_H 1 #define HAVE_SIGNAL_H 1 #define HAVE_STDARG_H 1 #define HAVE_UNISTD_H 1 #define HAVE_STDIO_H 1 #define HAVE_STRINGS_H 1 #define HAVE_SYS_IOCTL_H 1 #define HAVE_SYS_SOCKET_H 1 #define HAVE_SYS_TIME_H 1 #define HAVE_SYS_TYPES_H 1 #define HAVE_SETJMP_H 1 #define HAVE_SHADOW_H 1 #define HAVE_SYS_UTSNAME_H 1 #define HAVE_NETINET_IN_H 1 #define HAVE_ARPA_INET_H 1 #define HAVE_ARPA_NAMESER_H 1 #define HAVE_NET_ETHERNET_H 1 #define HAVE_LIBZ 1 #define HAVE_LIBRRD_TH 1 #define HAVE_NET_IF_H 1 #define HAVE_NETINET_IF_ETHER_H 1 #define HAVE_NETINET_IN_SYSTM_H 1 #define HAVE_NETINET_IP_H 1 #define HAVE_NETINET_IP_ICMP_H 1 #define HAVE_NETINET_TCP_H 1 #define HAVE_NETINET_UDP_H 1 #define HAVE_SYS_PARAM_H 1 #define HAVE_SYS_SYSCTL_H 1 #define HAVE_NET_ROUTE_H 1 #define HAVE_NET_PPP_DEFS_H 1 #define HAVE_CRYPT_H 1 #define HAVE_PWD_H 1 #define HAVE_SHADOW_H 1 #define HAVE_DIRENT_H 1 #define HAVE_DLFCN_H 1 #define HAVE_GETOPT_H 1 #define HAVE_INTTYPES_H 1 #define HAVE_MEMORY_H 1 #define HAVE_SYS_PARAM_H 1 #define HAVE_SYS_SELECT_H 1 #define HAVE_SYS_STAT_H 1 #define HAVE_SYS_UN_H 1 #define HAVE_SYS_WAIT_H 1 #define HAVE_ZLIB_H 1 #define HAVE_SCHED_H 1 #define HAVE_PTHREAD_H 1 #define HAVE_RW_LOCK 1 #define HAVE_SYS_SYSLOG_H 1 #define HAVE_SYSLOG_H 1 #define INET6 1 #define HAVE_NETINET_IP6_H 1 #define HAVE_NETINET_ICMP6_H 1 #define HAVE_SYSCTL 1 #define HAVE_FINITE 1 #define HAVE_ISINF 1 #define TIME_WITH_SYS_TIME 1 #define HAVE_STRUCT_TM_TM_ZONE 1 #define HAVE_TM_ZONE 1 #define HAVE_U_INT64_T 1 #define HAVE_U_INT32_T 1 #define HAVE_U_INT16_T 1 #define HAVE_U_INT8_T 1 #define HAVE_INT64_T 1 #define HAVE_INT32_T 1 #define HAVE_INT16_T 1 #define HAVE_INT8_T 1 #define HAVE_LIBC 1 #define HAVE_LIBCRYPT 1 #define HAVE_LIBC 1 #define HAVE_LIBC 1 #define HAVE_DLADDR 1 #define HAVE_LIBC 1 #define HAVE_LIBC 1 #define HAVE_LIBPTHREAD 1 #define HAVE_FORK 1 #define HAVE_VFORK 1 #define HAVE_WORKING_VFORK 1 #define HAVE_WORKING_FORK 1 #define RETSIGTYPE void #define LSTAT_FOLLOWS_SLASHED_SYMLINK 1 #define HAVE_STRFTIME 1 #define HAVE_ALARM 1 #define HAVE_ENDPWENT 1 #define HAVE_GETHOSTBYADDR 1 #define HAVE_GETHOSTBYNAME 1 #define HAVE_GETHOSTNAME 1 #define HAVE_GETHOSTBYADDR_R 1 #define HAVE_GETPASS 1 #define HAVE_GETTIMEOFDAY 1 #define HAVE_LOCALTIME_R 1 #define HAVE_MEMCHR 1 #define HAVE_MEMSET 1 #define HAVE_PUTENV 1 #define HAVE_SELECT 1 #define HAVE_SOCKET 1 #define HAVE_SNPRINTF 1 #define HAVE_SQRTF 1 #define HAVE_STRCASECMP 1 #define HAVE_STRNCASECMP 1 #define HAVE_STRCASESTR 1 #define HAVE_STRCHR 1 #define HAVE_STRRCHR 1 #define HAVE_STRCSPN 1 #define HAVE_STRDUP 1 #define HAVE_STRERROR 1 #define HAVE_STRPBRK 1 #define HAVE_STRSIGNAL 1 #define HAVE_STRSPN 1 #define HAVE_STRSTR 1 #define HAVE_STRTOUL 1 #define HAVE_UNAME 1 #define HAVE_STRTOK_R 1 #define MAKE_WITH_ZLIB 1 #define HAVE_PYTHON 1 #define HAVE_PTHREAD_ATFORK 1 #define HAVE_BACKTRACE 1 #define HAVE_GETOPT_LONG 1 #define HAVE_FACILITYNAMES 1 #define HAVE_IN6_ADDR 1 #define RETSIGTYPE void #define HAVE_GEOIP 1 #define CFG_DATAFILE_DIR "/usr/local/share/ntop" #define CFG_CONFIGFILE_DIR "/usr/local/etc/ntop" #define CFG_RUN_DIR "/usr/local/var/ntop" #define CFG_PLUGIN_DIR "/usr/local/lib/ntop/plugins" #define CFG_DBFILE_DIR "/usr/local/var/ntop" configure: exit 0 ## ---------------------- ## ## Running config.status. ## ## ---------------------- ## This file was extended by ntop config.status 3.4-pre2, which was generated by GNU Autoconf 2.63. Invocation command line was CONFIG_FILES = CONFIG_HEADERS = CONFIG_LINKS = CONFIG_COMMANDS = $ ./config.status config.h on netmon1 config.status:1155: creating config.h config.status:1377: config.h is unchanged On Tue, Apr 13, 2010 at 9:16 AM, josh summitt <[email protected]> wrote: > Yea i have flow tools and silk and a few others but none of them can make > sense out of the flow dumps that the nTop Netflow plugin generates. I read > something that said ntop generates netflow v5 dump files. Every tool i've > used to translate V5 netflow fails on these files. I read something else > that said these dump files are in a gnu db format or mySQL. > > I'm using nTop just as a netflow collector middle man so that i can get the > netflow data into the analytic software we are using here. Is there a better > way to collect netflow that will run on windows and support netflow V1-9 and > IPFIX? I would prefer to have the data in a csv format. > > > Thanks > Josh > > > > On Mon, Apr 12, 2010 at 8:33 PM, Gary Gatten <[email protected]> wrote: > >> I *think* there are several different "dumps". IIRC there is a dump >> and/or debug option that basically copies the flow records to a disk file as >> they're received and look just like netflow flows. >> >> ------------------------------ >> *From*: [email protected] < >> [email protected]> >> *To*: [email protected] <[email protected]> >> *Sent*: Mon Apr 12 20:03:17 2010 >> *Subject*: Re: [Ntop] Neflow dump format question. >> >> Josh, >> >> I ran into this problem recently and tried to get the open source >> flow-tools to compile on Open Suse 11.1 but was unable to get some of the >> prereq's to compile properly most notably the pypcap python module. >> >> Here's a good list of open source tools, including flow-tools. >> >> http://www.networkuptime.com/tools/netflow/ >> >> -Chris >> >> On Mon, Apr 12, 2010 at 3:00 PM, josh summitt <[email protected]> wrote: >> >>> I've been searching the forums and internet for the last few days and >>> have not found the answer so hopefully someone can answer this for me. When >>> using the Netflow plugin, what format are the netflow data dumps in? I need >>> to take netflow data and import it into analytic software that we are using. >>> What tool do i need to read this data and extract it to another system? >>> >>> >>> Thanks >>> Josh >>> >>> _______________________________________________ >>> Ntop mailing list >>> [email protected] >>> http://listgateway.unipi.it/mailman/listinfo/ntop >>> >>> >> >> _______________________________________________ >> Ntop mailing list >> [email protected] >> http://listgateway.unipi.it/mailman/listinfo/ntop >> >> > > _______________________________________________ > Ntop mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop > >
_______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop
