Hello community, here is the log from the commit of package fstrm for openSUSE:Factory checked in at 2018-01-09 14:36:12 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/fstrm (Old) and /work/SRC/openSUSE:Factory/.fstrm.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "fstrm" Tue Jan 9 14:36:12 2018 rev:3 rq:561491 version:0.3.2 Changes: -------- --- /work/SRC/openSUSE:Factory/fstrm/fstrm.changes 2015-09-11 09:04:51.000000000 +0200 +++ /work/SRC/openSUSE:Factory/.fstrm.new/fstrm.changes 2018-01-09 14:36:14.079516927 +0100 @@ -1,0 +2,21 @@ +Tue Jan 2 15:59:59 UTC 2018 - [email protected] + +- Compact description. Fix RPM groups. + +------------------------------------------------------------------- +Mon Jan 1 07:21:18 UTC 2018 - [email protected] + +- update to 0.3.2 + * Accommodate systems without pthread_condattr_setclock +- 0.3.1 + * Add support for '-' as a filename for stdin/stdout + * Destroy condition variable and mutexes in fstrm_iothr_destroy() +- 0.3.0 + * Output file rotation + * Change state properly in fstrm_writer_close() + * force output flush on sighup + * Build on OS X +- cleanup with spec-cleaner +- use dist tarball instead of github archive + +------------------------------------------------------------------- Old: ---- fstrm-0.2.0.tar.gz New: ---- fstrm-0.3.2.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ fstrm.spec ++++++ --- /var/tmp/diff_new_pack.jwa9qE/_old 2018-01-09 14:36:14.979474712 +0100 +++ /var/tmp/diff_new_pack.jwa9qE/_new 2018-01-09 14:36:14.983474525 +0100 @@ -1,7 +1,7 @@ # # spec file for package fstrm # -# Copyright (c) 2015 SUSE LINUX GmbH, Nuernberg, Germany. +# Copyright (c) 2018 SUSE LINUX GmbH, Nuernberg, Germany. # # All modifications and additions to the file contributed by third parties # remain the property of their copyright owners, unless otherwise agreed @@ -18,43 +18,37 @@ %define c_lib libfstrm0 Name: fstrm -Version: 0.2.0 +Version: 0.3.2 Release: 0 Summary: Frame Streams implementation in C License: Apache-2.0 -Group: System/Libraries +Group: Development/Libraries/C and C++ Url: https://github.com/farsightsec/fstrm -Source: https://github.com/farsightsec/fstrm/releases/download/v%{version}/%{name}-%{version}.tar.gz -BuildRequires: libevent-devel -BuildRequires: pkg-config +Source: https://dl.farsightsecurity.com/dist/%{name}/%{name}-%{version}.tar.gz BuildRequires: libevent-devel >= 2 -BuildRoot: %{_tmppath}/%{name}-%{version}-build +BuildRequires: pkgconfig %description -This is fstrm, a C implementation of the Frame Streams data transport protocol. - -Frame Streams is a light weight, binary clean protocol that allows for the -transport of arbitrarily encoded data payload sequences with minimal framing -overhead -- just four bytes per data frame. Frame Streams does not specify an -encoding format for data frames and can be used with any data serialization -format that produces byte sequences, such as Protocol Buffers, XML, JSON, -MessagePack, YAML, etc. Frame Streams can be used as both a streaming transport -over a reliable byte stream socket (TCP sockets, TLS connections, AF_UNIX -sockets, etc.) for data in motion as well as a file format for data at rest. A -"Content Type" header identifies the type of payload being carried over an -individual Frame Stream and allows cooperating programs to determine how to -interpret a given sequence of data payloads. - -fstrm is an optimized C implementation of Frame Streams that includes a fast, -lockless circular queue implementation and exposes library interfaces for -setting up a dedicated Frame Streams I/O thread and asynchronously submitting -data frames for transport from worker threads. It was originally written to -facilitate the addition of high speed binary logging to DNS servers written in -C using the dnstap log format. +fstrm is a C implementation of Frame Streams that includes a lockless +circular queue implementation and exposes library interfaces for +setting up a dedicated Frame Streams I/O thread and asynchronously +submitting data frames for transport from worker threads. + +Frame Streams is a protocol that allows for the transport of +arbitrarily encoded data payload sequences with just 4 bytes per data +frame. Frame Streams does not specify an encoding format for frames +and can be used with data serialization formats that produces byte +sequences, such as Protocol Buffers, XML, JSON, MessagePack, YAML, +etc. Frame Streams can be used both as a streaming transport over a +reliable byte stream socket (TCP, AF_UNIX, TLS, etc.) for data in +motion, as well as a file format for data at rest. A "Content Type" +header identifies the type of payload being carried over an +individual Frame Stream and allows cooperating programs to determine +how to interpret a given sequence of data payloads. %package -n %{c_lib} -Summary: Frame Streams implementation in C - Shared libary file -Group: Development/Libraries/C and C++ +Summary: Frame Streams implementation in C +Group: System/Libraries %description -n %{c_lib} This is fstrm, a C implementation of the Frame Streams data transport protocol. @@ -62,7 +56,7 @@ This packages holds the shared library file. %package devel -Summary: Frame Streams implementation in C - Development files +Summary: Development files for fstrm, a Frame Streams implementation in C Group: Development/Libraries/C and C++ Requires: %{c_lib} = %{version} Provides: libfstrm-devel = %{version}-%{release} @@ -80,25 +74,21 @@ make %{?_smp_mflags} %install -make DESTDIR=%{buildroot} install %{?_smp_mflags} +%make_install rm %{buildroot}%{_libdir}/libfstrm.la %post -n %{c_lib} -p /sbin/ldconfig - %postun -n %{c_lib} -p /sbin/ldconfig %files -%defattr(-,root,root) %doc ChangeLog README* COPYRIGHT LICENSE %{_bindir}/fstrm_capture %{_bindir}/fstrm_dump %files -n %{c_lib} -%defattr(-,root,root) %{_libdir}/libfstrm.so.* %files devel -%defattr(-,root,root) %{_includedir}/fstrm.h %{_includedir}/fstrm/ %{_libdir}/libfstrm.so ++++++ fstrm-0.2.0.tar.gz -> fstrm-0.3.2.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/fstrm-0.2.0/ChangeLog new/fstrm-0.3.2/ChangeLog --- old/fstrm-0.2.0/ChangeLog 2014-11-07 01:19:51.000000000 +0100 +++ new/fstrm-0.3.2/ChangeLog 2017-04-10 15:51:04.000000000 +0200 @@ -1,3 +1,29 @@ +fstrm (0.3.2) + + * Accomodate systems without pthread_condattr_setclock (Issue #34) + + -- Chris Mikkelson <[email protected]> Tue, 21 Mar 2017 19:15:11 -0500 + +fstrm (0.3.1) + + * Add support for '-' as a filename for stdin/stdout (PR #28) + + * destroy condition variable and mutexes in fstrm_iothr_destroy() (PR #25) + + -- Chris Mikkelson <[email protected]> Thu, 26 Jan 2017 16:05:06 -0600 + +fstrm (0.3.0) + + * Output file rotation (Issue #22). + + * Build on OS X (Issue #21). + + * Change state properly in fstrm_writer_close() (Issue #18). + + * force output flush on sighup (Issue #12). + + -- Ben April <[email protected]> Fri, 24 Jun 2016 22:29:56 -0500 + fstrm (0.2.0) * Fix fstrm_writer_open() to allow multiple opens (Issue #1). diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/fstrm-0.2.0/Makefile.in new/fstrm-0.3.2/Makefile.in --- old/fstrm-0.2.0/Makefile.in 2014-11-07 01:21:31.000000000 +0100 +++ new/fstrm-0.3.2/Makefile.in 2017-04-10 15:51:31.000000000 +0200 @@ -1724,8 +1724,8 @@ maintainer-clean-generic: @echo "This command is intended for maintainers to use" @echo "it deletes files that may require special tools to rebuild." -@HAVE_DOXYGEN_FALSE@clean-local: @HAVE_DOXYGEN_FALSE@html-local: +@HAVE_DOXYGEN_FALSE@clean-local: clean: clean-am clean-am: clean-binPROGRAMS clean-checkPROGRAMS clean-generic \ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/fstrm-0.2.0/aclocal.m4 new/fstrm-0.3.2/aclocal.m4 --- old/fstrm-0.2.0/aclocal.m4 2014-11-07 01:21:30.000000000 +0100 +++ new/fstrm-0.3.2/aclocal.m4 2017-04-10 15:51:29.000000000 +0200 @@ -103,10 +103,9 @@ # configured tree to be moved without reconfiguration. AC_DEFUN([AM_AUX_DIR_EXPAND], -[dnl Rely on autoconf to set up CDPATH properly. -AC_PREREQ([2.50])dnl -# expand $ac_aux_dir to an absolute path -am_aux_dir=`cd $ac_aux_dir && pwd` +[AC_REQUIRE([AC_CONFIG_AUX_DIR_DEFAULT])dnl +# Expand $ac_aux_dir to an absolute path. +am_aux_dir=`cd "$ac_aux_dir" && pwd` ]) # AM_COND_IF -*- Autoconf -*- diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/fstrm-0.2.0/build-aux/config.sub new/fstrm-0.3.2/build-aux/config.sub --- old/fstrm-0.2.0/build-aux/config.sub 2014-11-07 01:21:31.000000000 +0100 +++ new/fstrm-0.3.2/build-aux/config.sub 2017-04-10 15:51:31.000000000 +0200 @@ -2,7 +2,7 @@ # Configuration validation subroutine script. # Copyright 1992-2014 Free Software Foundation, Inc. -timestamp='2014-05-01' +timestamp='2014-09-11' # This file is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by @@ -302,6 +302,7 @@ | pdp10 | pdp11 | pj | pjl \ | powerpc | powerpc64 | powerpc64le | powerpcle \ | pyramid \ + | riscv32 | riscv64 \ | rl78 | rx \ | score \ | sh | sh[1234] | sh[24]a | sh[24]aeb | sh[23]e | sh[34]eb | sheb | shbe | shle | sh[1234]le | sh3ele \ @@ -828,6 +829,10 @@ basic_machine=powerpc-unknown os=-morphos ;; + moxiebox) + basic_machine=moxie-unknown + os=-moxiebox + ;; msdos) basic_machine=i386-pc os=-msdos @@ -1373,7 +1378,7 @@ | -cygwin* | -msys* | -pe* | -psos* | -moss* | -proelf* | -rtems* \ | -mingw32* | -mingw64* | -linux-gnu* | -linux-android* \ | -linux-newlib* | -linux-musl* | -linux-uclibc* \ - | -uxpv* | -beos* | -mpeix* | -udk* \ + | -uxpv* | -beos* | -mpeix* | -udk* | -moxiebox* \ | -interix* | -uwin* | -mks* | -rhapsody* | -darwin* | -opened* \ | -openstep* | -oskit* | -conix* | -pw32* | -nonstopux* \ | -storm-chaos* | -tops10* | -tenex* | -tops20* | -its* \ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/fstrm-0.2.0/build-aux/install-sh new/fstrm-0.3.2/build-aux/install-sh --- old/fstrm-0.2.0/build-aux/install-sh 2014-11-07 01:21:31.000000000 +0100 +++ new/fstrm-0.3.2/build-aux/install-sh 2017-04-10 15:51:31.000000000 +0200 @@ -345,34 +345,41 @@ # is incompatible with FreeBSD 'install' when (umask & 300) != 0. ;; *) + # $RANDOM is not portable (e.g. dash); use it when possible to + # lower collision chance tmpdir=${TMPDIR-/tmp}/ins$RANDOM-$$ - trap 'ret=$?; rmdir "$tmpdir/d" "$tmpdir" 2>/dev/null; exit $ret' 0 + trap 'ret=$?; rmdir "$tmpdir/a/b" "$tmpdir/a" "$tmpdir" 2>/dev/null; exit $ret' 0 + # As "mkdir -p" follows symlinks and we work in /tmp possibly; so + # create the $tmpdir first (and fail if unsuccessful) to make sure + # that nobody tries to guess the $tmpdir name. if (umask $mkdir_umask && - exec $mkdirprog $mkdir_mode -p -- "$tmpdir/d") >/dev/null 2>&1 + $mkdirprog $mkdir_mode "$tmpdir" && + exec $mkdirprog $mkdir_mode -p -- "$tmpdir/a/b") >/dev/null 2>&1 then if test -z "$dir_arg" || { # Check for POSIX incompatibilities with -m. # HP-UX 11.23 and IRIX 6.5 mkdir -m -p sets group- or # other-writable bit of parent directory when it shouldn't. # FreeBSD 6.1 mkdir -m -p sets mode of existing directory. - ls_ld_tmpdir=`ls -ld "$tmpdir"` + test_tmpdir="$tmpdir/a" + ls_ld_tmpdir=`ls -ld "$test_tmpdir"` case $ls_ld_tmpdir in d????-?r-*) different_mode=700;; d????-?--*) different_mode=755;; *) false;; esac && - $mkdirprog -m$different_mode -p -- "$tmpdir" && { - ls_ld_tmpdir_1=`ls -ld "$tmpdir"` + $mkdirprog -m$different_mode -p -- "$test_tmpdir" && { + ls_ld_tmpdir_1=`ls -ld "$test_tmpdir"` test "$ls_ld_tmpdir" = "$ls_ld_tmpdir_1" } } then posix_mkdir=: fi - rmdir "$tmpdir/d" "$tmpdir" + rmdir "$tmpdir/a/b" "$tmpdir/a" "$tmpdir" else # Remove any dirs left behind by ancient mkdir implementations. - rmdir ./$mkdir_mode ./-p ./-- 2>/dev/null + rmdir ./$mkdir_mode ./-p ./-- "$tmpdir" 2>/dev/null fi trap '' 0;; esac;; diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/fstrm-0.2.0/build-aux/ltmain.sh new/fstrm-0.3.2/build-aux/ltmain.sh --- old/fstrm-0.2.0/build-aux/ltmain.sh 2014-11-07 01:21:27.000000000 +0100 +++ new/fstrm-0.3.2/build-aux/ltmain.sh 2017-04-10 15:51:27.000000000 +0200 @@ -70,7 +70,7 @@ # compiler: $LTCC # compiler flags: $LTCFLAGS # linker: $LD (gnu? $with_gnu_ld) -# $progname: (GNU libtool) 2.4.2 Debian-2.4.2-1.10 +# $progname: (GNU libtool) 2.4.2 Debian-2.4.2-1.11 # automake: $automake_version # autoconf: $autoconf_version # @@ -80,7 +80,7 @@ PROGRAM=libtool PACKAGE=libtool -VERSION="2.4.2 Debian-2.4.2-1.10" +VERSION="2.4.2 Debian-2.4.2-1.11" TIMESTAMP="" package_revision=1.3337 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/fstrm-0.2.0/config.h.in new/fstrm-0.3.2/config.h.in --- old/fstrm-0.2.0/config.h.in 2014-11-07 01:21:33.000000000 +0100 +++ new/fstrm-0.3.2/config.h.in 2017-04-10 15:51:30.000000000 +0200 @@ -1,5 +1,8 @@ /* config.h.in. Generated from configure.ac by autoheader. */ +/* Define to 1 if you have the `clock_gettime' function. */ +#undef HAVE_CLOCK_GETTIME + /* Define to 1 if you have the declaration of `fflush_unlocked', and to 0 if you don't. */ #undef HAVE_DECL_FFLUSH_UNLOCKED @@ -24,6 +27,9 @@ /* Define if you have POSIX threads libraries and header files. */ #undef HAVE_PTHREAD +/* Define to 1 if you have the `pthread_condattr_setclock' function. */ +#undef HAVE_PTHREAD_CONDATTR_SETCLOCK + /* Have PTHREAD_PRIO_INHERIT. */ #undef HAVE_PTHREAD_PRIO_INHERIT diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/fstrm-0.2.0/configure new/fstrm-0.3.2/configure --- old/fstrm-0.2.0/configure 2014-11-07 01:21:30.000000000 +0100 +++ new/fstrm-0.3.2/configure 2017-04-10 15:51:30.000000000 +0200 @@ -1,6 +1,6 @@ #! /bin/sh # Guess values for system-dependent variables and create Makefiles. -# Generated by GNU Autoconf 2.69 for fstrm 0.2.0. +# Generated by GNU Autoconf 2.69 for fstrm 0.3.2. # # Report bugs to <https://github.com/farsightsec/fstrm/issues>. # @@ -590,8 +590,8 @@ # Identity of this package. PACKAGE_NAME='fstrm' PACKAGE_TARNAME='fstrm' -PACKAGE_VERSION='0.2.0' -PACKAGE_STRING='fstrm 0.2.0' +PACKAGE_VERSION='0.3.2' +PACKAGE_STRING='fstrm 0.3.2' PACKAGE_BUGREPORT='https://github.com/farsightsec/fstrm/issues' PACKAGE_URL='https://github.com/farsightsec/fstrm' @@ -1351,7 +1351,7 @@ # Omit some internal or obsolete options to make the list less imposing. # This message is too long to be a string in the A/UX 3.1 sh. cat <<_ACEOF -\`configure' configures fstrm 0.2.0 to adapt to many kinds of systems. +\`configure' configures fstrm 0.3.2 to adapt to many kinds of systems. Usage: $0 [OPTION]... [VAR=VALUE]... @@ -1421,7 +1421,7 @@ if test -n "$ac_init_help"; then case $ac_init_help in - short | recursive ) echo "Configuration of fstrm 0.2.0:";; + short | recursive ) echo "Configuration of fstrm 0.3.2:";; esac cat <<\_ACEOF @@ -1544,7 +1544,7 @@ test -n "$ac_init_help" && exit $ac_status if $ac_init_version; then cat <<\_ACEOF -fstrm configure 0.2.0 +fstrm configure 0.3.2 generated by GNU Autoconf 2.69 Copyright (C) 2012 Free Software Foundation, Inc. @@ -1959,7 +1959,7 @@ This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. -It was created by fstrm $as_me 0.2.0, which was +It was created by fstrm $as_me 0.3.2, which was generated by GNU Autoconf 2.69. Invocation command line was $ $0 $@ @@ -2512,8 +2512,8 @@ ac_script='s/[\\$]/&&/g;s/;s,x,x,$//' program_transform_name=`$as_echo "$program_transform_name" | sed "$ac_script"` -# expand $ac_aux_dir to an absolute path -am_aux_dir=`cd $ac_aux_dir && pwd` +# Expand $ac_aux_dir to an absolute path. +am_aux_dir=`cd "$ac_aux_dir" && pwd` if test x"${MISSING+set}" != xset; then case $am_aux_dir in @@ -2826,7 +2826,7 @@ # Define the identity of the package. PACKAGE='fstrm' - VERSION='0.2.0' + VERSION='0.3.2' cat >>confdefs.h <<_ACEOF @@ -12843,6 +12843,19 @@ fi +for ac_func in clock_gettime pthread_condattr_setclock +do : + as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh` +ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var" +if eval test \"x\$"$as_ac_var"\" = x"yes"; then : + cat >>confdefs.h <<_ACEOF +#define `$as_echo "HAVE_$ac_func" | $as_tr_cpp` 1 +_ACEOF + +fi +done + + { $as_echo "$as_me:${as_lineno-$LINENO}: checking for library containing socket" >&5 $as_echo_n "checking for library containing socket... " >&6; } if ${ac_cv_search_socket+:} false; then : @@ -13944,7 +13957,7 @@ # report actual input values of CONFIG_FILES etc. instead of their # values after options handling. ac_log=" -This file was extended by fstrm $as_me 0.2.0, which was +This file was extended by fstrm $as_me 0.3.2, which was generated by GNU Autoconf 2.69. Invocation command line was CONFIG_FILES = $CONFIG_FILES @@ -14011,7 +14024,7 @@ cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`" ac_cs_version="\\ -fstrm config.status 0.2.0 +fstrm config.status 0.3.2 configured by $0, generated by GNU Autoconf 2.69, with options \\"\$ac_cs_config\\" diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/fstrm-0.2.0/configure.ac new/fstrm-0.3.2/configure.ac --- old/fstrm-0.2.0/configure.ac 2014-11-07 01:19:59.000000000 +0100 +++ new/fstrm-0.3.2/configure.ac 2017-04-10 15:51:04.000000000 +0200 @@ -1,7 +1,7 @@ AC_PREREQ(2.64) AC_INIT([fstrm], - [0.2.0], + [0.3.2], [https://github.com/farsightsec/fstrm/issues], [fstrm], [https://github.com/farsightsec/fstrm]) @@ -47,6 +47,8 @@ ]) AC_SEARCH_LIBS([clock_gettime], [rt]) +AC_CHECK_FUNCS([clock_gettime pthread_condattr_setclock]) + AC_SEARCH_LIBS([socket], [socket]) AC_CHECK_DECLS([fread_unlocked, fwrite_unlocked, fflush_unlocked]) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/fstrm-0.2.0/fstrm/file.c new/fstrm-0.3.2/fstrm/file.c --- old/fstrm-0.2.0/fstrm/file.c 2014-05-02 19:14:39.000000000 +0200 +++ new/fstrm-0.3.2/fstrm/file.c 2017-04-10 15:51:04.000000000 +0200 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2014 by Farsight Security, Inc. + * Copyright (c) 2014, 2016 by Farsight Security, Inc. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -55,7 +55,10 @@ { struct fstrm__file *f = obj; if (f->fp == NULL && f->file_path != NULL) { - f->fp = fopen(f->file_path, f->file_mode); + if (!strcmp(f->file_path, "-")) + f->fp = f->file_mode[0] == 'r' ? stdin : stdout; + else + f->fp = fopen(f->file_path, f->file_mode); if (f->fp == NULL) return fstrm_res_failure; return fstrm_res_success; diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/fstrm-0.2.0/fstrm/fstrm-private.h new/fstrm-0.3.2/fstrm/fstrm-private.h --- old/fstrm-0.2.0/fstrm/fstrm-private.h 2014-05-02 19:14:39.000000000 +0200 +++ new/fstrm-0.3.2/fstrm/fstrm-private.h 2016-12-22 22:43:56.000000000 +0100 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2013-2014 by Farsight Security, Inc. + * Copyright (c) 2013-2016 by Farsight Security, Inc. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -36,6 +36,7 @@ #include "libmy/my_alloc.h" #include "libmy/my_memory_barrier.h" #include "libmy/my_queue.h" +#include "libmy/my_time.h" #include "libmy/read_bytes.h" #include "libmy/vector.h" @@ -162,6 +163,7 @@ /* time */ +#if HAVE_CLOCK_GETTIME bool fstrm__get_best_monotonic_clock_gettime(clockid_t *); bool fstrm__get_best_monotonic_clock_pthread(clockid_t *); @@ -169,8 +171,7 @@ bool fstrm__get_best_monotonic_clocks(clockid_t *clkid_gettime, clockid_t *clkid_pthread, char **errstr_out); - -int fstrm__pthread_cond_timedwait(clockid_t, pthread_cond_t *, pthread_mutex_t *, unsigned); +#endif /* queue */ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/fstrm-0.2.0/fstrm/iothr.c new/fstrm-0.3.2/fstrm/iothr.c --- old/fstrm-0.2.0/fstrm/iothr.c 2014-10-01 22:28:55.000000000 +0200 +++ new/fstrm-0.3.2/fstrm/iothr.c 2017-04-10 15:51:04.000000000 +0200 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2013-2014 by Farsight Security, Inc. + * Copyright (c) 2013, 2014, 2016-2017 by Farsight Security, Inc. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -81,9 +81,11 @@ /* Whether the I/O thread is shutting down. */ volatile bool shutting_down; +#if HAVE_CLOCK_GETTIME /* Optimal clockid_t's. */ clockid_t clkid_gettime; clockid_t clkid_pthread; +#endif /* * Conditional variable and lock, used by producer thread @@ -237,7 +239,7 @@ /* * Some platforms have a ridiculously low IOV_MAX, literally the lowest * value even allowed by POSIX, which is lower than our conservative - * FSTRM_IOTHR_OUTPUT_QUEUE_SIZE_DEFAULT. Accomodate these platforms by + * FSTRM_IOTHR_OUTPUT_QUEUE_SIZE_DEFAULT. Accommodate these platforms by * silently clamping output_queue_size to IOV_MAX. */ if (iothr->opt.output_queue_size > IOV_MAX) @@ -262,6 +264,7 @@ iothr->queue_ops = &my_queue_mutex_ops; } +#if HAVE_CLOCK_GETTIME /* Detect best clocks. */ if (!fstrm__get_best_monotonic_clocks(&iothr->clkid_gettime, &iothr->clkid_pthread, @@ -269,6 +272,7 @@ { goto fail; } +#endif /* Initialize the input queues. */ iothr->queues = my_calloc(iothr->opt.num_input_queues, @@ -290,8 +294,10 @@ res = pthread_condattr_init(&ca); assert(res == 0); +#if HAVE_CLOCK_GETTIME && HAVE_PTHREAD_CONDATTR_SETCLOCK res = pthread_condattr_setclock(&ca, iothr->clkid_pthread); assert(res == 0); +#endif res = pthread_cond_init(&iothr->cv, &ca); assert(res == 0); @@ -355,6 +361,9 @@ (*iothr)->shutting_down = true; pthread_cond_signal(&(*iothr)->cv); pthread_join((*iothr)->thr, NULL); + pthread_cond_destroy(&(*iothr)->cv); + pthread_mutex_destroy(&(*iothr)->cv_lock); + pthread_mutex_destroy(&(*iothr)->get_queue_lock); /* Destroy the writer by calling its 'destroy' method. */ (void)fstrm_writer_destroy(&(*iothr)->writer); @@ -544,13 +553,16 @@ if (likely(iothr->opened)) return; - int rv; time_t since; struct timespec ts; /* Check if the reopen interval has expired yet. */ - rv = clock_gettime(iothr->clkid_gettime, &ts); +#if HAVE_CLOCK_GETTIME + int rv = clock_gettime(iothr->clkid_gettime, &ts); assert(rv == 0); +#else + my_gettime(-1, &ts); +#endif since = ts.tv_sec - iothr->last_open_attempt; if (since < (time_t) iothr->opt.reopen_interval) return; @@ -598,9 +610,23 @@ if (count != 0) continue; - res = fstrm__pthread_cond_timedwait(iothr->clkid_pthread, - &iothr->cv, &iothr->cv_lock, - iothr->opt.flush_timeout); + struct timespec ts; +#if HAVE_CLOCK_GETTIME +#if HAVE_PTHREAD_CONDATTR_SETCLOCK + int rv = clock_gettime(iothr->clkid_pthread, &ts); +#else + int rv = clock_gettime(CLOCK_REALTIME, &ts); +#endif + assert(rv == 0); +#else + my_gettime(-1, &ts); +#endif + ts.tv_sec += iothr->opt.flush_timeout; + + pthread_mutex_lock(&iothr->cv_lock); + res = pthread_cond_timedwait(&iothr->cv, &iothr->cv_lock, &ts); + pthread_mutex_unlock(&iothr->cv_lock); + if (res == ETIMEDOUT) fstrm__iothr_flush_output(iothr); } diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/fstrm-0.2.0/fstrm/time.c new/fstrm-0.3.2/fstrm/time.c --- old/fstrm-0.2.0/fstrm/time.c 2014-10-01 22:40:41.000000000 +0200 +++ new/fstrm-0.3.2/fstrm/time.c 2017-04-10 15:51:04.000000000 +0200 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2013-2014 by Farsight Security, Inc. + * Copyright (c) 2013-2014, 2016-2017 by Farsight Security, Inc. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -16,6 +16,9 @@ #include "fstrm-private.h" +#if HAVE_CLOCK_GETTIME + +#if HAVE_PTHREAD_CONDATTR_SETCLOCK bool fstrm__get_best_monotonic_clock_pthread(clockid_t *c) { @@ -72,6 +75,7 @@ assert(rc == 0); return res; } +#endif /* HAVE_PTHREAD_CONDATTR_SETCLOCK */ bool fstrm__get_best_monotonic_clock_gettime(clockid_t *c) @@ -114,6 +118,7 @@ return false; } +#if HAVE_PTHREAD_CONDATTR_SETCLOCK if (clkid_pthread != NULL && !fstrm__get_best_monotonic_clock_pthread(clkid_pthread)) { @@ -121,23 +126,9 @@ *err = my_strdup("no clock available for pthread_cond_timedwait()"); return false; } +#endif return true; } -int -fstrm__pthread_cond_timedwait(clockid_t clock_id, - pthread_cond_t *cond, - pthread_mutex_t *mutex, - unsigned seconds) -{ - int res; - struct timespec ts; - res = clock_gettime(clock_id, &ts); - assert(res == 0); - ts.tv_sec += seconds; - pthread_mutex_lock(mutex); - res = pthread_cond_timedwait(cond, mutex, &ts); - pthread_mutex_unlock(mutex); - return res; -} +#endif /* HAVE_CLOCK_GETTIME */ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/fstrm-0.2.0/fstrm/writer.c new/fstrm-0.3.2/fstrm/writer.c --- old/fstrm-0.2.0/fstrm/writer.c 2014-08-15 21:37:47.000000000 +0200 +++ new/fstrm-0.3.2/fstrm/writer.c 2016-12-22 22:43:56.000000000 +0100 @@ -302,6 +302,8 @@ if (w->state != fstrm_writer_state_opened) return fstrm_res_failure; + w->state = fstrm_writer_state_closed; + /* Write the STOP frame. */ res = fstrm__rdwr_write_control(w->rdwr, FSTRM_CONTROL_STOP, NULL); if (res != fstrm_res_success) { @@ -320,7 +322,6 @@ } res = fstrm_rdwr_close(w->rdwr); - w->state = fstrm_writer_state_closed; return res; } diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/fstrm-0.2.0/src/fstrm_capture.c new/fstrm-0.3.2/src/fstrm_capture.c --- old/fstrm-0.2.0/src/fstrm_capture.c 2014-10-24 23:15:20.000000000 +0200 +++ new/fstrm-0.3.2/src/fstrm_capture.c 2016-12-22 22:43:56.000000000 +0100 @@ -27,6 +27,7 @@ #include <stdio.h> #include <stdlib.h> #include <string.h> +#include <time.h> #include <unistd.h> #include <event2/buffer.h> @@ -95,19 +96,27 @@ evutil_socket_t listen_fd; struct event_base *ev_base; struct evconnlistener *ev_connlistener; + struct event *ev_sighup; FILE *output_file; + char *output_fname; + time_t output_open_timestamp; size_t bytes_written; size_t count_written; + + struct tm *(*calendar_fn)(const time_t *, struct tm *); }; struct capture_args { bool help; int debug; + bool localtime; + bool gmtime; char *str_content_type; char *str_read_unix; char *str_write_fname; + int split_seconds; }; static struct capture g_program_ctx; @@ -120,30 +129,48 @@ NULL, "display this help text and exit" }, - { 'd', "debug", + { 'd', "debug", ARGV_INCR, &g_program_args.debug, NULL, "increment debugging level" }, - { 't', "type", + { 't', "type", ARGV_CHAR_P, &g_program_args.str_content_type, "<STRING>", "Frame Streams content type" }, - { 'u', "unix", + { 'u', "unix", ARGV_CHAR_P, &g_program_args.str_read_unix, "<FILENAME>", "Unix socket path to read from" }, - { 'w', "write", + { 'w', "write", ARGV_CHAR_P, &g_program_args.str_write_fname, "<FILENAME>", "file path to write Frame Streams data to" }, + { 's', "split", + ARGV_INT, + &g_program_args.split_seconds, + "<SECONDS>", + "seconds before rotating output file" }, + + { '\0', "localtime", + ARGV_BOOL, + &g_program_args.localtime, + NULL, + "filter -w path with strftime (local time)" }, + + { '\0', "gmtime", + ARGV_BOOL, + &g_program_args.gmtime, + NULL, + "filter -w path with strftime (UTC)" }, + { ARGV_LAST }, }; @@ -236,7 +263,7 @@ } static bool -parse_args(const int argc, char **argv) +parse_args(const int argc, char **argv, struct capture *ctx) { argv_version_string = PACKAGE_VERSION; @@ -251,7 +278,23 @@ if (g_program_args.str_read_unix == NULL) usage("Unix socket path to read from (--unix) is not set"); if (g_program_args.str_write_fname == NULL) - usage("file path to write Frame Streams data to (--write) is not set"); + usage("File path to write Frame Streams data to (--write) is not set"); + if (strcmp(g_program_args.str_write_fname, "-") == 0) { + if (isatty(STDOUT_FILENO) == 1) + usage("Refusing to write binary output to a terminal"); + if (g_program_args.split_seconds != 0) + usage("Cannot use output splitting when writing to stdout"); + } + if (g_program_args.localtime && g_program_args.gmtime) + usage("--localtime and --gmtime are mutually exclusive"); + if (g_program_args.split_seconds && !g_program_args.localtime && !g_program_args.gmtime) + usage("--split requires either --localtime or --gmtime"); + + /* Set calendar function, if needed. */ + if (g_program_args.localtime) + ctx->calendar_fn = localtime_r; + else if (g_program_args.gmtime) + ctx->calendar_fn = gmtime_r; return true; } @@ -369,33 +412,86 @@ return false; } +static const char * +update_output_fname(struct capture *ctx) +{ + time_t time_now = {0}; + struct tm tm_now = {0}; + + /* Get current broken-down time representation. */ + tzset(); + time_now = time(NULL); + ctx->calendar_fn(&time_now, &tm_now); + + /* Save current time. */ + ctx->output_open_timestamp = time_now; + + /* + * Filter ctx->args->str_write_fname with strftime(), store output in + * ctx->output_fname. Assume strftime() lengthens the string by no more + * than 256 bytes. + */ + if (ctx->output_fname != NULL) + my_free(ctx->output_fname); + const size_t len_output_fname = strlen(ctx->args->str_write_fname) + 256; + ctx->output_fname = my_calloc(1, len_output_fname); + + if (strftime(ctx->output_fname, len_output_fname, + ctx->args->str_write_fname, &tm_now) <= 0) + { + my_free(ctx->output_fname); + fprintf(stderr, "%s: strftime() failed on format string \"%s\"\n", + argv_program, ctx->args->str_write_fname); + return NULL; + } + + return ctx->output_fname; +} + static bool open_write_file(struct capture *ctx) { const char *fname = ctx->args->str_write_fname; - mode_t open_mode = S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH | S_IWOTH; - int open_flags = O_CREAT | O_WRONLY | O_TRUNC; + + if (strcmp(fname, "-") == 0) { + /* Use already opened FILE* for stdout. */ + ctx->output_file = stdout; + } else { + mode_t open_mode = S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH | S_IWOTH; + int open_flags = O_CREAT | O_WRONLY | O_TRUNC; #if defined(O_CLOEXEC) - open_flags |= O_CLOEXEC; + open_flags |= O_CLOEXEC; #endif - /* Open the file descriptor. */ - int fd = open(fname, open_flags, open_mode); - if (fd == -1) { - fprintf(stderr, "%s: failed to open output file %s\n", - argv_program, fname); - return false; - } + /* Rewrite the output filename if needed. */ + if (ctx->calendar_fn) { + fname = update_output_fname(ctx); + if (fname == NULL) + return false; + } - /* Open the FILE object. */ - ctx->output_file = fdopen(fd, "w"); - if (!ctx->output_file) { - close(fd); - fprintf(stderr, "%s: failed to fdopen output file %s\n", - argv_program, fname); - return false; + /* Open the file descriptor. */ + int fd = open(fname, open_flags, open_mode); + if (fd == -1) { + fprintf(stderr, "%s: failed to open output file %s\n", + argv_program, fname); + return false; + } + + /* Open the FILE object. */ + ctx->output_file = fdopen(fd, "w"); + if (!ctx->output_file) { + close(fd); + fprintf(stderr, "%s: failed to fdopen output file %s\n", + argv_program, fname); + return false; + } } + /* Reset output statistics. */ + ctx->count_written = 0; + ctx->bytes_written = 0; + /* Write the START frame. */ if (!open_write_start(ctx)) { fclose(ctx->output_file); @@ -425,7 +521,7 @@ /* Success. */ fprintf(stderr, "%s: closed output file %s (wrote %zd frames, %zd bytes)\n", - argv_program, ctx->args->str_write_fname, + argv_program, ctx->output_fname ? : ctx->args->str_write_fname, ctx->count_written, ctx->bytes_written); return true; } @@ -483,6 +579,30 @@ conn->ctx->bytes_written += bytes_read; } +static void +maybe_rotate_output(struct conn *conn) +{ + /* Output file rotation requested? */ + if (conn->ctx->args->split_seconds > 0) { + time_t t_now = time(NULL); + + /* Is it time to rotate? */ + if (t_now >= conn->ctx->output_open_timestamp + conn->ctx->args->split_seconds) { + /* Rotate output file, fail hard if unsuccessful. */ + if (!close_write_file(conn->ctx)) { + fprintf(stderr, "%s: %s: close_write_file() failed\n", + argv_program, __func__); + exit(EXIT_FAILURE); + } + if (!open_write_file(conn->ctx)) { + fprintf(stderr, "%s: %s: open_write_file() failed\n", + argv_program, __func__); + exit(EXIT_FAILURE); + } + } + } +} + static bool send_frame(struct conn *conn, const void *data, size_t size) { @@ -794,6 +914,9 @@ if (conn->len_frame_payload > 0) { /* This is a data frame. */ process_data_frame(conn); + + /* Check if it's time to rotate the output file. */ + maybe_rotate_output(conn); } else { /* This is a control frame. */ if (!load_control_frame(conn)) { @@ -848,6 +971,16 @@ evutil_socket_error_to_string(err)); } +static void +do_sighup(evutil_socket_t sig, short events, void *user_data) +{ + struct capture *ctx = user_data; + if (ctx->output_file) { + fflush(ctx->output_file); + fprintf(stderr, "%s: received SIGHUP, flushing output\n", argv_program); + } +} + static bool setup_event_loop(struct capture *ctx) { @@ -871,6 +1004,11 @@ } evconnlistener_set_error_cb(ctx->ev_connlistener, cb_accept_error); + /* Register our SIGHUP handler. */ + ctx->ev_sighup = evsignal_new(ctx->ev_base, SIGHUP, &do_sighup, + &g_program_ctx); + evsignal_add(ctx->ev_sighup, NULL); + /* Success. */ return true; } @@ -903,17 +1041,20 @@ cleanup(struct capture *ctx) { argv_cleanup(g_args); + if (ctx->ev_sighup != NULL) + event_free(ctx->ev_sighup); if (ctx->ev_connlistener != NULL) evconnlistener_free(ctx->ev_connlistener); if (ctx->ev_base != NULL) event_base_free(ctx->ev_base); + my_free(ctx->output_fname); } int main(int argc, char **argv) { /* Parse arguments. */ - if (!parse_args(argc, argv)) { + if (!parse_args(argc, argv, &g_program_ctx)) { usage(NULL); return EXIT_FAILURE; } diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/fstrm-0.2.0/t/test_fstrm_io_file.c new/fstrm-0.3.2/t/test_fstrm_io_file.c --- old/fstrm-0.2.0/t/test_fstrm_io_file.c 2014-05-02 19:14:39.000000000 +0200 +++ new/fstrm-0.3.2/t/test_fstrm_io_file.c 2016-12-22 22:43:56.000000000 +0100 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2013-2014 by Farsight Security, Inc. + * Copyright (c) 2013-2016 by Farsight Security, Inc. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -272,7 +272,13 @@ assert(0); /* not reached */ } - my_gettime(CLOCK_MONOTONIC, &ts_a); +#if HAVE_CLOCK_GETTIME + const clockid_t clock = CLOCK_MONOTONIC; +#else + const int clock = -1; +#endif + my_gettime(clock, &ts_a); + printf("creating %u producer threads\n", num_threads); for (unsigned i = 0; i < num_threads; i++) @@ -285,7 +291,7 @@ printf("destroying fstrm_iothr object\n"); fstrm_iothr_destroy(&iothr); - my_gettime(CLOCK_MONOTONIC, &ts_b); + my_gettime(clock, &ts_b); my_timespec_sub(&ts_a, &ts_b); elapsed = my_timespec_to_double(&ts_b); printf("completed in %.2f seconds\n", elapsed); diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/fstrm-0.2.0/t/test_fstrm_io_unix.c new/fstrm-0.3.2/t/test_fstrm_io_unix.c --- old/fstrm-0.2.0/t/test_fstrm_io_unix.c 2014-05-02 19:14:39.000000000 +0200 +++ new/fstrm-0.3.2/t/test_fstrm_io_unix.c 2016-12-22 22:43:56.000000000 +0100 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2013-2014 by Farsight Security, Inc. + * Copyright (c) 2013-2016 by Farsight Security, Inc. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -443,7 +443,12 @@ assert(0); /* not reached */ } - my_gettime(CLOCK_MONOTONIC, &ts_a); +#if HAVE_CLOCK_GETTIME + const clockid_t clock = CLOCK_MONOTONIC; +#else + const int clock = -1; +#endif + my_gettime(clock, &ts_a); printf("creating %u producer threads\n", num_threads); for (unsigned i = 0; i < num_threads; i++) @@ -459,7 +464,7 @@ printf("joining consumer thread\n"); pthread_join(test_consumer.thr, (void **) NULL); - my_gettime(CLOCK_MONOTONIC, &ts_b); + my_gettime(clock, &ts_b); my_timespec_sub(&ts_a, &ts_b); elapsed = my_timespec_to_double(&ts_b); printf("completed in %.2f seconds\n", elapsed); diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/fstrm-0.2.0/t/test_queue.c new/fstrm-0.3.2/t/test_queue.c --- old/fstrm-0.2.0/t/test_queue.c 2014-05-02 19:14:39.000000000 +0200 +++ new/fstrm-0.3.2/t/test_queue.c 2016-12-22 22:43:56.000000000 +0100 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2013-2014 by Farsight Security, Inc. + * Copyright (c) 2013-2016 by Farsight Security, Inc. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -235,7 +235,12 @@ pthread_t thr_p; pthread_t thr_c; - my_gettime(CLOCK_MONOTONIC, &ts_a); +#if HAVE_CLOCK_GETTIME + const clockid_t clock = CLOCK_MONOTONIC; +#else + const int clock = -1; +#endif + my_gettime(clock, &ts_a); pthread_create(&thr_p, NULL, thr_producer, NULL); pthread_create(&thr_c, NULL, thr_consumer, NULL); @@ -247,7 +252,7 @@ send_shutdown_message(q); pthread_join(thr_c, (void **) &cs); - my_gettime(CLOCK_MONOTONIC, &ts_b); + my_gettime(clock, &ts_b); res = check_stats(ps, cs); print_stats(&ts_a, &ts_b, ps, cs);
