Module Name: othersrc
Committed By: dyoung
Date: Tue Jan 26 02:22:57 UTC 2016
Modified Files:
othersrc/external/bsd/arfe: Makefile README
othersrc/external/bsd/arfe/dt: Makefile README core.c core.h dec.c
dec.h dt.c hex.c hex.h ipv4.c ipv4.h macaddr.c macaddr.h
portability.h sym.c sym.h testit
othersrc/external/bsd/arfe/dt/rr: ifconfig.0-logger-ifconfig.0
logger-ifconfig.0-ifconfig.0
othersrc/external/bsd/arfe/it: Makefile README it.c
othersrc/external/bsd/arfe/tt: Makefile README testit.sh tt.c
Added Files:
othersrc/external/bsd/arfe: Makefile.inc
othersrc/external/bsd/arfe/dt: eol.c eol.h space.c space.h
othersrc/external/bsd/arfe/hlog: .cvsignore Makefile hlog.c hlog.h
othersrc/external/bsd/arfe/mk: helpers.mk
othersrc/external/bsd/arfe/tt/tests/6: expected-output input match
transform
Log Message:
Bring NetBSD CVS up-to-date with my private Subversion repository.
In dt/core.c, delete dead scratch_t members L1 and L2. Pull the gaps
count out of subcell_t and add to scratch_t members G[0..1] and Gclocc
to hold the gaps count when/where I use it. This gives me back a bit of
speed, but I'm still not back to the previous best: affine gap penalties
are expensive.
Add a dummy 'test' target to it/Makefile. Really need to come up with
some tests for IT. Some version of the DT tests oughta do the trick.
Thanks, Thomas Klausner ([email protected]), for the heads-up about some
Makefile problems. Fix them: recurse on 'test' target. Make 'test'
target depend on dependall.
Fix/improve usage messages.
Disable debug assertions for now.
Make 'make cleandir' remove *.gcov.
In dt/sym.c, don't dereference a NULL pointer trying to emit a symbol
that is too long. XXX ought to revisit this.
Add the hierarchical-logging library that I developed for CUWiN and
start to use it to switch debugging printfs on and off.
Repair the 'tags' rule, making it write tags to .CURDIR instead of
.OBJDIR.
Add two new scanners, one for whitespace up to and including end-of-line
(EOL), and one for whitespace other than whitespace at EOL. Bring
expected test results up-to-date with these beneficial changes to
whitespace handling.
Add mk/helpers.mk that provides utilities PRINTOBJDIR and PRINTOBJDIROF.
Use relative paths like 'dt/core.h' for ARFE header files. Add
CPPFLAGS+=-I$(.CURDIR)/.. to all of the makefiles to make this work.
Add DPINCS to makefiles that were missing it.
Factor out some initialization and argument parsing; put it into
arfe_parse_options().
Improve file_to_slice() error reporting: print the filename concerned.
Add a utility function, cloccs_are_equal(), that returns true if and
only if the string comprising the left-hand clocc is equal to the string
comprising the right-hand clocc. Use it here and there, especially in
the `tt' implementation.
Tweak the costs for opening and extending gaps, and tweak clocc_score(),
to get more desirable outputs from dt, it, and tt. Update tests to
match new expectations.
Add instrumentation that prints a sub-table of the table computed by
findsplitn() as an HTML table.
Fix some bugs in table initialization in findsplitn() & count_records()
that were found with the help of the new instrumentation.
For count_records() debug output, record and print a few generations of
record boundaries.
Add a new member to clocc_t, the number of potential counterpart
clocc_t's in a counterpart template, and for each match-template clocc,
count the number of counterparts it has in the transform template---see
count_match_occ_transform_counterparts(). Eventually I will use the
counterparts to tweak scores for clocc-alignment between inputs and
match templates in tt.
Fix a bug in emit_transformed_text() where it wasn't considering all of
the match clocc_t's in a hash bucket.
Expand cloccs_t and the size of the clocc_t hash tables, too. XXX needs
to be dynamic!
In dt, it, and tt (?), instead of running findlcs(), run count_records()
and quit if the -c option is passed. This is just a convenience while I
test count_records(), which is a work in progress.
Fix some NULL pointer dereferences in the decimal- and hexadecimal-number
scanners.
Run the really long-running dt tests last instead of first.
Make tt/testit.sh run the tests on its command line or else all of the
tests in tests/.
In tt, allocate the class occurrences for the transform template on the
heap instead of the stack, so that we don't run out of stack space
and segfault.
Add a test for 'tt' that extracts the packet input/output statistics
from ifconfig -v output and nothing else.
Fix a bug in count_match_occ_transform_counterparts() where I was
calling TAILQ_NEXT() on an uninitialized element. I'm not sure how I
missed the bug before, since it crashes 'tt' in its first test every
time.
Pick up some changes from the NetBSD CVS repository by wiz@. Change
__attribute__((__noreturn__)) to __dead. Prepare for committing to
NetBSD CVS by adding a .cvsignore or two, making Subversion ignore some
CVS/ directories, etc.
To generate a diff of this commit:
cvs rdiff -u -r1.2 -r1.3 othersrc/external/bsd/arfe/Makefile
cvs rdiff -u -r0 -r1.1 othersrc/external/bsd/arfe/Makefile.inc
cvs rdiff -u -r1.3 -r1.4 othersrc/external/bsd/arfe/README
cvs rdiff -u -r1.7 -r1.8 othersrc/external/bsd/arfe/dt/Makefile
cvs rdiff -u -r1.12 -r1.13 othersrc/external/bsd/arfe/dt/README
cvs rdiff -u -r1.6 -r1.7 othersrc/external/bsd/arfe/dt/core.c \
othersrc/external/bsd/arfe/dt/macaddr.c
cvs rdiff -u -r1.5 -r1.6 othersrc/external/bsd/arfe/dt/core.h
cvs rdiff -u -r1.3 -r1.4 othersrc/external/bsd/arfe/dt/dec.c \
othersrc/external/bsd/arfe/dt/dec.h \
othersrc/external/bsd/arfe/dt/macaddr.h
cvs rdiff -u -r1.17 -r1.18 othersrc/external/bsd/arfe/dt/dt.c
cvs rdiff -u -r0 -r1.1 othersrc/external/bsd/arfe/dt/eol.c \
othersrc/external/bsd/arfe/dt/eol.h othersrc/external/bsd/arfe/dt/space.c \
othersrc/external/bsd/arfe/dt/space.h
cvs rdiff -u -r1.8 -r1.9 othersrc/external/bsd/arfe/dt/hex.c \
othersrc/external/bsd/arfe/dt/hex.h othersrc/external/bsd/arfe/dt/ipv4.c \
othersrc/external/bsd/arfe/dt/ipv4.h
cvs rdiff -u -r1.4 -r1.5 othersrc/external/bsd/arfe/dt/portability.h \
othersrc/external/bsd/arfe/dt/testit
cvs rdiff -u -r1.1 -r1.2 othersrc/external/bsd/arfe/dt/sym.c \
othersrc/external/bsd/arfe/dt/sym.h
cvs rdiff -u -r1.6 -r1.7 \
othersrc/external/bsd/arfe/dt/rr/ifconfig.0-logger-ifconfig.0 \
othersrc/external/bsd/arfe/dt/rr/logger-ifconfig.0-ifconfig.0
cvs rdiff -u -r0 -r1.1 othersrc/external/bsd/arfe/hlog/.cvsignore \
othersrc/external/bsd/arfe/hlog/Makefile \
othersrc/external/bsd/arfe/hlog/hlog.c \
othersrc/external/bsd/arfe/hlog/hlog.h
cvs rdiff -u -r1.6 -r1.7 othersrc/external/bsd/arfe/it/Makefile
cvs rdiff -u -r1.10 -r1.11 othersrc/external/bsd/arfe/it/README
cvs rdiff -u -r1.3 -r1.4 othersrc/external/bsd/arfe/it/it.c
cvs rdiff -u -r0 -r1.1 othersrc/external/bsd/arfe/mk/helpers.mk
cvs rdiff -u -r1.4 -r1.5 othersrc/external/bsd/arfe/tt/Makefile \
othersrc/external/bsd/arfe/tt/tt.c
cvs rdiff -u -r1.7 -r1.8 othersrc/external/bsd/arfe/tt/README
cvs rdiff -u -r1.2 -r1.3 othersrc/external/bsd/arfe/tt/testit.sh
cvs rdiff -u -r0 -r1.1 othersrc/external/bsd/arfe/tt/tests/6/expected-output \
othersrc/external/bsd/arfe/tt/tests/6/input \
othersrc/external/bsd/arfe/tt/tests/6/match \
othersrc/external/bsd/arfe/tt/tests/6/transform
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
Modified files:
Index: othersrc/external/bsd/arfe/Makefile
diff -u othersrc/external/bsd/arfe/Makefile:1.2 othersrc/external/bsd/arfe/Makefile:1.3
--- othersrc/external/bsd/arfe/Makefile:1.2 Fri Sep 11 01:50:42 2015
+++ othersrc/external/bsd/arfe/Makefile Tue Jan 26 02:22:56 2016
@@ -1,5 +1,6 @@
.include <bsd.own.mk>
-SUBDIR=dt it tt
+SUBDIR=hlog .WAIT dt it tt
+TARGETS+=test
.include <bsd.subdir.mk>
Index: othersrc/external/bsd/arfe/README
diff -u othersrc/external/bsd/arfe/README:1.3 othersrc/external/bsd/arfe/README:1.4
--- othersrc/external/bsd/arfe/README:1.3 Thu Dec 3 03:29:01 2015
+++ othersrc/external/bsd/arfe/README Tue Jan 26 02:22:56 2016
@@ -1,5 +1,5 @@
-$ARFE: README 305 2015-12-03 03:28:28Z dyoung $
-$NetBSD: README,v 1.3 2015/12/03 03:29:01 dyoung Exp $
+$ARFE: README 323 2016-01-25 20:29:30Z dyoung $
+$NetBSD: README,v 1.4 2016/01/26 02:22:56 dyoung Exp $
ARFE is a suite of tools for processing record- and field-oriented
digital texts. ARFE strives to make a useful set of automatic
Index: othersrc/external/bsd/arfe/dt/Makefile
diff -u othersrc/external/bsd/arfe/dt/Makefile:1.7 othersrc/external/bsd/arfe/dt/Makefile:1.8
--- othersrc/external/bsd/arfe/dt/Makefile:1.7 Wed Dec 2 23:39:51 2015
+++ othersrc/external/bsd/arfe/dt/Makefile Tue Jan 26 02:22:57 2016
@@ -1,12 +1,13 @@
-# $ARFE: Makefile 294 2015-11-27 16:33:21Z dyoung $
+# $ARFE: Makefile 319 2016-01-13 22:35:13Z dyoung $
NOMAN=
.include <bsd.own.mk>
+.include "../mk/helpers.mk"
-DPINCS+=core.h dec.h hex.h ipv4.h macaddr.h sym.h
PROG=dt
-SRCS+=core.c dec.c dt.c hex.c ipv4.c macaddr.c sym.c
+SRCS+=core.c dec.c dt.c eol.c hex.c ipv4.c macaddr.c space.c sym.c
#CPPFLAGS+=-DHB_DEBUG
#CPPFLAGS+=-DHB_ASSERT
+CPPFLAGS+=-I$(.CURDIR)/..
DBG+=-g -O3
#DBG+=-g -O0
#COPTS+=-pg
@@ -14,7 +15,10 @@ DBG+=-g -O3
CFLAGS+=-std=c99
WARNS=5
+DPINCS+=core.h dec.h eol.h hex.h ipv4.h macaddr.h space.h sym.h
+
CLEANFILES+=*.gcda *.gcno
+CLEANDIRFILES+=*.gcov
#
# profiling stuff
@@ -22,7 +26,11 @@ CLEANFILES+=*.gcda *.gcno
#DBG+=-fprofile-arcs -ftest-coverage
#LDADD+=-lgcov
-test:
+LIBHLOG != ${PRINTOBJDIROF} ${.CURDIR}/../hlog
+DPADD+=${LIBHLOG}/libhlog.a
+LDADD+=-L${LIBHLOG} -lhlog
+
+test: dependall
@sh $(.CURDIR)/testit
.include <bsd.prog.mk>
Index: othersrc/external/bsd/arfe/dt/README
diff -u othersrc/external/bsd/arfe/dt/README:1.12 othersrc/external/bsd/arfe/dt/README:1.13
--- othersrc/external/bsd/arfe/dt/README:1.12 Thu Dec 3 03:29:01 2015
+++ othersrc/external/bsd/arfe/dt/README Tue Jan 26 02:22:57 2016
@@ -1,5 +1,5 @@
-$ARFE: README 303 2015-12-03 03:26:34Z dyoung $
-$NetBSD: README,v 1.12 2015/12/03 03:29:01 dyoung Exp $
+$ARFE: README 323 2016-01-25 20:29:30Z dyoung $
+$NetBSD: README,v 1.13 2016/01/26 02:22:57 dyoung Exp $
DT---(d)ifferentiate (t)ext---finds a longest common subsequence (LCS)
of two texts where the numbers, "symbols" starting with a letter or
Index: othersrc/external/bsd/arfe/dt/core.c
diff -u othersrc/external/bsd/arfe/dt/core.c:1.6 othersrc/external/bsd/arfe/dt/core.c:1.7
--- othersrc/external/bsd/arfe/dt/core.c:1.6 Sat Dec 5 08:44:10 2015
+++ othersrc/external/bsd/arfe/dt/core.c Tue Jan 26 02:22:57 2016
@@ -1,5 +1,5 @@
-/* $NetBSD: core.c,v 1.6 2015/12/05 08:44:10 wiz Exp $ */
-/* $ARFE: core.c 302 2015-12-02 23:44:24Z dyoung $ */
+/* $NetBSD: core.c,v 1.7 2016/01/26 02:22:57 dyoung Exp $ */
+/* $ARFE: core.c 323 2016-01-25 20:29:30Z dyoung $ */
/*-
* Copyright (c) 2014,2015 David Young <[email protected]>
@@ -32,6 +32,7 @@
#include <stdbool.h>
#include <stdlib.h>
#include <string.h>
+#include <unistd.h> /* for getopt(3) */
#include <sys/param.h> /* for MAX() */
#include <sys/mman.h>
@@ -41,12 +42,27 @@
#include <netinet/in.h> /* for inet_net_pton */
#include <arpa/inet.h> /* for inet_net_pton */
-#include "core.h"
-#include "dec.h"
-#include "hex.h"
-#include "ipv4.h"
-#include "macaddr.h"
-#include "sym.h"
+#include "hlog/hlog.h"
+#include "dt/core.h"
+#include "dt/dec.h"
+#include "dt/eol.h"
+#include "dt/hex.h"
+#include "dt/ipv4.h"
+#include "dt/macaddr.h"
+#include "dt/space.h"
+#include "dt/sym.h"
+
+typedef struct _cbuf {
+ char string[sizeof("Space")];
+} cbuf_t;
+
+typedef struct _indices {
+ ssize_t i, j;
+} indices_t;
+
+typedef struct _score {
+ ssize_t del, ins, least, match;
+} score_t;
union freeslice;
typedef union freeslice freeslice_t;
@@ -60,13 +76,33 @@ static bool freeslices_initialized __ali
static freeslice_t *first_freeslice __aligned(COHERENCY_UNIT);
static freeslice_t freeslices[2048];
-static const int gap_open = 1, gap_extend = 6;
+static const int gap_open = 12, gap_extend = 2;
+#define INFINITY (SSIZE_MAX / 2)
static const subcell_t infinity = {
- .score = SSIZE_MAX / 2
- , .gaps = SSIZE_MAX / 2
+ .score = INFINITY
, .splitn = 0
};
+HLOG_SINK_DECL(match_candidate);
+HLOG_SINK_SHORT_DEFN(match_candidate, all);
+
+HLOG_SINK_DECL(match_clocc);
+HLOG_SINK_DECL(match_dec);
+HLOG_SINK_DECL(match_hex);
+HLOG_SINK_DECL(match_space);
+HLOG_SINK_DECL(match_sym);
+
+HLOG_SINK_SHORT_DEFN(match_clocc, all);
+HLOG_SINK_SHORT_DEFN(match_dec, match_clocc);
+HLOG_SINK_SHORT_DEFN(match_hex, match_clocc);
+HLOG_SINK_SHORT_DEFN(match_sym, match_clocc);
+HLOG_SINK_SHORT_DEFN(match_space, match_clocc);
+
+HLOG_SINK_DECL(find_clocc);
+HLOG_SINK_DECL(find_dec);
+HLOG_SINK_SHORT_DEFN(find_clocc, all);
+HLOG_SINK_SHORT_DEFN(find_dec, find_clocc);
+
static void
init_freeslices(void)
{
@@ -125,15 +161,15 @@ file_to_slice(const char *fn)
void *p;
if (fd == -1)
- err(EXIT_FAILURE, "%s: open", __func__);
+ err(EXIT_FAILURE, "%s: open(\"%s\")", __func__, fn);
if (fstat(fd, &st) == -1)
- err(EXIT_FAILURE, "%s: stat", __func__);
+ err(EXIT_FAILURE, "%s: fstat [%s]", __func__, fn);
p = mmap(NULL, st.st_size, PROT_READ, MAP_SHARED|MAP_FILE, fd, 0);
if (p == MAP_FAILED)
- err(EXIT_FAILURE, "%s: mmap", __func__);
+ err(EXIT_FAILURE, "%s: mmap [%s]", __func__, fn);
return newslice(p, st.st_size);
}
@@ -345,6 +381,25 @@ clocc_hash(const slice_t *s, const clocc
return hash;
}
+static bool
+cloccs_are_equal(const slice_t *ls, const slice_t *rs,
+ const clocc_t *locc, const clocc_t *rocc)
+{
+ int i, n;
+
+ dbg_assert(locc->first <= locc->last);
+
+ if (locc->last - locc->first != rocc->last - rocc->first)
+ return false;
+
+ n = locc->last - locc->first + 1;
+ for (i = 0; i < n; i++) {
+ if (get(ls, locc->first + i) != get(rs, rocc->first + i))
+ return false;
+ }
+ return true;
+}
+
/* Look up the class occurrence `occ` on slice `s` in the class occurrences
* `occs` occurring on slice `os`.
*/
@@ -356,18 +411,9 @@ clocc_cross_lookup(cloccs_t *occs, const
unsigned int hash = clocc_hash(s, occ);
clocc_bucket_t *bkt = &tbl->bucket[hash % __arraycount(tbl->bucket)];
clocc_t *bocc;
- int i, n;
TAILQ_FOREACH(bocc, bkt, bucket) {
- assert(bocc->first <= bocc->last);
- if (bocc->last - bocc->first != occ->last - occ->first)
- continue;
- n = bocc->last - bocc->first + 1;
- for (i = 0; i < n; i++) {
- if (get(os, bocc->first + i) != get(s, occ->first + i))
- break;
- }
- if (i == n)
+ if (cloccs_are_equal(os, s, bocc, occ))
return bocc;
}
return NULL;
@@ -459,7 +505,7 @@ hex_clocc(const scratch_t *scratch, int
column = cloccs_to_column(a, b);
- dbg2_printf("0x%jx @ %d / 0x%jx @ %d -> column %d\n",
+ HLOG_LAZY(&log_match_sym, "0x%jx @ %d / 0x%jx @ %d -> column %d",
a->val_u.u_uintmax, a->column,
b->val_u.u_uintmax, b->column, column);
@@ -538,9 +584,10 @@ dec_clocc(const scratch_t *scratch, int
align_decimal_points(&a->val_u.u_dec, &b->val_u.u_dec, &val.a, &val.b);
- dbg2_printf("%jd / 10^%d @ %d ; %jd / 10^%d @ %d -> column %d\n",
- val.a.num, val.a.log10denom, a->column,
- val.b.num, val.b.log10denom, b->column, column);
+ HLOG_LAZY(&log_match_dec,
+ "%jd / 10^%d @ %d / col %d ; %jd / 10^%d @ %d / col %d -> col %d",
+ val.a.num, val.a.log10denom, a->first, a->column,
+ val.b.num, val.b.log10denom, b->first, b->column, column);
switch (scratch->dec_op) {
case '-':
@@ -571,13 +618,53 @@ dec_clocc(const scratch_t *scratch, int
}
static chain_t *
+space_clocc(const scratch_t *scratch, int idx_a, int idx_b)
+{
+ const clocc_t * const a = &scratch->Aoccs.occs[idx_a],
+ * const b = &scratch->Boccs.occs[idx_b];
+
+ HLOG_LAZY(&log_match_space, "space @ %d ; space @ %d",
+ a->column,
+ b->column);
+
+ ssize_t m = a->last - a->first + 1, n = b->last - b->first + 1;
+ return newchain_with_match_idx(
+ strndup(getp(scratch->Abasis, a->first), MIN(m, n)), 0, idx_b);
+}
+
+static chain_t *
+eol_clocc(const scratch_t *scratch, int idx_a, int idx_b)
+{
+ const clocc_t * const a = &scratch->Aoccs.occs[idx_a],
+ * const b = &scratch->Boccs.occs[idx_b];
+
+ dbg2_printf("eol @ %d ; eol @ %d\n",
+ a->column,
+ b->column);
+
+ if (a->last - a->first != b->last - b->first)
+ goto out;
+
+ ssize_t i, n = a->last - a->first + 1;
+ for (i = 0; i < n; i++) {
+ if (get(scratch->Abasis, a->first + i) !=
+ get(scratch->Bbasis, b->first + i))
+ goto out;
+ }
+ return newchain_with_match_idx(
+ strndup(getp(scratch->Abasis, a->first), n), 0, idx_b);
+out:
+ return newchain_with_match_idx(strdup("\n"), 0, idx_b);
+}
+
+static chain_t *
sym_clocc(const scratch_t *scratch, int idx_a, int idx_b)
{
const clocc_t * const a = &scratch->Aoccs.occs[idx_a],
* const b = &scratch->Boccs.occs[idx_b];
const int column = cloccs_to_column(a, b);
- dbg2_printf("%s @ %d ; %s @ %d -> column %d\n",
+ HLOG_LAZY(&log_match_sym, "%s @ %d ; %s @ %d -> column %d",
a->val_u.u_sym, a->column,
b->val_u.u_sym, b->column, column);
@@ -598,6 +685,12 @@ clocc(const scratch_t *scratch, int idx_
(a->qual == b->qual) ? b->qual : QUAL_NONE);
}
+ if (b->kind == KIND_SPACE)
+ return space_clocc(scratch, idx_a, idx_b);
+
+ if (b->kind == KIND_EOL)
+ return eol_clocc(scratch, idx_a, idx_b);
+
if (b->kind == KIND_SYM)
return sym_clocc(scratch, idx_a, idx_b);
@@ -619,12 +712,65 @@ empty(void)
return newchain(strdup(""), 0);
}
+/* XXX things to try: prioritize candw.ins over other cases;
+ * XXX individually set candw.{ins,del,match} in clocc_score
+ * XXX
+ * XXX score numbers differently: between -gap_open * 4 and -gap_open
+ * XXX based on decimal in the same place, difference in log10 magnitude
+ * XXX (careful of sign!)
+ */
static inline ssize_t
-clocc_score(clocc_kind_t kind, ssize_t a, ssize_t b)
+clocc_score(clocc_kind_t kind, const score_t candw, const score_t cand)
{
- if (kind == KIND_SYM)
- return gap_extend * (MAX(a, b) + MIN(a, b)) / 2;
- return gap_extend * (MIN(a, b) - MAX(a, b)) - gap_open;
+ if (kind == KIND_SYM || kind == KIND_SPACE) {
+ score_t diff = {
+ .match = MAX(0, cand.match - candw.match)
+ , .ins = MAX(0, cand.ins - candw.ins)
+ , .del = MAX(0, cand.del - candw.del)
+ };
+ diff.least = MIN(diff.match, MIN(diff.ins, diff.del));
+
+ return MAX(diff.least - gap_extend * 2, gap_open * 2);
+ }
+#if 0
+ /* XXX this didn't work any better than -gap_open * 4 by itself */
+ return diff.least - gap_open * 4;
+#endif
+ return -gap_open * 4;
+}
+
+static int
+cbprintf(cbuf_t *bp, const char *fmt, ...)
+{
+ int rc;
+ va_list ap;
+
+ va_start(ap, fmt);
+ rc = vsnprintf(&bp->string[0], sizeof(bp->string), fmt, ap);
+ va_end(ap);
+ return (rc < 0 || rc >= (int)sizeof(bp->string)) ? -1 : 0;
+}
+
+static int
+char_to_string(cbuf_t *bp, int c)
+{
+ switch (c) {
+ case -1:
+ return cbprintf(bp, "Blank");
+ case '\t':
+ return cbprintf(bp, "Tab");
+ case ' ':
+ return cbprintf(bp, "Space");
+ case '\r':
+ return cbprintf(bp, "CR");
+ case '\n':
+ return cbprintf(bp, "NL");
+ default:
+ if (isspace((int)(unsigned char)c))
+ return cbprintf(bp, "0x%x", (unsigned char)c);
+ else
+ return cbprintf(bp, "%c", c);
+ }
}
static void
@@ -642,16 +788,18 @@ findsplitn(const slice_t *A, const slice
ssize_t A, B;
} wlen;
bool inside_clocc = false;
+ indices_t first = {.i = 0, .j = 0}, last = first;
+ tablescope_t * const scope = scratch->tablescope.halfw > 0 ?
+ &scratch->tablescope : NULL;
K[0] = scratch->K[0];
K[1] = scratch->K[1];
-
/* A copy of the K[] at which a clocc last began. */
Kclocc = scratch->Kclocc;
for (j = 0; j < n + 1; j++) {
K[0][j].match = infinity;
- K[0][j].ins.score = gap_open + j * gap_extend;
+ K[0][j].ins.score = gap_open + (j - 1) * gap_extend;
K[0][j].ins.splitn = 0;
K[0][j].del = infinity;
}
@@ -661,11 +809,40 @@ findsplitn(const slice_t *A, const slice
Kcur = K[1] + 1;
Kclocc++;
+ /* it costs nothing to match 0 characters of A with 0 characters of B */
+ Kprev[-1].match = (subcell_t) {.score = 0, .splitn = 0};
+ Kprev[-1].ins = infinity;
+
+ if (scope != NULL) {
+ printf("<table>\n");
+ printf("<tr><th> </th><th> </th>");
+ first = (indices_t) {.i = scope->i - scope->halfw,
+ .j = scope->j - scope->halfw};
+ last =
+ (indices_t) {.i = MIN(scope->i + scope->halfw - 1, m - 1),
+ .j = MIN(scope->j + scope->halfw - 1, n - 1)};
+ for (j = MAX(0, first.j); j <= last.j; j++) {
+ cbuf_t b;
+ const char c = get(B, j);
+ const int rc = char_to_string(&b, c);
+ assert(rc == 0);
+ printf("<th>%s</th>", b.string);
+ }
+ printf("</tr>\n");
+ printf("<tr><td> </td>");
+ for (j = MAX(-1, first.j - 1); j <= last.j; j++) {
+ printf("<td><div class=\"ins\">i <span class=\"score\">%zd</span>/<span class=\"splitn\">%d</span><div class=\"del\">d <span class=\"score\">%zd</span>/<span class=\"splitn\">%d</span></div><div class=\"match\">m <span class=\"score\">%zd</span>/<span class=\"splitn\">%d</span></div></td>",
+ Kprev[j].ins.score, Kprev[j].ins.splitn,
+ Kprev[j].del.score, Kprev[j].del.splitn,
+ Kprev[j].match.score, Kprev[j].match.splitn);
+ }
+ printf("</tr>\n");
+ }
+
for (i = 0; i < m; i++) {
- Kprev[-1].match = infinity;
Kcur[-1].match = infinity;
- Kprev[-1].del.score = gap_open + i * gap_extend;
- Kcur[-1].del.score = gap_open + (i + 1) * gap_extend;
+ Kprev[-1].del.score = gap_open + MAX(0, i - 1) * gap_extend;
+ Kcur[-1].del.score = gap_open + i * gap_extend;
Kprev[-1].ins = infinity;
Kcur[-1].ins = infinity;
const bool clocc_starts_this_row =
@@ -688,9 +865,7 @@ findsplitn(const slice_t *A, const slice
for (j = 0; j < n; j++) {
- struct {
- ssize_t del, ins, least, match;
- } cand, candw;
+ score_t cand, candw;
/* del ^
* ins <
@@ -713,12 +888,14 @@ findsplitn(const slice_t *A, const slice
Kcur[j].del.splitn = Kprev[j].match.splitn;
Kcur[j].del.score = cand.least;
+
dbg_printf("%s set K[%zd][%zd].del.score to %zd\n",
__func__, i, j, Kcur[j].del.score);
cand.match =
Kcur[j - 1].match.score + gap_open + gap_extend;
- cand.del = Kcur[j - 1].del.score + gap_open + gap_extend;
+ cand.del =
+ Kcur[j - 1].del.score + gap_open + gap_extend;
cand.ins = Kcur[j - 1].ins.score + gap_extend;
cand.least = MIN(cand.match, MIN(cand.del, cand.ins));
@@ -740,37 +917,39 @@ findsplitn(const slice_t *A, const slice
cand.match = Kprev[j - 1].match.score;
cand.del = Kprev[j - 1].del.score;
cand.ins = Kprev[j - 1].ins.score;
- cand.least =
- MIN(cand.match, MIN(cand.del, cand.ins));
if (Ai != get(B, j)) {
- cand.match += gap_open + gap_extend * 2;
- cand.del += gap_open + gap_extend * 2;
- cand.ins += gap_open + gap_extend * 2;
- cand.least += gap_open + gap_extend * 2;
+ cand.match += gap_open * 2 + gap_extend * 2;
+ cand.del += gap_extend + gap_open + gap_extend;
+ cand.ins += gap_extend + gap_open + gap_extend;
}
+ cand.least =
+ MIN(cand.match, MIN(cand.del, cand.ins));
+
if (clocc_ends_this_row &&
clocc_ends_in_slice_at(&scratch->Boccs,
scratch->Bbasis, B, j,
&wlen.B, &kind.B) &&
kind.A == kind.B && /* match kind */
wlen.B <= j + 1 /* must not begin outside of B */) {
+ candw.match = Kclocc[j - wlen.B].match.score;
+ candw.del = Kclocc[j - wlen.B].del.score;
+ candw.ins = Kclocc[j - wlen.B].ins.score;
+ candw.least = MIN(candw.match,
+ MIN(candw.del, candw.ins));
const ssize_t step =
- clocc_score(kind.A, wlen.A, wlen.B);
- dbg_printf("%s visits clocc boundary at A[%d] "
- "and B[%d] step %zu\n", __func__,
+ clocc_score(kind.A, candw, cand);
+ HLOG_LAZY(&log_match_candidate,
+ "%s visits clocc boundary at A[%d] "
+ "and B[%d] step %zd", __func__,
index_versus_basis(scratch->Abasis, A, i),
index_versus_basis(scratch->Bbasis, B, j),
step);
- candw.match =
- Kclocc[j - wlen.B].match.score + step;
- candw.del =
- Kclocc[j - wlen.B].del.score + step;
- candw.ins =
- Kclocc[j - wlen.B].ins.score + step;
- candw.least = MIN(candw.match,
- MIN(candw.del, candw.ins));
+ candw.match += step;
+ candw.del += step;
+ candw.ins += step;
+ candw.least += step;
} else {
candw = cand;
}
@@ -806,10 +985,33 @@ findsplitn(const slice_t *A, const slice
inside_clocc = false;
}
+ if (scope != NULL && first.i <= i && i <= last.i) {
+ cbuf_t Abuf;
+ printf("<tr>");
+ int rc = char_to_string(&Abuf, Ai);
+ assert(rc == 0);
+ printf("<td>%s</td>", Abuf.string);
+ for (j = MAX(-1, first.j - 1); j <= last.j; j++) {
+ cbuf_t Bbuf;
+ rc = char_to_string(&Bbuf, (j < 0) ? -1 : get(B, j));
+ assert(rc == 0);
+ printf("<td>%s / %s<div class=\"ins\">i <span class=\"score\">%zd</span>/<span class=\"splitn\">%d</span><div class=\"del\">d <span class=\"score\">%zd</span>/<span class=\"splitn\">%d</span></div><div class=\"match\">m <span class=\"score\">%zd</span>/<span class=\"splitn\">%d</span></div></td>",
+ Abuf.string, Bbuf.string,
+ Kcur[j].ins.score, Kcur[j].ins.splitn,
+ Kcur[j].del.score, Kcur[j].del.splitn,
+ Kcur[j].match.score, Kcur[j].match.splitn);
+ }
+ printf("</tr>\n");
+ }
Ktmp = Kprev;
Kprev = Kcur;
Kcur = Ktmp;
}
+ if (scope != NULL) {
+ printf("</table>\n");
+ fflush(stdout);
+ scratch->tablescope.halfw = 0;
+ }
#if 0
dbg_assert(Kcur[-1].score == m - 1 && Kprev[-1].score == m);
#endif
@@ -829,7 +1031,7 @@ count_records(const slice_t *A, const sl
{
const ssize_t m = length(A);
const ssize_t n = length(B);
- ssize_t i, i0, j;
+ ssize_t i, j;
cell_t *K[2], *Kclocc, *Kcur, *Kprev, *Ktmp;
struct {
clocc_kind_t A, B;
@@ -838,21 +1040,31 @@ count_records(const slice_t *A, const sl
ssize_t A, B;
} wlen;
bool inside_clocc = false;
+ indices_t first = {.i = 0, .j = 0}, last = first;
+ gapcell_t *G[2], *Gclocc, *Gcur, *Gprev, *Gtmp;
+ ssize_t i0;
+ tablescope_t * const scope = scratch->tablescope.halfw > 0 ?
+ &scratch->tablescope : NULL;
K[0] = scratch->K[0];
K[1] = scratch->K[1];
-
/* A copy of the K[] at which a clocc last began. */
Kclocc = scratch->Kclocc;
+ Gclocc = scratch->Gclocc;
+ G[0] = scratch->G[0];
+ G[1] = scratch->G[1];
+
for (j = 0; j < n + 1; j++) {
K[0][j].match = infinity;
-
- K[0][j].ins.score = gap_open + j * gap_extend;
- K[0][j].ins.gaps = 1;
+ K[0][j].ins.score = gap_open + (j - 1) * gap_extend;
K[0][j].ins.splitn = 0;
-
K[0][j].del = infinity;
+
+ G[0][j].match = INFINITY;
+ G[0][j].match = 0;
+ G[0][j].ins = 1;
+ G[0][j].del = INFINITY;
}
/* make -1 a valid index on all arrays */
@@ -860,58 +1072,98 @@ count_records(const slice_t *A, const sl
Kcur = K[1] + 1;
Kclocc++;
+ Gprev = G[0] + 1;
+ Gcur = G[1] + 1;
+ Gclocc++;
+
+ /* it costs nothing to match 0 characters of A with 0 characters of B */
+ Kprev[-1].match = (subcell_t) {.score = 0, .splitn = 0};
+ Kprev[-1].ins = infinity;
+
ssize_t prev_gaps = SSIZE_MAX;
ssize_t prev_score = SSIZE_MAX;
ssize_t prev_diff = SSIZE_MAX;
-#if 0
- ssize_t prev_avg = SSIZE_MAX;
- ssize_t prev_avg_diff = 0;
-#endif
size_t nrec = 0;
+ if (scope != NULL) {
+ printf("<table>\n");
+ printf("<tr><th> </th><th> </th>");
+ first = (indices_t) {.i = scope->i - scope->halfw,
+ .j = scope->j - scope->halfw};
+ last =
+ (indices_t) {.i = MIN(scope->i + scope->halfw - 1, m - 1),
+ .j = MIN(scope->j + scope->halfw - 1, n - 1)};
+ for (j = MAX(0, first.j); j <= last.j; j++) {
+ cbuf_t b;
+ const char c = get(B, j);
+ const int rc = char_to_string(&b, c);
+ assert(rc == 0);
+ printf("<th>%s</th>", b.string);
+ }
+ printf("</tr>\n");
+ printf("<tr><td> </td>");
+ for (j = MAX(-1, first.j - 1); j <= last.j; j++) {
+ printf("<td><div class=\"ins\">i <span class=\"score\">%zd</span>/<span class=\"splitn\">%d</span><div class=\"del\">d <span class=\"score\">%zd</span>/<span class=\"splitn\">%d</span></div><div class=\"match\">m <span class=\"score\">%zd</span>/<span class=\"splitn\">%d</span></div></td>",
+ Kprev[j].ins.score, Kprev[j].ins.splitn,
+ Kprev[j].del.score, Kprev[j].del.splitn,
+ Kprev[j].match.score, Kprev[j].match.splitn);
+ }
+ printf("</tr>\n");
+ }
+
for (i0 = i = 0; ; i0++, i++) {
- Kprev[-1].match = infinity;
Kcur[-1].match = infinity;
/* TBD: i0, or i? */
- Kprev[-1].del.score = gap_open + i0 * gap_extend;
- Kprev[-1].del.gaps = 1;
- Kcur[-1].del.score = gap_open + (i0 + 1) * gap_extend;
- Kcur[-1].del.gaps = 1;
+ Kprev[-1].del.score = gap_open + MAX(0, i0 - 1) * gap_extend;
+ Kcur[-1].del.score = gap_open + i0 * gap_extend;
Kprev[-1].ins = infinity;
Kcur[-1].ins = infinity;
+ Gprev[-1].match = INFINITY;
+ Gcur[-1].match = INFINITY;
+ Gprev[-1].del = 1;
+ Gcur[-1].del = 1;
+ Gprev[-1].ins = INFINITY;
+ Gcur[-1].ins = INFINITY;
+
if (i == m) {
dbg_assert(!inside_clocc);
+ if (scope != NULL && nrec == 0) {
+ printf("</table>\n");
+ fflush(stdout);
+ }
nrec++;
const ssize_t least =
MIN(MIN(Kprev[n - 1].del.score,
Kprev[n - 1].ins.score),
Kprev[n - 1].match.score);
ssize_t gaps;
- int splitn;
- if (least == Kprev[n - 1].del.score) {
- splitn = Kprev[n - 1].del.splitn;
- gaps = Kprev[n - 1].del.gaps;
- } else if (least == Kprev[n - 1].ins.score) {
+ int oosplitn, osplitn, splitn;
+ if (least == Kprev[n - 1].ins.score) {
+ oosplitn = Kprev[n - 1].ins.oosplitn;
+ osplitn = Kprev[n - 1].ins.osplitn;
splitn = Kprev[n - 1].ins.splitn;
- gaps = Kprev[n - 1].ins.gaps;
+ gaps = Gprev[n - 1].ins;
+ } else if (least == Kprev[n - 1].del.score) {
+ oosplitn = Kprev[n - 1].del.oosplitn;
+ osplitn = Kprev[n - 1].del.osplitn;
+ splitn = Kprev[n - 1].del.splitn;
+ gaps = Gprev[n - 1].del;
} else /* if (least == Kprev[n - 1].match.score) */ {
+ oosplitn = Kprev[n - 1].match.oosplitn;
+ osplitn = Kprev[n - 1].match.osplitn;
splitn = Kprev[n - 1].match.splitn;
- gaps = Kprev[n - 1].match.gaps;
+ gaps = Gprev[n - 1].match;
}
#if 1
const ssize_t avg = gaps / nrec;
- fprintf(stderr, "%s: %zu records, splitn %d, gaps %zu, d gaps %zd, score %zd -> %zd, "
+ fprintf(stderr, "%s: %zu records, splitn %d|%d|%d, gaps %zu, d gaps %zd, score %zd -> %zd, "
"d %zd, dd %zd, avg g / r %zd\n",
- __func__, nrec, splitn, gaps, gaps - prev_gaps, prev_score, least,
+ __func__, nrec, oosplitn, osplitn, splitn, gaps, gaps - prev_gaps, prev_score, least,
least - prev_score, least - prev_score - prev_diff, avg);
#endif
prev_diff = least - prev_score;
prev_score = least;
-#if 0
- prev_avg_diff = avg - prev_avg;
- prev_avg = avg;
-#endif
prev_gaps = gaps;
i = 0;
}
@@ -925,6 +1177,8 @@ count_records(const slice_t *A, const sl
dbg_printf("%s: row %zd, clocc starts\n", __func__, i);
memcpy(&Kclocc[-1], &Kprev[-1],
sizeof(Kclocc[0]) * (n + 1));
+ memcpy(&Gclocc[-1], &Gprev[-1],
+ sizeof(Gclocc[0]) * (n + 1));
inside_clocc = true;
}
@@ -936,9 +1190,7 @@ count_records(const slice_t *A, const sl
for (j = 0; j < n; j++) {
- struct {
- ssize_t del, ins, least, match;
- } cand, candw;
+ score_t cand, candw;
/* del ^
* ins <
@@ -955,40 +1207,46 @@ count_records(const slice_t *A, const sl
if (cand.least == cand.ins) {
Kcur[j].del = Kprev[j].ins;
- Kcur[j].del.gaps++;
+ Gcur[j].del = Gprev[j].ins + 1;
} else if (cand.least == cand.del) {
Kcur[j].del = Kprev[j].del;
+ Gcur[j].del = Gprev[j].del;
#if 1
if (i == 0)
- Kcur[j].del.gaps++;
+ Gcur[j].del++;
#endif
} else /* if (cand.least == cand.match) */ {
Kcur[j].del = Kprev[j].match;
- Kcur[j].del.gaps++;
+ Gcur[j].del = Gprev[j].match + 1;
}
Kcur[j].del.score = cand.least;
- if (i == 0)
+ if (i == 0) {
+ Kcur[j].del.oosplitn = Kcur[j].del.osplitn;
+ Kcur[j].del.osplitn = Kcur[j].del.splitn;
Kcur[j].del.splitn = j;
+ }
dbg_printf("%s set K[%zd][%zd].del.score to %zd\n",
__func__, i, j, Kcur[j].del.score);
cand.match =
Kcur[j - 1].match.score + gap_open + gap_extend;
- cand.del = Kcur[j - 1].del.score + gap_open + gap_extend;
+ cand.del =
+ Kcur[j - 1].del.score + gap_open + gap_extend;
cand.ins = Kcur[j - 1].ins.score + gap_extend;
cand.least = MIN(cand.match, MIN(cand.del, cand.ins));
if (cand.least == cand.ins) {
Kcur[j].ins = Kcur[j - 1].ins;
+ Gcur[j].ins = Gcur[j - 1].ins;
} else if (cand.least == cand.del) {
Kcur[j].ins = Kcur[j - 1].del;
- Kcur[j].ins.gaps++;
+ Gcur[j].ins = Gcur[j - 1].del + 1;
} else /* if (cand.least == cand.match) */ {
Kcur[j].ins = Kcur[j - 1].match;
- Kcur[j].ins.gaps++;
+ Gcur[j].ins = Gcur[j - 1].match + 1;
}
Kcur[j].ins.score = cand.least;
@@ -999,66 +1257,104 @@ count_records(const slice_t *A, const sl
cand.match = Kprev[j - 1].match.score;
cand.del = Kprev[j - 1].del.score;
cand.ins = Kprev[j - 1].ins.score;
- cand.least =
- MIN(cand.match, MIN(cand.del, cand.ins));
if (Ai != get(B, j)) {
- cand.match += gap_open + gap_extend * 2;
- cand.del += gap_open + gap_extend * 2;
- cand.ins += gap_open + gap_extend * 2;
- cand.least += gap_open + gap_extend * 2;
+ cand.match += gap_open * 2 + gap_extend * 2;
+ cand.del += gap_extend + gap_open + gap_extend;
+ cand.ins += gap_extend + gap_open + gap_extend;
}
+ cand.least =
+ MIN(cand.match, MIN(cand.del, cand.ins));
+
if (clocc_ends_this_row &&
clocc_ends_in_slice_at(&scratch->Boccs,
scratch->Bbasis, B, j,
&wlen.B, &kind.B) &&
kind.A == kind.B && /* match kind */
wlen.B <= j + 1 /* must not begin outside of B */) {
+ candw.match = Kclocc[j - wlen.B].match.score;
+ candw.del = Kclocc[j - wlen.B].del.score;
+ candw.ins = Kclocc[j - wlen.B].ins.score;
+ candw.least = MIN(candw.match,
+ MIN(candw.del, candw.ins));
const ssize_t step =
- clocc_score(kind.A, wlen.A, wlen.B);
- dbg_printf("%s visits clocc boundary at A[%d] "
- "and B[%d] step %zu\n", __func__,
+ clocc_score(kind.A, candw, cand);
+ HLOG_LAZY(&log_match_candidate,
+ "%s visits clocc boundary at A[%d] "
+ "and B[%d] step %zd", __func__,
index_versus_basis(scratch->Abasis, A, i),
index_versus_basis(scratch->Bbasis, B, j),
step);
dbg_assert(i != 0);
- candw.match =
- Kclocc[j - wlen.B].match.score + step;
- candw.del =
- Kclocc[j - wlen.B].del.score + step;
- candw.ins =
- Kclocc[j - wlen.B].ins.score + step;
- candw.least = MIN(candw.match,
- MIN(candw.del, candw.ins));
+ candw.match += step;
+ candw.del += step;
+ candw.ins += step;
+ candw.least += step;
} else {
candw = cand;
}
const ssize_t least = MIN(cand.least, candw.least);
if (least == cand.ins) {
Kcur[j].match = Kprev[j - 1].ins;
- if (i == 0)
+ Gcur[j].match = Gprev[j - 1].ins;
+ if (i == 0) {
+ Kcur[j].match.oosplitn =
+ Kcur[j].match.osplitn;
+ Kcur[j].match.osplitn =
+ Kcur[j].match.splitn;
Kcur[j].match.splitn = j - 1;
+ }
} else if (least == cand.del) {
Kcur[j].match = Kprev[j - 1].del;
- if (i == 0)
+ Gcur[j].match = Gprev[j - 1].del;
+ if (i == 0) {
+ Kcur[j].match.oosplitn =
+ Kcur[j].match.osplitn;
+ Kcur[j].match.osplitn =
+ Kcur[j].match.splitn;
Kcur[j].match.splitn = j - 1;
+ }
} else if (least == cand.match) {
Kcur[j].match = Kprev[j - 1].match;
- if (i == 0)
+ Gcur[j].match = Gprev[j - 1].match;
+ if (i == 0) {
+ Kcur[j].match.oosplitn =
+ Kcur[j].match.osplitn;
+ Kcur[j].match.osplitn =
+ Kcur[j].match.splitn;
Kcur[j].match.splitn = j - 1;
+ }
} else if (least == candw.ins) {
Kcur[j].match = Kclocc[j - wlen.B].ins;
- if (i == 0)
+ Gcur[j].match = Gclocc[j - wlen.B].ins;
+ if (i == 0) {
+ Kcur[j].match.oosplitn =
+ Kcur[j].match.osplitn;
+ Kcur[j].match.osplitn =
+ Kcur[j].match.splitn;
Kcur[j].match.splitn = j - wlen.B;
+ }
} else if (least == candw.del) {
Kcur[j].match = Kclocc[j - wlen.B].del;
- if (i == 0)
+ Gcur[j].match = Gclocc[j - wlen.B].del;
+ if (i == 0) {
+ Kcur[j].match.oosplitn =
+ Kcur[j].match.osplitn;
+ Kcur[j].match.osplitn =
+ Kcur[j].match.splitn;
Kcur[j].match.splitn = j - wlen.B;
+ }
} else /* if (least == candw.match) */ {
Kcur[j].match = Kclocc[j - wlen.B].match;
- if (i == 0)
+ Gcur[j].match = Gclocc[j - wlen.B].match;
+ if (i == 0) {
+ Kcur[j].match.oosplitn =
+ Kcur[j].match.osplitn;
+ Kcur[j].match.osplitn =
+ Kcur[j].match.splitn;
Kcur[j].match.splitn = j - wlen.B;
+ }
}
Kcur[j].match.score = least;
dbg_printf("%s set K[%zd][%zd].match.score to %zu%s\n",
@@ -1070,9 +1366,31 @@ count_records(const slice_t *A, const sl
inside_clocc = false;
}
+ if (scope != NULL && nrec == 0 && first.i <= i && i <= last.i) {
+ cbuf_t Abuf;
+ printf("<tr>");
+ int rc = char_to_string(&Abuf, Ai);
+ assert(rc == 0);
+ printf("<td>%s</td>", Abuf.string);
+ for (j = MAX(-1, first.j - 1); j <= last.j; j++) {
+ cbuf_t Bbuf;
+ rc = char_to_string(&Bbuf, (j < 0) ? -1 : get(B, j));
+ assert(rc == 0);
+ printf("<td>%s / %s<div class=\"ins\">i <span class=\"score\">%zd</span>/<span class=\"splitn\">%d</span><div class=\"del\">d <span class=\"score\">%zd</span>/<span class=\"splitn\">%d</span></div><div class=\"match\">m <span class=\"score\">%zd</span>/<span class=\"splitn\">%d</span></div></td>",
+ Abuf.string, Bbuf.string,
+ Kcur[j].ins.score, Kcur[j].ins.splitn,
+ Kcur[j].del.score, Kcur[j].del.splitn,
+ Kcur[j].match.score, Kcur[j].match.splitn);
+ }
+ printf("</tr>\n");
+ }
Ktmp = Kprev;
Kprev = Kcur;
Kcur = Ktmp;
+
+ Gtmp = Gprev;
+ Gprev = Gcur;
+ Gcur = Gtmp;
}
}
@@ -1275,8 +1593,10 @@ findlcs(const slice_t *A, const slice_t
0, n - 1, &start, &end, kind);
if (ivlidx.B != -1) {
+ score_t zero = {0, 0, 0, 0};
+ score_t one = {1, 1, 1, 1};
return findlcs_return(m, n, origin, lcsp,
- clocc_score(kind, end - start + 1, m),
+ clocc_score(kind, zero, one),
clocc(scratch, ivlidx.A, ivlidx.B));
}
}
@@ -1324,15 +1644,19 @@ findlcs(const slice_t *A, const slice_t
}
void
-scratch_init(scratch_t *scratch, size_t n, slice_t *Abasis, slice_t *Bbasis)
-{
- scratch->L1 = malloc(sizeof(scratch->L1[0]) * (n + 1));
- scratch->L2 = malloc(sizeof(scratch->L2[0]) * (n + 1));
+scratch_init(scratch_t *scratch, size_t n, slice_t *Abasis, slice_t *Bbasis,
+ tablescope_t scope)
+{
+ scratch->G[0] = malloc(sizeof(scratch->G[0][0]) * (n + 1));
+ scratch->G[1] = malloc(sizeof(scratch->G[1][0]) * (n + 1));
+ scratch->Gclocc = malloc(sizeof(scratch->Gclocc[0]) * (n + 1));
+
scratch->K[0] = malloc(sizeof(scratch->K[0][0]) * (n + 1));
scratch->K[1] = malloc(sizeof(scratch->K[1][0]) * (n + 1));
scratch->Kclocc = malloc(sizeof(scratch->Kclocc[0]) * (n + 1));
scratch->Abasis = Abasis;
scratch->Bbasis = Bbasis;
+ scratch->tablescope = scope;
}
#if 0
@@ -1363,10 +1687,47 @@ newocc(cloccs_t *o)
errno = ENOMEM;
return NULL;
}
+ o->occs[o->noccs].counterparts = 0;
return &o->occs[o->noccs++];
}
static void
+emit_eol_clocc(eol_detection_t *d, void *arg)
+{
+ cloccs_t *o = arg;
+ clocc_t *occ;
+
+ dbg_printf("found eol [%d, %d]\n", d->d_idx.start, d->d_idx.stop);
+
+ if ((occ = newocc(o)) == NULL)
+ err(EXIT_FAILURE, "%s: newocc", __func__);
+
+ occ->column = d->d_column.start;
+ occ->first = d->d_idx.start;
+ occ->last = d->d_idx.stop;
+ occ->kind = KIND_EOL;
+ occ->qual = QUAL_NONE;
+}
+
+static void
+emit_space_clocc(space_detection_t *d, void *arg)
+{
+ cloccs_t *o = arg;
+ clocc_t *occ;
+
+ dbg_printf("found space @ [%d, %d]\n", d->d_idx.start, d->d_idx.stop);
+
+ if ((occ = newocc(o)) == NULL)
+ err(EXIT_FAILURE, "%s: newocc", __func__);
+
+ occ->column = d->d_column.start;
+ occ->first = d->d_idx.start;
+ occ->last = d->d_idx.stop;
+ occ->kind = KIND_SPACE;
+ occ->qual = QUAL_NONE;
+}
+
+static void
emit_sym_clocc(sym_detection_t *d, void *arg)
{
cloccs_t *o = arg;
@@ -1467,7 +1828,8 @@ emit_dec_clocc(dec_detection_t *d, void
int sign = 1;
char *next;
- dbg_printf("found decimal string %s @ [%d, %d]\n", d->d_digits,
+ HLOG_LAZY(&log_find_dec,
+ "found decimal string %s @ [%d, %d]", d->d_digits,
d->d_idx.start, d->d_idx.stop);
if ((occ = newocc(o)) == NULL)
@@ -1619,13 +1981,15 @@ cloccs_hash(cloccs_t *o, const slice_t *
}
void
-cloccs_init(cloccs_t *o, const slice_t *s)
+cloccs_init(cloccs_t *o, const slice_t *s, bool nosyms)
{
dec_parser_t *dec_parser;
hex_parser_t *hex_parser;
ipv4_parser_t *ipv4_parser;
mac_parser_t *mac_parser;
sym_parser_t *sym_parser;
+ eol_parser_t *eol_parser;
+ space_parser_t *space_parser;
size_t column;
size_t n = length(s);
unsigned int j;
@@ -1633,6 +1997,47 @@ cloccs_init(cloccs_t *o, const slice_t *
o->noccs = 0;
clocc_htbl_init(&o->htbl);
+ space_parser = space_parser_alloc(emit_space_clocc, o);
+ if (space_parser == NULL)
+ goto parse_eol;
+
+ column = 0;
+ for (j = 0; j < n; j++) {
+ char c;
+
+ c = get(s, j);
+ ++column;
+ space_parser_drive(space_parser, j, column, c);
+ if (c == '\n')
+ column = 0;
+ }
+ space_parser_drive(space_parser, n, column + 1, -1);
+
+ space_parser_free(space_parser);
+
+parse_eol:
+ eol_parser = eol_parser_alloc(emit_eol_clocc, o);
+ if (eol_parser == NULL)
+ goto parse_sym;
+
+ column = 0;
+ for (j = 0; j < n; j++) {
+ char c;
+
+ c = get(s, j);
+ ++column;
+ eol_parser_drive(eol_parser, j, column, c);
+ if (c == '\n')
+ column = 0;
+ }
+ eol_parser_drive(eol_parser, n, column + 1, -1);
+
+ eol_parser_free(eol_parser);
+
+parse_sym:
+ if (nosyms)
+ goto parse_mac;
+
sym_parser = sym_parser_alloc(emit_sym_clocc, o);
if (sym_parser == NULL)
goto parse_mac;
@@ -1763,12 +2168,45 @@ printchain(chain_t *c)
}
void
+count_match_occ_transform_counterparts(cloccs_t *Moccs, cloccs_t *Toccs,
+ const slice_t *match, const slice_t *transform)
+{
+ int i;
+
+ for (i = 0; i < Toccs->noccs; i++) {
+ clocc_t *Tocc = &Toccs->occs[i];
+ clocc_t *Mcur;
+ clocc_t *Mfirst = clocc_cross_lookup(Moccs, match, transform,
+ Tocc);
+
+ if (Mfirst == NULL)
+ continue;
+
+ Mfirst->counterparts++;
+
+ for (Mcur = TAILQ_NEXT(Mfirst, bucket);
+ Mcur != NULL;
+ Mcur = TAILQ_NEXT(Mcur, bucket)) {
+ if (!cloccs_are_equal(match, match, Mfirst, Mcur))
+ continue;
+ Mcur->counterparts++;
+ }
+ }
+}
+
+void
emit_transformed_text(chain_t *O, cloccs_t *Moccs, cloccs_t *Toccs,
const slice_t *match, const slice_t *transform)
{
chainelt_t *ce;
int i, next;
+ /* Walk the output chain, O, and at each chain element that
+ * findlcs() made by aligning a match class-occurrence with
+ * a template class-occurrence, link the match class
+ * occurrence (occ) to the chain element (ce) through
+ * occ->match.
+ */
TAILQ_FOREACH(ce, &O->c_head, ce_link) {
clocc_t *occ = &Moccs->occs[ce->ce_match_idx];
@@ -1789,6 +2227,21 @@ emit_transformed_text(chain_t *O, cloccs
fputc(c, stdout); // XXX check error
}
+ if (Mocc != NULL && Mocc->match == NULL) {
+ clocc_t *cand; /* candidate */
+
+ for (cand = TAILQ_NEXT(Mocc, bucket);
+ cand != NULL;
+ cand = TAILQ_NEXT(cand, bucket)) {
+ if (cand->match == NULL)
+ continue;
+ if (!cloccs_are_equal(match, match, Mocc, cand))
+ continue;
+ Mocc = cand;
+ break;
+ }
+ }
+
if (Mocc != NULL && Mocc->match != NULL) {
fprintf(stdout, "%s", Mocc->match->ce_content);
next = Tocc->last + 1;
@@ -1801,3 +2254,70 @@ emit_transformed_text(chain_t *O, cloccs
fputc(c, stdout); // XXX check error
}
}
+
+int
+arfe_parse_options(int argc, char **argv, int *argcp, char ***argvp,
+ arfe_options_t *opts)
+{
+ int ch;
+ char *end;
+ static const arfe_options_t arfe_default_options = {
+ .docount = false
+ , .nosyms = false
+ , .scope = {.i = 0, .j = 0, .halfw = 0}
+ };
+
+ *opts = arfe_default_options;
+
+ hlog_start(HLOG_M_STDERR, LOG_DAEMON);
+ if (hlog_set_state("all", HLOG_SINK_S_PASS, 0) == -1)
+ err(EXIT_FAILURE, "%s: hlog_set_state", __func__);
+
+ while ((ch = getopt(argc, argv, "ch:i:j:l:L:S")) != -1) {
+ switch (ch) {
+ case 'l':
+ case 'L':
+ if (hlog_set_state(optarg,
+ (ch == 'l') ? HLOG_SINK_S_ON : HLOG_SINK_S_OFF,
+ 0) == -1) {
+ err(EXIT_FAILURE, "%s: hlog_set_state",
+ __func__);
+ }
+ break;
+ case 'i':
+ opts->scope.i = strtol(optarg, &end, 10);
+ if (opts->scope.i == LONG_MIN || opts->scope.i == LONG_MAX)
+ err(EXIT_FAILURE, "error parsing -i");
+ else if (*end != '\0')
+ errx(EXIT_FAILURE, "invalid -i argument");
+ break;
+ case 'j':
+ opts->scope.j = strtol(optarg, &end, 10);
+ if (opts->scope.j == LONG_MIN || opts->scope.j == LONG_MAX)
+ err(EXIT_FAILURE, "error parsing -j");
+ else if (*end != '\0')
+ errx(EXIT_FAILURE, "invalid -j argument");
+ break;
+ case 'h':
+ opts->scope.halfw = strtol(optarg, &end, 10);
+ if (opts->scope.halfw == LONG_MIN || opts->scope.halfw == LONG_MAX)
+ err(EXIT_FAILURE, "error parsing -w");
+ else if (*end != '\0' || opts->scope.halfw < 0)
+ errx(EXIT_FAILURE, "invalid -w argument");
+ break;
+ case 'c':
+ opts->docount = true;
+ break;
+ case 'S':
+ opts->nosyms = true;
+ break;
+ case '?':
+ default:
+ return -1;
+ }
+ }
+
+ *argcp = argc - optind;
+ *argvp = argv + optind;
+ return 0;
+}
Index: othersrc/external/bsd/arfe/dt/macaddr.c
diff -u othersrc/external/bsd/arfe/dt/macaddr.c:1.6 othersrc/external/bsd/arfe/dt/macaddr.c:1.7
--- othersrc/external/bsd/arfe/dt/macaddr.c:1.6 Thu Dec 3 03:29:01 2015
+++ othersrc/external/bsd/arfe/dt/macaddr.c Tue Jan 26 02:22:57 2016
@@ -1,5 +1,5 @@
-/* $NetBSD: macaddr.c,v 1.6 2015/12/03 03:29:01 dyoung Exp $ */
-/* $ARFE: macaddr.c 302 2015-12-02 23:44:24Z dyoung $ */
+/* $NetBSD: macaddr.c,v 1.7 2016/01/26 02:22:57 dyoung Exp $ */
+/* $ARFE: macaddr.c 323 2016-01-25 20:29:30Z dyoung $ */
/*-
* Copyright (c) 2014,2015 David Young <[email protected]>
Index: othersrc/external/bsd/arfe/dt/core.h
diff -u othersrc/external/bsd/arfe/dt/core.h:1.5 othersrc/external/bsd/arfe/dt/core.h:1.6
--- othersrc/external/bsd/arfe/dt/core.h:1.5 Thu Dec 3 03:29:01 2015
+++ othersrc/external/bsd/arfe/dt/core.h Tue Jan 26 02:22:57 2016
@@ -1,5 +1,5 @@
-/* $NetBSD: core.h,v 1.5 2015/12/03 03:29:01 dyoung Exp $ */
-/* $ARFE: core.h 302 2015-12-02 23:44:24Z dyoung $ */
+/* $NetBSD: core.h,v 1.6 2016/01/26 02:22:57 dyoung Exp $ */
+/* $ARFE: core.h 323 2016-01-25 20:29:30Z dyoung $ */
/*-
* Copyright (c) 2014,2015 David Young <[email protected]>
@@ -101,7 +101,9 @@ typedef enum {
, KIND_INTMAX_0xHEX
, KIND_IPv4
, KIND_MACADDR
+ , KIND_SPACE
, KIND_SYM
+ , KIND_EOL
} clocc_kind_t;
typedef enum {
@@ -132,6 +134,7 @@ typedef struct clocc {
uint8_t u_macaddr[6];
char u_sym[64]; /* XXX don't limit like this */
} val_u;
+ unsigned int counterparts;
TAILQ_ENTRY(clocc) bucket;
chainelt_t *match;
} clocc_t;
@@ -161,30 +164,39 @@ enum {
};
typedef struct clocc_htbl {
- clocc_bucket_t bucket[4096];
+ clocc_bucket_t bucket[8192];
} clocc_htbl_t;
typedef struct cloccs {
int noccs;
- clocc_t occs[4096];
+ clocc_t occs[8192];
clocc_htbl_t htbl;
} cloccs_t;
typedef struct subcell {
ssize_t score;
+ int oosplitn;
+ int osplitn;
int splitn;
- ssize_t gaps;
} subcell_t;
typedef struct cell {
subcell_t ins, del, match;
} cell_t;
+typedef struct gapcell {
+ ssize_t ins, del, match;
+} gapcell_t;
+
+typedef struct _tablescope {
+ ssize_t halfw, i, j;
+} tablescope_t;
+
typedef struct scratch {
- size_t *L1; /* n + 1 */
- size_t *L2; /* n + 1 */
- cell_t *K[2]; /* 2 x (n + 1) */
- cell_t *Kclocc; /* n + 1 */
+ gapcell_t *G[2]; /* 2 x (n + 1) */
+ gapcell_t *Gclocc; /* n + 1 */
+ cell_t *K[2]; /* 2 x (n + 1) */
+ cell_t *Kclocc; /* n + 1 */
cloccs_t Aoccs, Boccs;
slice_t *Abasis, *Bbasis;
char ipv4_op; /* 'p' for prefix (smallest containing subnet),
@@ -199,18 +211,31 @@ typedef struct scratch {
char hex_op; /* '&' or '|' for bitwise-AND and -OR, 'l' for copy
* left argument
*/
+ /* If halfw != 0, count_records() emits debugging information
+ * for all cells of the dynamic program in the region
+ * [i - halfw, i + halfw - 1] x [j - halfw, j + halfw - 1].
+ */
+ tablescope_t tablescope;
} scratch_t;
+typedef struct _arfe_options {
+ tablescope_t scope;
+ bool docount, nosyms;
+} arfe_options_t;
+
+int arfe_parse_options(int, char **, int *, char ***, arfe_options_t *);
slice_t *file_to_slice(const char *);
void emit_transformed_text(chain_t *, cloccs_t *, cloccs_t *,
const slice_t *, const slice_t *);
void printchain(chain_t *);
-void cloccs_init(cloccs_t *, const slice_t *);
+void cloccs_init(cloccs_t *, const slice_t *, bool);
void cloccs_hash(cloccs_t *, const slice_t *);
chain_t *findlcs(const slice_t *, const slice_t *, scratch_t *, const origin_t,
size_t *);
-void scratch_init(scratch_t *, size_t, slice_t *, slice_t *);
+void scratch_init(scratch_t *, size_t, slice_t *, slice_t *, tablescope_t);
ssize_t length(const slice_t *);
size_t count_records(const slice_t *, const slice_t *, scratch_t *);
+void count_match_occ_transform_counterparts(cloccs_t *, cloccs_t *,
+ const slice_t *, const slice_t *);
#endif /* _CORE_H_ */
Index: othersrc/external/bsd/arfe/dt/dec.c
diff -u othersrc/external/bsd/arfe/dt/dec.c:1.3 othersrc/external/bsd/arfe/dt/dec.c:1.4
--- othersrc/external/bsd/arfe/dt/dec.c:1.3 Thu Dec 3 03:29:01 2015
+++ othersrc/external/bsd/arfe/dt/dec.c Tue Jan 26 02:22:57 2016
@@ -1,5 +1,5 @@
-/* $NetBSD: dec.c,v 1.3 2015/12/03 03:29:01 dyoung Exp $ */
-/* $ARFE: dec.c 302 2015-12-02 23:44:24Z dyoung $ */
+/* $NetBSD: dec.c,v 1.4 2016/01/26 02:22:57 dyoung Exp $ */
+/* $ARFE: dec.c 323 2016-01-25 20:29:30Z dyoung $ */
/*-
* Copyright (c) 2014,2015 David Young <[email protected]>
@@ -245,8 +245,10 @@ dec_parser_drive(dec_parser_t *p, int id
case DEC_OP_EMIT:
if (p->p_top == NULL)
dbg_printf("decimal string too long\n");
- *p->p_top = '\0';
- (*p->p_emitter)(&p->p_detection, p->p_arg);
+ else {
+ *p->p_top = '\0';
+ (*p->p_emitter)(&p->p_detection, p->p_arg);
+ }
break;
case DEC_OP_NONE:
break;
Index: othersrc/external/bsd/arfe/dt/dec.h
diff -u othersrc/external/bsd/arfe/dt/dec.h:1.3 othersrc/external/bsd/arfe/dt/dec.h:1.4
--- othersrc/external/bsd/arfe/dt/dec.h:1.3 Thu Dec 3 03:29:01 2015
+++ othersrc/external/bsd/arfe/dt/dec.h Tue Jan 26 02:22:57 2016
@@ -1,5 +1,5 @@
-/* $NetBSD: dec.h,v 1.3 2015/12/03 03:29:01 dyoung Exp $ */
-/* $ARFE: dec.h 302 2015-12-02 23:44:24Z dyoung $ */
+/* $NetBSD: dec.h,v 1.4 2016/01/26 02:22:57 dyoung Exp $ */
+/* $ARFE: dec.h 323 2016-01-25 20:29:30Z dyoung $ */
/*-
* Copyright (c) 2014,2015 David Young <[email protected]>
Index: othersrc/external/bsd/arfe/dt/macaddr.h
diff -u othersrc/external/bsd/arfe/dt/macaddr.h:1.3 othersrc/external/bsd/arfe/dt/macaddr.h:1.4
--- othersrc/external/bsd/arfe/dt/macaddr.h:1.3 Thu Dec 3 03:29:01 2015
+++ othersrc/external/bsd/arfe/dt/macaddr.h Tue Jan 26 02:22:57 2016
@@ -1,5 +1,5 @@
-/* $NetBSD: macaddr.h,v 1.3 2015/12/03 03:29:01 dyoung Exp $ */
-/* $ARFE: macaddr.h 302 2015-12-02 23:44:24Z dyoung $ */
+/* $NetBSD: macaddr.h,v 1.4 2016/01/26 02:22:57 dyoung Exp $ */
+/* $ARFE: macaddr.h 323 2016-01-25 20:29:30Z dyoung $ */
/*-
* Copyright (c) 2014,2015 David Young <[email protected]>
Index: othersrc/external/bsd/arfe/dt/dt.c
diff -u othersrc/external/bsd/arfe/dt/dt.c:1.17 othersrc/external/bsd/arfe/dt/dt.c:1.18
--- othersrc/external/bsd/arfe/dt/dt.c:1.17 Sat Dec 5 08:47:23 2015
+++ othersrc/external/bsd/arfe/dt/dt.c Tue Jan 26 02:22:57 2016
@@ -1,5 +1,5 @@
-/* $NetBSD: dt.c,v 1.17 2015/12/05 08:47:23 wiz Exp $ */
-/* $ARFE: dt.c 302 2015-12-02 23:44:24Z dyoung $ */
+/* $NetBSD: dt.c,v 1.18 2016/01/26 02:22:57 dyoung Exp $ */
+/* $ARFE: dt.c 323 2016-01-25 20:29:30Z dyoung $ */
/*-
* Copyright (c) 2014,2015 David Young <[email protected]>
@@ -44,36 +44,42 @@
#include "core.h"
-static void usage(void) __attribute__((__noreturn__));
+static void usage(void) __dead;
static void
usage(void)
{
- fprintf(stderr, "usage: %s string-a string-b\n", getprogname());
+ fprintf(stderr, "usage: %s file1 file2\n", getprogname());
exit(EXIT_FAILURE);
}
int
-main(int argc, char **argv)
+main(int argc0, char **argv0)
{
slice_t *left, *match;
size_t n;
chain_t *O;
scratch_t scratch;
size_t lcs;
+ arfe_options_t opts;
+ int argc;
+ char **argv;
- setprogname(argv[0]);
+ setprogname(argv0[0]);
- if (argc != 3)
+ if (arfe_parse_options(argc0, argv0, &argc, &argv, &opts) == -1)
usage();
- left = file_to_slice(argv[1]);
- match = file_to_slice(argv[2]);
+ if (argc != 2)
+ usage();
+
+ left = file_to_slice(argv[0]);
+ match = file_to_slice(argv[1]);
n = length(match);
- scratch_init(&scratch, n, left, match);
- cloccs_init(&scratch.Aoccs, left);
- cloccs_init(&scratch.Boccs, match);
+ scratch_init(&scratch, n, left, match, opts.scope);
+ cloccs_init(&scratch.Aoccs, left, opts.nosyms);
+ cloccs_init(&scratch.Boccs, match, opts.nosyms);
cloccs_hash(&scratch.Boccs, match);
scratch.macaddr_op = 'p';
@@ -81,10 +87,12 @@ main(int argc, char **argv)
scratch.dec_op = '-';
scratch.hex_op = '&';
-#if 0
- const size_t nrec = count_records(left, match, &scratch);
- fprintf(stderr, "%zu records\n", nrec);
-#endif
+ if (opts.docount) {
+ const size_t nrec = count_records(left, match, &scratch);
+ printf("%zu\n", nrec);
+ return EXIT_SUCCESS;
+ }
+
O = findlcs(left, match, &scratch, (origin_t){0, 0}, &lcs);
dbg_printf("lcs = %zu\n", lcs);
Index: othersrc/external/bsd/arfe/dt/hex.c
diff -u othersrc/external/bsd/arfe/dt/hex.c:1.8 othersrc/external/bsd/arfe/dt/hex.c:1.9
--- othersrc/external/bsd/arfe/dt/hex.c:1.8 Thu Dec 3 03:29:01 2015
+++ othersrc/external/bsd/arfe/dt/hex.c Tue Jan 26 02:22:57 2016
@@ -1,5 +1,5 @@
-/* $NetBSD: hex.c,v 1.8 2015/12/03 03:29:01 dyoung Exp $ */
-/* $ARFE: hex.c 302 2015-12-02 23:44:24Z dyoung $ */
+/* $NetBSD: hex.c,v 1.9 2016/01/26 02:22:57 dyoung Exp $ */
+/* $ARFE: hex.c 323 2016-01-25 20:29:30Z dyoung $ */
/*-
* Copyright (c) 2014,2015 David Young <[email protected]>
@@ -241,8 +241,10 @@ hex_parser_drive(hex_parser_t *p, int id
case HEX_OP_EMIT:
if (p->p_top == NULL)
dbg_printf("hex string too long\n");
- *p->p_top = '\0';
- (*p->p_emitter)(&p->p_detection, p->p_arg, use0x);
+ else {
+ *p->p_top = '\0';
+ (*p->p_emitter)(&p->p_detection, p->p_arg, use0x);
+ }
break;
case HEX_OP_NONE:
break;
Index: othersrc/external/bsd/arfe/dt/hex.h
diff -u othersrc/external/bsd/arfe/dt/hex.h:1.8 othersrc/external/bsd/arfe/dt/hex.h:1.9
--- othersrc/external/bsd/arfe/dt/hex.h:1.8 Thu Dec 3 03:29:01 2015
+++ othersrc/external/bsd/arfe/dt/hex.h Tue Jan 26 02:22:57 2016
@@ -1,5 +1,5 @@
-/* $NetBSD: hex.h,v 1.8 2015/12/03 03:29:01 dyoung Exp $ */
-/* $ARFE: hex.h 302 2015-12-02 23:44:24Z dyoung $ */
+/* $NetBSD: hex.h,v 1.9 2016/01/26 02:22:57 dyoung Exp $ */
+/* $ARFE: hex.h 323 2016-01-25 20:29:30Z dyoung $ */
/*-
* Copyright (c) 2014,2015 David Young <[email protected]>
Index: othersrc/external/bsd/arfe/dt/ipv4.c
diff -u othersrc/external/bsd/arfe/dt/ipv4.c:1.8 othersrc/external/bsd/arfe/dt/ipv4.c:1.9
--- othersrc/external/bsd/arfe/dt/ipv4.c:1.8 Thu Dec 3 03:29:01 2015
+++ othersrc/external/bsd/arfe/dt/ipv4.c Tue Jan 26 02:22:57 2016
@@ -1,5 +1,5 @@
-/* $NetBSD: ipv4.c,v 1.8 2015/12/03 03:29:01 dyoung Exp $ */
-/* $ARFE: ipv4.c 302 2015-12-02 23:44:24Z dyoung $ */
+/* $NetBSD: ipv4.c,v 1.9 2016/01/26 02:22:57 dyoung Exp $ */
+/* $ARFE: ipv4.c 323 2016-01-25 20:29:30Z dyoung $ */
/*-
* Copyright (c) 2014,2015 David Young <[email protected]>
Index: othersrc/external/bsd/arfe/dt/ipv4.h
diff -u othersrc/external/bsd/arfe/dt/ipv4.h:1.8 othersrc/external/bsd/arfe/dt/ipv4.h:1.9
--- othersrc/external/bsd/arfe/dt/ipv4.h:1.8 Thu Dec 3 03:29:01 2015
+++ othersrc/external/bsd/arfe/dt/ipv4.h Tue Jan 26 02:22:57 2016
@@ -1,5 +1,5 @@
-/* $NetBSD: ipv4.h,v 1.8 2015/12/03 03:29:01 dyoung Exp $ */
-/* $ARFE: ipv4.h 302 2015-12-02 23:44:24Z dyoung $ */
+/* $NetBSD: ipv4.h,v 1.9 2016/01/26 02:22:57 dyoung Exp $ */
+/* $ARFE: ipv4.h 323 2016-01-25 20:29:30Z dyoung $ */
/*-
* Copyright (c) 2014,2015 David Young <[email protected]>
Index: othersrc/external/bsd/arfe/dt/portability.h
diff -u othersrc/external/bsd/arfe/dt/portability.h:1.4 othersrc/external/bsd/arfe/dt/portability.h:1.5
--- othersrc/external/bsd/arfe/dt/portability.h:1.4 Thu Dec 3 03:29:01 2015
+++ othersrc/external/bsd/arfe/dt/portability.h Tue Jan 26 02:22:57 2016
@@ -1,5 +1,5 @@
-/* $NetBSD: portability.h,v 1.4 2015/12/03 03:29:01 dyoung Exp $ */
-/* $ARFE: portability.h 302 2015-12-02 23:44:24Z dyoung $ */
+/* $NetBSD: portability.h,v 1.5 2016/01/26 02:22:57 dyoung Exp $ */
+/* $ARFE: portability.h 323 2016-01-25 20:29:30Z dyoung $ */
#ifndef __NetBSD__
#define __noinline /* */
Index: othersrc/external/bsd/arfe/dt/testit
diff -u othersrc/external/bsd/arfe/dt/testit:1.4 othersrc/external/bsd/arfe/dt/testit:1.5
--- othersrc/external/bsd/arfe/dt/testit:1.4 Wed Dec 2 23:39:51 2015
+++ othersrc/external/bsd/arfe/dt/testit Tue Jan 26 02:22:57 2016
@@ -1,5 +1,5 @@
#!/bin/sh
-# $ARFE: testit 297 2015-12-02 23:16:23Z dyoung $
+# $ARFE: testit 319 2016-01-13 22:35:13Z dyoung $
DT=${OBJDIR:-.}/dt
@@ -17,8 +17,6 @@ while read l r ; do
${DT} t/$l t/$r > $o
fi
done <<EOF
-netstat-s.0 netstat-s.1
-netstat-s.1 netstat-s.0
daily-output-2 daily-output-5
daily-output-5 daily-output-2
daily-output-5 daily-output-6
@@ -33,4 +31,6 @@ wm0-a wm0-b
wm0-b wm0-a
if-re0 if-wm0
if-wm0 if-re0
+netstat-s.0 netstat-s.1
+netstat-s.1 netstat-s.0
EOF
Index: othersrc/external/bsd/arfe/dt/sym.c
diff -u othersrc/external/bsd/arfe/dt/sym.c:1.1 othersrc/external/bsd/arfe/dt/sym.c:1.2
--- othersrc/external/bsd/arfe/dt/sym.c:1.1 Wed Dec 2 23:42:45 2015
+++ othersrc/external/bsd/arfe/dt/sym.c Tue Jan 26 02:22:57 2016
@@ -1,5 +1,5 @@
-/* $NetBSD: sym.c,v 1.1 2015/12/02 23:42:45 dyoung Exp $ */
-/* $ARFE$ */
+/* $NetBSD: sym.c,v 1.2 2016/01/26 02:22:57 dyoung Exp $ */
+/* $ARFE: sym.c 316 2015-12-17 21:34:56Z dyoung $ */
/*-
* Copyright (c) 2014,2015 David Young <[email protected]>
@@ -204,8 +204,10 @@ sym_parser_drive(sym_parser_t *p, int id
case SYM_OP_EMIT:
if (p->p_top == NULL)
dbg_printf("symbol too long\n");
- *p->p_top = '\0';
- (*p->p_emitter)(&p->p_detection, p->p_arg);
+ else {
+ *p->p_top = '\0';
+ (*p->p_emitter)(&p->p_detection, p->p_arg);
+ }
break;
case SYM_OP_NONE:
break;
Index: othersrc/external/bsd/arfe/dt/sym.h
diff -u othersrc/external/bsd/arfe/dt/sym.h:1.1 othersrc/external/bsd/arfe/dt/sym.h:1.2
--- othersrc/external/bsd/arfe/dt/sym.h:1.1 Wed Dec 2 23:42:45 2015
+++ othersrc/external/bsd/arfe/dt/sym.h Tue Jan 26 02:22:57 2016
@@ -1,5 +1,5 @@
-/* $NetBSD: sym.h,v 1.1 2015/12/02 23:42:45 dyoung Exp $ */
-/* $ARFE$ */
+/* $NetBSD: sym.h,v 1.2 2016/01/26 02:22:57 dyoung Exp $ */
+/* $ARFE: sym.h 314 2015-12-09 16:22:21Z dyoung $ */
/*-
* Copyright (c) 2014,2015 David Young <[email protected]>
Index: othersrc/external/bsd/arfe/dt/rr/ifconfig.0-logger-ifconfig.0
diff -u othersrc/external/bsd/arfe/dt/rr/ifconfig.0-logger-ifconfig.0:1.6 othersrc/external/bsd/arfe/dt/rr/ifconfig.0-logger-ifconfig.0:1.7
--- othersrc/external/bsd/arfe/dt/rr/ifconfig.0-logger-ifconfig.0:1.6 Wed Dec 2 23:39:51 2015
+++ othersrc/external/bsd/arfe/dt/rr/ifconfig.0-logger-ifconfig.0 Tue Jan 26 02:22:57 2016
@@ -1,10 +1,10 @@
wm0: flags=0<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 0
-capabilities=2bf80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Tx,UDP6CSUM_Tx>
-enabled=2bf80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Tx,UDP6CSUM_Tx>
-address: 00:0a:0b:cd:01:ef
-media: Ethernet autoselect (0baseT full-duplex)
-status: active
-input: 0 packets, 0 bytes, 0 multicasts
-output: 0 packets, 0 bytes, 0 multicasts
-inet 10.0.1.17 netmask 0xffffff00 broadcast 10.0.1.255
-inet6 fe80::20a:bff:fecd:1ef%wm0 prefixlen 0 scopeid 0x1
+ capabilities=2bf80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Tx,UDP6CSUM_Tx>
+ enabled=2bf80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Tx,UDP6CSUM_Tx>
+ address: 00:0a:0b:cd:01:ef
+ media: Ethernet autoselect (0baseT full-duplex)
+ status: active
+ input: 0 packets, 0 bytes, 0 multicasts
+ output: 0 packets, 0 bytes, 0 multicasts
+ inet 10.0.1.17 netmask 0xffffff00 broadcast 10.0.1.255
+ inet6 fe80::20a:bff:fecd:1ef%wm0 prefixlen 0 scopeid 0x1
Index: othersrc/external/bsd/arfe/dt/rr/logger-ifconfig.0-ifconfig.0
diff -u othersrc/external/bsd/arfe/dt/rr/logger-ifconfig.0-ifconfig.0:1.6 othersrc/external/bsd/arfe/dt/rr/logger-ifconfig.0-ifconfig.0:1.7
--- othersrc/external/bsd/arfe/dt/rr/logger-ifconfig.0-ifconfig.0:1.6 Wed Dec 2 23:39:51 2015
+++ othersrc/external/bsd/arfe/dt/rr/logger-ifconfig.0-ifconfig.0 Tue Jan 26 02:22:57 2016
@@ -1,10 +1,10 @@
wm0: flags=0<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 0
-capabilities=2bf80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Tx,UDP6CSUM_Tx>
-enabled=2bf80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Tx,UDP6CSUM_Tx>
-address: 00:0a:0b:cd:01:ef
-media: Ethernet autoselect (0baseT full-duplex)
-status: active
-input: 0 packets, 0 bytes, 0 multicasts
-output: 0 packets, 0 bytes, 0 multicasts
-inet 10.0.1.17 netmask 0xffffff00 broadcast 10.0.1.255
-inet6 fe80::20a:bff:fecd:1ef%wm0 prefixlen 0 scopeid 0x1
+ capabilities=2bf80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Tx,UDP6CSUM_Tx>
+ enabled=2bf80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Tx,UDP6CSUM_Tx>
+ address: 00:0a:0b:cd:01:ef
+ media: Ethernet autoselect (0baseT full-duplex)
+ status: active
+ input: 0 packets, 0 bytes, 0 multicasts
+ output: 0 packets, 0 bytes, 0 multicasts
+ inet 10.0.1.17 netmask 0xffffff00 broadcast 10.0.1.255
+ inet6 fe80::20a:bff:fecd:1ef%wm0 prefixlen 0 scopeid 0x1
Index: othersrc/external/bsd/arfe/it/Makefile
diff -u othersrc/external/bsd/arfe/it/Makefile:1.6 othersrc/external/bsd/arfe/it/Makefile:1.7
--- othersrc/external/bsd/arfe/it/Makefile:1.6 Wed Dec 2 23:39:51 2015
+++ othersrc/external/bsd/arfe/it/Makefile Tue Jan 26 02:22:57 2016
@@ -1,10 +1,12 @@
NOMAN=
.include <bsd.own.mk>
+.include "../mk/helpers.mk"
+DPINCS+=core.h dec.h eol.h hex.h ipv4.h macaddr.h space.h sym.h
PROG=it
-SRCS+=core.c dec.c hex.c ipv4.c it.c macaddr.c sym.c
+SRCS+=core.c dec.c eol.c hex.c ipv4.c it.c macaddr.c space.c sym.c
#CPPFLAGS+=-DHB_DEBUG
-CPPFLAGS+=-I$(.CURDIR)/../dt
+CPPFLAGS+=-I$(.CURDIR)/..
DBG+=-g -O3
#DBG+=-g -O0
#COPTS+=-pg
@@ -14,4 +16,11 @@ WARNS=5
.PATH: $(.CURDIR)/../dt
+LIBHLOG != ${PRINTOBJDIROF} ${.CURDIR}/../hlog
+DPADD+=${LIBHLOG}/libhlog.a
+LDADD+=-L${LIBHLOG} -lhlog
+
+test: dependall
+ @echo "No tests to run."
+
.include <bsd.prog.mk>
Index: othersrc/external/bsd/arfe/it/README
diff -u othersrc/external/bsd/arfe/it/README:1.10 othersrc/external/bsd/arfe/it/README:1.11
--- othersrc/external/bsd/arfe/it/README:1.10 Thu Dec 3 03:29:01 2015
+++ othersrc/external/bsd/arfe/it/README Tue Jan 26 02:22:57 2016
@@ -1,5 +1,5 @@
-$ARFE: README 304 2015-12-03 03:27:26Z dyoung $
-$NetBSD: README,v 1.10 2015/12/03 03:29:01 dyoung Exp $
+$ARFE: README 323 2016-01-25 20:29:30Z dyoung $
+$NetBSD: README,v 1.11 2016/01/26 02:22:57 dyoung Exp $
IT---(i)ntegrate (t)ext---is a variation on DT that adds decimal numbers
instead of subtracts, and bitwise-ORs hexadecimal numbers instead of
Index: othersrc/external/bsd/arfe/it/it.c
diff -u othersrc/external/bsd/arfe/it/it.c:1.3 othersrc/external/bsd/arfe/it/it.c:1.4
--- othersrc/external/bsd/arfe/it/it.c:1.3 Sat Dec 5 08:47:24 2015
+++ othersrc/external/bsd/arfe/it/it.c Tue Jan 26 02:22:57 2016
@@ -1,4 +1,4 @@
-/* $NetBSD: it.c,v 1.3 2015/12/05 08:47:24 wiz Exp $ */
+/* $NetBSD: it.c,v 1.4 2016/01/26 02:22:57 dyoung Exp $ */
/* $ARFE: dt.c 247 2015-09-14 02:52:57Z dyoung $ */
/*-
@@ -36,44 +36,51 @@
#include <stdlib.h>
#include <limits.h>
#include <string.h>
+#include <unistd.h>
#include <sys/param.h> /* for MAX() */
#include <sys/queue.h> /* for TAILQ_*() */
#include <sys/mman.h>
#include <sys/stat.h>
-#include "core.h"
+#include "dt/core.h"
-static void usage(void) __attribute__((__noreturn__));
+static void usage(void) __dead;
static void
usage(void)
{
- fprintf(stderr, "usage: %s string-a string-b\n", getprogname());
+ fprintf(stderr, "usage: %s file1 file2\n", getprogname());
exit(EXIT_FAILURE);
}
int
-main(int argc, char **argv)
+main(int argc0, char **argv0)
{
slice_t *left, *match;
size_t n;
chain_t *O;
scratch_t scratch;
size_t lcs;
+ arfe_options_t opts;
+ int argc;
+ char **argv;
- setprogname(argv[0]);
+ setprogname(argv0[0]);
- if (argc != 3)
+ if (arfe_parse_options(argc0, argv0, &argc, &argv, &opts) == -1)
usage();
- left = file_to_slice(argv[1]);
- match = file_to_slice(argv[2]);
+ if (argc != 2)
+ usage();
+
+ left = file_to_slice(argv[0]);
+ match = file_to_slice(argv[1]);
n = length(match);
- scratch_init(&scratch, n, left, match);
- cloccs_init(&scratch.Aoccs, left);
- cloccs_init(&scratch.Boccs, match);
+ scratch_init(&scratch, n, left, match, opts.scope);
+ cloccs_init(&scratch.Aoccs, left, opts.nosyms);
+ cloccs_init(&scratch.Boccs, match, opts.nosyms);
cloccs_hash(&scratch.Boccs, match);
scratch.macaddr_op = 'p';
@@ -81,6 +88,12 @@ main(int argc, char **argv)
scratch.dec_op = '+';
scratch.hex_op = '|';
+ if (opts.docount) {
+ const size_t nrec = count_records(left, match, &scratch);
+ printf("%zu\n", nrec);
+ return EXIT_SUCCESS;
+ }
+
O = findlcs(left, match, &scratch, (origin_t){0, 0}, &lcs);
dbg_printf("lcs = %zu\n", lcs);
Index: othersrc/external/bsd/arfe/tt/Makefile
diff -u othersrc/external/bsd/arfe/tt/Makefile:1.4 othersrc/external/bsd/arfe/tt/Makefile:1.5
--- othersrc/external/bsd/arfe/tt/Makefile:1.4 Wed Dec 2 23:39:51 2015
+++ othersrc/external/bsd/arfe/tt/Makefile Tue Jan 26 02:22:57 2016
@@ -1,19 +1,26 @@
NOMAN=
.include <bsd.own.mk>
+.include "../mk/helpers.mk"
PROG=tt
-SRCS+=core.c dec.c hex.c ipv4.c macaddr.c sym.c tt.c
+SRCS+=core.c dec.c eol.c hex.c ipv4.c macaddr.c space.c sym.c tt.c
#CPPFLAGS+=-DHB_DEBUG
-CPPFLAGS+=-DHB_ASSERT
-CPPFLAGS+=-I$(.CURDIR)/../dt
-DBG+=-g -O3
-#DBG+=-g -O0
+#CPPFLAGS+=-DHB_ASSERT
+CPPFLAGS+=-I$(.CURDIR)/..
+#DBG+=-g -O3
+DBG+=-g -O0
#COPTS+=-pg
#LDFLAGS+=-pg
CFLAGS+=-std=c99
WARNS=5
-test: ${.OBJDIR}/${PROG}
+LIBHLOG != ${PRINTOBJDIROF} ${.CURDIR}/../hlog
+DPADD+=${LIBHLOG}/libhlog.a
+LDADD+=-L${LIBHLOG} -lhlog
+
+DPINCS+=dec.h eol.h hex.h ipv4.h macaddr.h space.h sym.h
+
+test: dependall
@OBJDIR=${.OBJDIR} sh testit.sh
.PATH: $(.CURDIR)/../dt
Index: othersrc/external/bsd/arfe/tt/tt.c
diff -u othersrc/external/bsd/arfe/tt/tt.c:1.4 othersrc/external/bsd/arfe/tt/tt.c:1.5
--- othersrc/external/bsd/arfe/tt/tt.c:1.4 Sat Dec 5 08:47:24 2015
+++ othersrc/external/bsd/arfe/tt/tt.c Tue Jan 26 02:22:57 2016
@@ -1,4 +1,4 @@
-/* $NetBSD: tt.c,v 1.4 2015/12/05 08:47:24 wiz Exp $ */
+/* $NetBSD: tt.c,v 1.5 2016/01/26 02:22:57 dyoung Exp $ */
/* $ARFE$ */
/*-
@@ -42,52 +42,66 @@
#include <sys/mman.h>
#include <sys/stat.h>
-#include "core.h"
+#include "dt/core.h"
+#include "hlog/hlog.h"
-static void usage(void) __attribute__((__noreturn__));
+static void usage(void) __dead;
static void
usage(void)
{
- fprintf(stderr, "usage: %s string-a string-b\n", getprogname());
+ fprintf(stderr, "usage: %s input match-template transform-template\n",
+ getprogname());
exit(EXIT_FAILURE);
}
int
-main(int argc, char **argv)
+main(int argc0, char **argv0)
{
slice_t *left, *match, *transform;
size_t n;
chain_t *O;
scratch_t scratch;
size_t lcs;
- cloccs_t Toccs;
+ cloccs_t *Toccs;
+ arfe_options_t opts;
+ int argc;
+ char **argv;
- setprogname(argv[0]);
+ setprogname(argv0[0]);
- if (argc != 4)
+ arfe_parse_options(argc0, argv0, &argc, &argv, &opts);
+
+ if (argc != 3)
usage();
- left = file_to_slice(argv[1]);
- match = file_to_slice(argv[2]);
+ left = file_to_slice(argv[0]);
+ match = file_to_slice(argv[1]);
n = length(match);
- scratch_init(&scratch, n, left, match);
- cloccs_init(&scratch.Aoccs, left);
- cloccs_init(&scratch.Boccs, match);
+ scratch_init(&scratch, n, left, match, opts.scope);
+ cloccs_init(&scratch.Aoccs, left, opts.nosyms);
+ cloccs_init(&scratch.Boccs, match, opts.nosyms);
cloccs_hash(&scratch.Boccs, match);
scratch.macaddr_op = 'l';
scratch.ipv4_op = 'l';
scratch.dec_op = 'l';
scratch.hex_op = 'l';
- transform = file_to_slice(argv[3]);
- cloccs_init(&Toccs, transform);
+ transform = file_to_slice(argv[2]);
+
+ if ((Toccs = malloc(sizeof(*Toccs))) == NULL)
+ err(EXIT_FAILURE, "%s: could not allocate Toccs", __func__);
+
+ cloccs_init(Toccs, transform, opts.nosyms);
+
+ count_match_occ_transform_counterparts(&scratch.Boccs, Toccs,
+ match, transform);
O = findlcs(left, match, &scratch, (origin_t){0, 0}, &lcs);
dbg_printf("lcs = %zu\n", lcs);
- emit_transformed_text(O, &scratch.Boccs, &Toccs, match, transform);
+ emit_transformed_text(O, &scratch.Boccs, Toccs, match, transform);
dbg_printf("\n");
return EXIT_SUCCESS;
Index: othersrc/external/bsd/arfe/tt/README
diff -u othersrc/external/bsd/arfe/tt/README:1.7 othersrc/external/bsd/arfe/tt/README:1.8
--- othersrc/external/bsd/arfe/tt/README:1.7 Thu Dec 3 03:29:02 2015
+++ othersrc/external/bsd/arfe/tt/README Tue Jan 26 02:22:57 2016
@@ -1,5 +1,5 @@
-$ARFE: README 304 2015-12-03 03:27:26Z dyoung $
-$NetBSD: README,v 1.7 2015/12/03 03:29:02 dyoung Exp $
+$ARFE: README 323 2016-01-25 20:29:30Z dyoung $
+$NetBSD: README,v 1.8 2016/01/26 02:22:57 dyoung Exp $
TT---(t)ransform (t)ext---transforms its input based on a
match/transform-template pair that exemplifies the changes that should
Index: othersrc/external/bsd/arfe/tt/testit.sh
diff -u othersrc/external/bsd/arfe/tt/testit.sh:1.2 othersrc/external/bsd/arfe/tt/testit.sh:1.3
--- othersrc/external/bsd/arfe/tt/testit.sh:1.2 Fri Sep 11 02:12:57 2015
+++ othersrc/external/bsd/arfe/tt/testit.sh Tue Jan 26 02:22:57 2016
@@ -2,8 +2,13 @@
TT=${OBJDIR:-.}/tt
-for i in $(ls tests)
-do
+if [ $# -eq 0 ]; then
+ set -- $(ls tests)
+fi
+
+while [ $# -ne 0 ]; do
+ i="$1"
+ shift
[ x${i##[0-9]} = x${i} ] && continue
d=./tests/${i}
if [ -e $d/expected-output ]; then
Added files:
Index: othersrc/external/bsd/arfe/Makefile.inc
diff -u /dev/null othersrc/external/bsd/arfe/Makefile.inc:1.1
--- /dev/null Tue Jan 26 02:22:57 2016
+++ othersrc/external/bsd/arfe/Makefile.inc Tue Jan 26 02:22:56 2016
@@ -0,0 +1,5 @@
+tags: $(SRCS) $(DPINCS) $(INCS)
+ @rm -f $(.CURDIR)/tags
+ @ctags -atwd $(.ALLSRC)
+ @sort -o $(.CURDIR)/tags $(.CURDIR)/tags
+
Index: othersrc/external/bsd/arfe/dt/eol.c
diff -u /dev/null othersrc/external/bsd/arfe/dt/eol.c:1.1
--- /dev/null Tue Jan 26 02:22:57 2016
+++ othersrc/external/bsd/arfe/dt/eol.c Tue Jan 26 02:22:57 2016
@@ -0,0 +1,192 @@
+/* $NetBSD: eol.c,v 1.1 2016/01/26 02:22:57 dyoung Exp $ */
+/* $ARFE: eol.c 319 2016-01-13 22:35:13Z dyoung $ */
+
+/*-
+ * Copyright (c) 2014,2015 David Young <[email protected]>
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
+ * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
+ * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
+ * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
+ * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+#include <assert.h>
+#include <ctype.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "core.h"
+#include "eol.h"
+
+typedef enum eol_op {
+ EOL_OP_NONE = 0
+ , EOL_OP_PUSH
+ , EOL_OP_EMIT
+ , EOL_OP_EMIT_ONE
+ , EOL_OP_START
+} eol_op_t;
+
+struct eol_state;
+typedef struct eol_state eol_state_t;
+
+typedef struct eol_transition {
+ const eol_state_t *t_newstate;
+ eol_op_t t_op;
+} eol_transition_t;
+
+struct eol_state {
+ const char *s_descr;
+ eol_transition_t s_space;
+ eol_transition_t s_newline;
+ eol_transition_t s_other;
+ eol_transition_t s_eof;
+};
+
+enum eol_state_idxs {
+ READ_SPACE
+ , READ_EOF
+ , READ_OTHER
+};
+
+#define read_space eol_states[READ_SPACE]
+#define read_eof eol_states[READ_EOF]
+#define read_other eol_states[READ_OTHER]
+
+static const eol_state_t eol_states[] = {[READ_EOF] = {
+ .s_descr = "read eof",
+ .s_other = {&read_eof, EOL_OP_NONE},
+}, [READ_OTHER] = {
+ .s_descr = "read other",
+ .s_other = {&read_other, EOL_OP_NONE},
+ .s_space = {&read_space, EOL_OP_START},
+ .s_newline = {&read_other, EOL_OP_EMIT_ONE},
+}, [READ_SPACE] = {
+ .s_descr = "read space",
+ .s_space = {&read_space, EOL_OP_PUSH},
+ .s_newline = {&read_other, EOL_OP_EMIT},
+ .s_other = {&read_other, EOL_OP_NONE},
+}};
+
+struct eol_parser {
+ eol_detection_t p_detection;
+#define p_column p_detection.d_column
+#define p_chars p_detection.d_chars
+#define p_idx p_detection.d_idx
+ char *p_top;
+ const eol_state_t *p_state;
+ eol_emitter_t p_emitter;
+ void *p_arg;
+};
+
+void
+eol_parser_free(eol_parser_t *p)
+{
+ free(p);
+}
+
+static void
+eol_parser_init(eol_parser_t *p, eol_emitter_t emitter, void *arg)
+{
+ memset(p, 0, sizeof(*p));
+ p->p_state = &read_other;
+ p->p_emitter = emitter;
+ p->p_arg = arg;
+}
+
+eol_parser_t *
+eol_parser_alloc(eol_emitter_t emitter, void *arg)
+{
+ eol_parser_t *p;
+
+ p = malloc(sizeof(*p));
+ if (p == NULL)
+ return NULL;
+
+ eol_parser_init(p, emitter, arg);
+ return p;
+}
+
+static const char *
+eol_op_string(eol_op_t op)
+{
+ switch (op) {
+ case EOL_OP_NONE:
+ return "none";
+ case EOL_OP_EMIT:
+ return "emit";
+ case EOL_OP_PUSH:
+ return "push";
+ case EOL_OP_START:
+ return "start";
+ default:
+ return "unknown";
+ }
+}
+
+void
+eol_parser_drive(eol_parser_t *p, int idx, int column, int c)
+{
+ const eol_transition_t *t;
+ const eol_state_t *ns, *s;
+
+ s = p->p_state;
+
+ if (c == -1 && (ns = s->s_eof.t_newstate) != NULL)
+ t = &s->s_eof;
+ else if (c == '\n' && (ns = s->s_newline.t_newstate) != NULL)
+ t = &s->s_newline;
+ else if (isspace(c) && (ns = s->s_space.t_newstate) != NULL)
+ t = &s->s_space;
+ else if ((ns = s->s_other.t_newstate) != NULL)
+ t = &s->s_other;
+ else
+ ns = NULL;
+
+ assert(ns != NULL);
+ if (c == -1) {
+ dbg_printf("%s: transition (%s, eof) -> %s (op %s)\n", __func__,
+ s->s_descr, ns->s_descr, eol_op_string(t->t_op));
+ } else {
+ dbg_printf("%s: transition (%s, %c) -> %s (op %s)\n", __func__,
+ s->s_descr, c, ns->s_descr, eol_op_string(t->t_op));
+ }
+
+ switch (t->t_op) {
+ case EOL_OP_START:
+ p->p_column.start = column;
+ p->p_idx.start = idx;
+ /*FALLTHROUGH*/
+ case EOL_OP_PUSH:
+ p->p_column.stop = column;
+ p->p_idx.stop = idx;
+ break;
+ case EOL_OP_EMIT_ONE:
+ p->p_column.start = column;
+ p->p_idx.start = idx;
+ /*FALLTHROUGH*/
+ case EOL_OP_EMIT:
+ p->p_column.stop = column;
+ p->p_idx.stop = idx;
+ (*p->p_emitter)(&p->p_detection, p->p_arg);
+ break;
+ case EOL_OP_NONE:
+ break;
+ }
+
+ p->p_state = ns;
+}
Index: othersrc/external/bsd/arfe/dt/eol.h
diff -u /dev/null othersrc/external/bsd/arfe/dt/eol.h:1.1
--- /dev/null Tue Jan 26 02:22:57 2016
+++ othersrc/external/bsd/arfe/dt/eol.h Tue Jan 26 02:22:57 2016
@@ -0,0 +1,49 @@
+/* $NetBSD: eol.h,v 1.1 2016/01/26 02:22:57 dyoung Exp $ */
+/* $ARFE: eol.h 319 2016-01-13 22:35:13Z dyoung $ */
+
+/*-
+ * Copyright (c) 2014,2015 David Young <[email protected]>
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
+ * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
+ * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
+ * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
+ * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+#ifndef _EOL_H
+#define _EOL_H
+
+#include <stdbool.h>
+
+typedef struct eol_detection {
+ struct {
+ int start, stop;
+ } d_column, d_idx;
+ /* XXX shouldn't limit this like this */
+ char d_chars[64];
+} eol_detection_t;
+
+struct eol_parser;
+typedef struct eol_parser eol_parser_t;
+
+typedef void (*eol_emitter_t)(eol_detection_t *, void *);
+eol_parser_t *eol_parser_alloc(eol_emitter_t, void *);
+void eol_parser_drive(eol_parser_t *, int, int, int);
+void eol_parser_free(eol_parser_t *);
+
+#endif /* _EOL_H */
Index: othersrc/external/bsd/arfe/dt/space.c
diff -u /dev/null othersrc/external/bsd/arfe/dt/space.c:1.1
--- /dev/null Tue Jan 26 02:22:57 2016
+++ othersrc/external/bsd/arfe/dt/space.c Tue Jan 26 02:22:57 2016
@@ -0,0 +1,180 @@
+/* $NetBSD: space.c,v 1.1 2016/01/26 02:22:57 dyoung Exp $ */
+/* $ARFE: space.c 319 2016-01-13 22:35:13Z dyoung $ */
+
+/*-
+ * Copyright (c) 2014,2015 David Young <[email protected]>
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
+ * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
+ * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
+ * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
+ * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+#include <assert.h>
+#include <ctype.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "core.h"
+#include "space.h"
+
+typedef enum space_op {
+ SPACE_OP_NONE = 0
+ , SPACE_OP_PUSH
+ , SPACE_OP_EMIT
+ , SPACE_OP_START
+} space_op_t;
+
+struct space_state;
+typedef struct space_state space_state_t;
+
+typedef struct space_transition {
+ const space_state_t *t_newstate;
+ space_op_t t_op;
+} space_transition_t;
+
+struct space_state {
+ const char *s_descr;
+ space_transition_t s_other;
+ space_transition_t s_space;
+ space_transition_t s_eof;
+};
+
+enum space_state_idxs {
+ READ_SPACE
+ , READ_EOF
+ , READ_OTHER
+};
+
+#define read_space space_states[READ_SPACE]
+#define read_eof space_states[READ_EOF]
+#define read_other space_states[READ_OTHER]
+
+static const space_state_t space_states[] = {[READ_SPACE] = {
+ .s_descr = "read start",
+ .s_other = {&read_other, SPACE_OP_EMIT},
+ .s_space = {&read_space, SPACE_OP_PUSH},
+ .s_eof = {&read_eof, SPACE_OP_EMIT},
+}, [READ_EOF] = {
+ .s_descr = "read eof",
+ .s_other = {&read_eof, SPACE_OP_NONE},
+}, [READ_OTHER] = {
+ .s_descr = "read other",
+ .s_other = {&read_other, SPACE_OP_NONE},
+ .s_space = {&read_space, SPACE_OP_START},
+}};
+
+struct space_parser {
+ space_detection_t p_detection;
+#define p_column p_detection.d_column
+#define p_idx p_detection.d_idx
+ const space_state_t *p_state;
+ space_emitter_t p_emitter;
+ void *p_arg;
+};
+
+void
+space_parser_free(space_parser_t *p)
+{
+ free(p);
+}
+
+static void
+space_parser_init(space_parser_t *p, space_emitter_t emitter, void *arg)
+{
+ memset(p, 0, sizeof(*p));
+ p->p_state = &read_other;
+ p->p_emitter = emitter;
+ p->p_arg = arg;
+}
+
+space_parser_t *
+space_parser_alloc(space_emitter_t emitter, void *arg)
+{
+ space_parser_t *p;
+
+ p = malloc(sizeof(*p));
+ if (p == NULL)
+ return NULL;
+
+ space_parser_init(p, emitter, arg);
+ return p;
+}
+
+static const char *
+space_op_string(space_op_t op)
+{
+ switch (op) {
+ case SPACE_OP_NONE:
+ return "none";
+ case SPACE_OP_EMIT:
+ return "emit";
+ case SPACE_OP_PUSH:
+ return "push";
+ case SPACE_OP_START:
+ return "start";
+ default:
+ return "unknown";
+ }
+}
+
+void
+space_parser_drive(space_parser_t *p, int idx, int column, int c)
+{
+ const space_transition_t *t;
+ const space_state_t *ns, *s;
+
+ s = p->p_state;
+
+ if (c == -1 && (ns = s->s_eof.t_newstate) != NULL)
+ t = &s->s_eof;
+ else if (c != '\n' && isspace(c) &&
+ (ns = s->s_space.t_newstate) != NULL)
+ t = &s->s_space;
+ else if ((ns = s->s_other.t_newstate) != NULL)
+ t = &s->s_other;
+ else
+ ns = NULL;
+
+ assert(ns != NULL);
+ if (c == -1) {
+ dbg_printf("%s: transition (%s, eof) -> %s (op %s)\n", __func__,
+ s->s_descr, ns->s_descr, space_op_string(t->t_op));
+ } else {
+ dbg_printf("%s: transition (%s, %c) -> %s (op %s)\n", __func__,
+ s->s_descr, c, ns->s_descr, space_op_string(t->t_op));
+ }
+
+ switch (t->t_op) {
+ case SPACE_OP_START:
+ p->p_column.start = column;
+ p->p_idx.start = idx;
+ /*FALLTHROUGH*/
+ case SPACE_OP_PUSH:
+ p->p_column.stop = column;
+ p->p_idx.stop = idx;
+ break;
+ case SPACE_OP_EMIT:
+ (*p->p_emitter)(&p->p_detection, p->p_arg);
+ break;
+ case SPACE_OP_NONE:
+ break;
+ }
+
+ p->p_state = ns;
+}
Index: othersrc/external/bsd/arfe/dt/space.h
diff -u /dev/null othersrc/external/bsd/arfe/dt/space.h:1.1
--- /dev/null Tue Jan 26 02:22:57 2016
+++ othersrc/external/bsd/arfe/dt/space.h Tue Jan 26 02:22:57 2016
@@ -0,0 +1,47 @@
+/* $NetBSD: space.h,v 1.1 2016/01/26 02:22:57 dyoung Exp $ */
+/* $ARFE: space.h 319 2016-01-13 22:35:13Z dyoung $ */
+
+/*-
+ * Copyright (c) 2014,2015 David Young <[email protected]>
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
+ * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
+ * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
+ * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
+ * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+#ifndef _SPACE_H
+#define _SPACE_H
+
+#include <stdbool.h>
+
+typedef struct space_detection {
+ struct {
+ int start, stop;
+ } d_column, d_idx;
+} space_detection_t;
+
+struct space_parser;
+typedef struct space_parser space_parser_t;
+
+typedef void (*space_emitter_t)(space_detection_t *, void *);
+space_parser_t *space_parser_alloc(space_emitter_t, void *);
+void space_parser_drive(space_parser_t *, int, int, int);
+void space_parser_free(space_parser_t *);
+
+#endif /* _SPACE_H */
Index: othersrc/external/bsd/arfe/hlog/.cvsignore
diff -u /dev/null othersrc/external/bsd/arfe/hlog/.cvsignore:1.1
--- /dev/null Tue Jan 26 02:22:57 2016
+++ othersrc/external/bsd/arfe/hlog/.cvsignore Tue Jan 26 02:22:57 2016
@@ -0,0 +1,5 @@
+*.a
+*.d
+*.o
+.depend
+.gdbinit
Index: othersrc/external/bsd/arfe/hlog/Makefile
diff -u /dev/null othersrc/external/bsd/arfe/hlog/Makefile:1.1
--- /dev/null Tue Jan 26 02:22:57 2016
+++ othersrc/external/bsd/arfe/hlog/Makefile Tue Jan 26 02:22:57 2016
@@ -0,0 +1,20 @@
+# $ARFE: Makefile 319 2016-01-13 22:35:13Z dyoung $
+
+NOMAN=
+.include <bsd.own.mk>
+
+CPPFLAGS+=-I${.CURDIR}/..
+
+LIBISPRIVATE=yes
+DBG+=-g -O3
+WARNS=5
+
+LIB= hlog
+INCS= hlog.h
+SRCS= hlog.c
+
+test: dependall
+ @echo "No tests to run."
+
+#.include "../mk/libs.mk"
+.include <bsd.lib.mk>
Index: othersrc/external/bsd/arfe/hlog/hlog.c
diff -u /dev/null othersrc/external/bsd/arfe/hlog/hlog.c:1.1
--- /dev/null Tue Jan 26 02:22:57 2016
+++ othersrc/external/bsd/arfe/hlog/hlog.c Tue Jan 26 02:22:57 2016
@@ -0,0 +1,420 @@
+/* $ARFE: hlog.c 319 2016-01-13 22:35:13Z dyoung $ */
+/*
+ * Copyright (c) 2004, 2005, 2006, 2007 David Young. All rights reserved.
+ *
+ * This file contains code contributed by David Young.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials provided
+ * with the distribution.
+ * 3. David Young's name may not be used to endorse or promote
+ * products derived from this software without specific prior
+ * written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY DAVID YOUNG ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+ * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL DAVID
+ * YOUNG BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
+ * TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+ * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY
+ * OF SUCH DAMAGE.
+ */
+/*
+ * Copyright (c) 2004 Urbana-Champaign Independent Media Center.
+ * All rights reserved.
+ *
+ * This code was written by the Champaign-Urbana Community Wireless
+ * Network Project and its contributors.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials provided
+ * with the distribution.
+ * 3. All advertising materials mentioning features or use of this
+ * software must display the following acknowledgement:
+ * This product includes software developed by the Urbana-Champaign
+ * Independent Media Center.
+ * 4. Urbana-Champaign Independent Media Center's name may not be
+ * used to endorse or promote products derived from this software
+ * without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE URBANA-CHAMPAIGN INDEPENDENT
+ * MEDIA CENTER ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
+ * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ * DISCLAIMED. IN NO EVENT SHALL THE URBANA-CHAMPAIGN INDEPENDENT
+ * MEDIA CENTER BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+ * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+ * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+#include <err.h>
+#include <errno.h>
+#include <stdarg.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <time.h>
+
+#include <sys/param.h>
+#include <sys/cdefs.h>
+
+#include "hlog/hlog.h"
+
+#ifndef lint
+__RCSID("$Id: hlog.c,v 1.1 2016/01/26 02:22:57 dyoung Exp $");
+#endif
+
+static enum hlog_mode hlog_mode = HLOG_M_STDERR;
+TAILQ_HEAD(, hlog_sink) hlog_sinks = TAILQ_HEAD_INITIALIZER(hlog_sinks);
+
+HLOG_SINK_TOP_DEFN(all);
+
+HLOG_SINK_SHORT_DEFN(arithmetic, all);
+HLOG_SINK_SHORT_DEFN(mem, all);
+
+HLOG_SINK_MEDIUM_DEFN(sys, all, LOG_NOTICE);
+HLOG_SINK_MEDIUM_DEFN(notice, all, LOG_NOTICE);
+HLOG_SINK_MEDIUM_DEFN(err, all, LOG_ERR);
+HLOG_SINK_MEDIUM_DEFN(warn, all, LOG_WARNING);
+
+int
+hlog_start(enum hlog_mode mode, int facility)
+{
+ hlog_mode = mode;
+
+ if (mode == HLOG_M_STDERR)
+ return 0;
+ else if (mode != HLOG_M_SYSLOG)
+ return -1;
+
+ openlog(getprogname(), 0, facility);
+ (void)setlogmask(LOG_UPTO(LOG_DEBUG));
+ return 0;
+}
+
+static void
+hlog_print_time(void)
+{
+ time_t t;
+ struct tm *tm;
+ char buf[80];
+
+ t = time(NULL);
+ tm = gmtime(&t);
+ if (strftime(buf, sizeof(buf),"%Y%m%d%H%M%S ", tm) == 0) {
+ warnx("%s: strftime failed", __func__);
+ return;
+ }
+ (void)fprintf(stderr, "%s", buf);
+}
+
+void
+hlog_vlog(int priority, const char *fmt, va_list ap)
+{
+ if (hlog_mode == HLOG_M_SYSLOG) {
+ vsyslog(priority, fmt, ap);
+ return;
+ } else {
+ hlog_print_time();
+ (void)vfprintf(stderr, fmt, ap);
+ (void)fputc('\n', stderr);
+ }
+}
+
+static char *
+message_extend_stderr(const char *fmt0)
+{
+ static const char sep[] = ": ";
+ const char *m;
+ char *fmt;
+ size_t fmtlen;
+
+ m = strerror(errno);
+
+ fmtlen = strlen(fmt0) + strlen(m) + sizeof(sep);
+
+ if ((fmt = malloc(fmtlen)) == NULL) {
+ syslog(LOG_ERR, "%s: malloc failed: %s", __func__,
+ strerror(errno));
+ return NULL;
+ }
+
+ /* Note well the safe strcpy, strcat usage. Thank you. */
+ strcpy(fmt, fmt0);
+ strcat(fmt, sep);
+ strcat(fmt, m);
+
+ return fmt;
+}
+
+static char *
+message_extend_syslogd(const char *fmt0)
+{
+ static const char minterp[] = ": %m";
+ char *fmt;
+ size_t fmtlen;
+
+ fmtlen = sizeof(minterp) + strlen(fmt0);
+
+ if ((fmt = malloc(fmtlen)) == NULL) {
+ syslog(LOG_ERR, "%s: malloc failed: %s", __func__,
+ strerror(errno));
+ return NULL;
+ }
+
+ /* Note well the safe strcpy, strcat usage. Thank you. */
+ strcpy(fmt, fmt0);
+ strcat(fmt, minterp);
+
+ return fmt;
+}
+
+static char *
+message_extend(const char *fmt0)
+{
+ switch (hlog_mode) {
+ case HLOG_M_SYSLOG:
+ return message_extend_syslogd(fmt0);
+ default:
+ return message_extend_stderr(fmt0);
+ }
+}
+
+void
+hlog_verr(int status, const char *fmt0, va_list ap)
+{
+ char *fmt;
+
+ if ((fmt = message_extend(fmt0)) == NULL)
+ exit(status);
+ hlog_vlog(LOG_ERR, fmt, ap);
+ free(fmt);
+
+ exit(status);
+}
+
+void
+hlog_verrx(int status, const char *fmt, va_list ap)
+{
+ hlog_vlog(LOG_ERR, fmt, ap);
+ exit(status);
+}
+
+void
+hlog_vwarn(const char *fmt0, va_list ap)
+{
+ char *fmt;
+
+ if ((fmt = message_extend(fmt0)) == NULL)
+ return;
+ hlog_vlog(LOG_WARNING, fmt, ap);
+ free(fmt);
+}
+
+void
+hlog_vwarnx(const char *fmt, va_list ap)
+{
+ hlog_vlog(LOG_WARNING, fmt, ap);
+}
+
+void
+hlog_err(int status, const char *fmt, ...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ hlog_verr(status, fmt, ap);
+ va_end(ap);
+}
+
+void
+hlog_errx(int status, const char *fmt, ...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ hlog_verrx(status, fmt, ap);
+ va_end(ap);
+}
+
+void
+hlog_warn(const char *fmt, ...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ hlog_vwarn(fmt, ap);
+ va_end(ap);
+}
+
+void
+hlog_warnx(const char *fmt, ...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ hlog_vwarnx(fmt, ap);
+ va_end(ap);
+}
+
+struct hlog_sink *
+hlog_sink_find_active(struct hlog_sink *ls0)
+{
+ struct hlog_sink *ls;
+
+ HLOG_SINK_FOREACH(ls, ls0) {
+ switch (ls->ls_state) {
+ case HLOG_SINK_S_PASS:
+ continue;
+ case HLOG_SINK_S_OFF:
+ return NULL;
+ case HLOG_SINK_S_ON:
+ default:
+ return ls;
+ }
+ }
+ return NULL;
+}
+
+void
+hlog_always_log(struct hlog_sink *ls, const char *fmt, ...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ hlog_vlog(ls->ls_priority, fmt, ap);
+ va_end(ap);
+}
+
+void
+hlog_log(struct hlog_sink *ls0, const char *fmt, ...)
+{
+ struct hlog_sink *ls;
+ va_list ap;
+
+ if ((ls = hlog_sink_find_active(ls0)) == NULL)
+ return;
+
+ va_start(ap, fmt);
+ hlog_vlog(ls->ls_priority, fmt, ap);
+ va_end(ap);
+}
+
+struct hlog_sink *
+hlog_sink_lookup(const char *name)
+{
+ struct hlog_sink *ls;
+ TAILQ_FOREACH(ls, &hlog_sinks, ls_next) {
+ if (strcmp(ls->ls_name, name) == 0)
+ return ls;
+ }
+ return NULL;
+}
+
+static struct hlog_sink *
+hlog_sink_create(const char *name)
+{
+ struct hlog_sink *ls;
+
+ if ((ls = calloc(1, sizeof(*ls))) == NULL)
+ return NULL;
+ else if ((ls->ls_name0 = strdup(name)) == NULL) {
+ free(ls);
+ return NULL;
+ }
+ ls->ls_name = ls->ls_name0;
+ ls->ls_rendezvous = 1;
+ return ls;
+}
+
+static void
+hlog_sink_destroy(struct hlog_sink *ls)
+{
+ /*LINTED*/
+ if (ls->ls_name0 != NULL)
+ free(ls->ls_name0);
+ free(ls);
+}
+
+int
+hlog_set_state(const char *name, enum hlog_sink_state state, int rendezvous)
+{
+ struct hlog_sink *ls;
+ errno = 0;
+
+ switch (state) {
+ case HLOG_SINK_S_PASS:
+ case HLOG_SINK_S_OFF:
+ case HLOG_SINK_S_ON:
+ break;
+ default:
+ errno = EINVAL;
+ return -1;
+ }
+ if ((ls = hlog_sink_lookup(name)) == NULL && !rendezvous) {
+ errno = ESRCH;
+ return -1;
+ } else if (ls == NULL) {
+ if ((ls = hlog_sink_create(name)) == NULL)
+ return -1;
+ TAILQ_INSERT_TAIL(&hlog_sinks, ls, ls_next);
+ }
+ ls->ls_state = state;
+ return 0;
+}
+
+void
+hlog_sink_register(struct hlog_sink *ls_arg)
+{
+ struct hlog_sink *ls;
+ if ((ls = hlog_sink_lookup(ls_arg->ls_name)) == NULL ||
+ ls->ls_rendezvous) {
+ TAILQ_INSERT_TAIL(&hlog_sinks, ls_arg, ls_next);
+ if (ls == NULL)
+ return;
+ warnx("%s: rendezvous with log-sink '%s'", __func__,
+ ls->ls_name);
+ ls_arg->ls_state = ls->ls_state;
+ TAILQ_REMOVE(&hlog_sinks, ls, ls_next);
+ hlog_sink_destroy(ls);
+ } else
+ warnx("%s: duplicate log-sink, '%s'", __func__, ls->ls_name);
+}
+
+/*ARGSUSED*/
+int
+log_printer(void *arg __unused, const char *fmt, ...)
+{
+ va_list ap;
+ struct hlog_sink *ls;
+ if ((ls = hlog_sink_find_active(&log_notice)) == NULL) {
+ errno = EAGAIN;
+ return -1;
+ }
+
+ va_start(ap, fmt);
+ hlog_vlog(ls->ls_priority, fmt, ap);
+ va_end(ap);
+ return 0;
+}
Index: othersrc/external/bsd/arfe/hlog/hlog.h
diff -u /dev/null othersrc/external/bsd/arfe/hlog/hlog.h:1.1
--- /dev/null Tue Jan 26 02:22:57 2016
+++ othersrc/external/bsd/arfe/hlog/hlog.h Tue Jan 26 02:22:57 2016
@@ -0,0 +1,188 @@
+/* $ARFE: hlog.h 319 2016-01-13 22:35:13Z dyoung $ */
+/*
+ * Copyright (c) 2004, 2005, 2006, 2007 David Young. All rights reserved.
+ *
+ * This file contains code contributed by David Young.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials provided
+ * with the distribution.
+ * 3. David Young's name may not be used to endorse or promote
+ * products derived from this software without specific prior
+ * written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY DAVID YOUNG ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+ * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL DAVID
+ * YOUNG BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
+ * TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+ * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY
+ * OF SUCH DAMAGE.
+ */
+/*
+ * Copyright (c) 2004 Urbana-Champaign Independent Media Center.
+ * All rights reserved.
+ *
+ * This code was written by the Champaign-Urbana Community Wireless
+ * Network Project and its contributors.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials provided
+ * with the distribution.
+ * 3. All advertising materials mentioning features or use of this
+ * software must display the following acknowledgement:
+ * This product includes software developed by the Urbana-Champaign
+ * Independent Media Center.
+ * 4. Urbana-Champaign Independent Media Center's name may not be
+ * used to endorse or promote products derived from this software
+ * without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE URBANA-CHAMPAIGN INDEPENDENT
+ * MEDIA CENTER ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
+ * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ * DISCLAIMED. IN NO EVENT SHALL THE URBANA-CHAMPAIGN INDEPENDENT
+ * MEDIA CENTER BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+ * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+ * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+#ifndef _HLOG_H
+#define _HLOG_H
+
+#include <stdarg.h>
+#include <syslog.h>
+#include <sys/cdefs.h>
+#include <sys/queue.h>
+
+enum hlog_mode {HLOG_M_SYSLOG = 0, HLOG_M_STDERR};
+
+enum hlog_sink_state {
+ HLOG_SINK_S_PASS = 0
+ , HLOG_SINK_S_OFF = 1
+ , HLOG_SINK_S_ON = 2
+};
+
+struct hlog_sink {
+ const char *ls_name;
+ char *ls_name0;
+ struct hlog_sink *ls_parent;
+ enum hlog_sink_state ls_state;
+ int ls_priority;
+ int ls_rendezvous;
+ TAILQ_ENTRY(hlog_sink) ls_next;
+};
+
+#define HLOG_CONSTRUCTOR(__sym) \
+void hlog_constructor_##__sym(void) __attribute__((constructor)); \
+void \
+hlog_constructor_##__sym(void) \
+{ \
+ hlog_sink_register(&__sym); \
+} \
+void hlog_constructor_##__sym(void) __attribute__((constructor))
+
+#define HLOG_SINK_FOREACH(__le, __le0) \
+ for ((__le) = (__le0); (__le) != NULL; (__le) = (__le)->ls_parent)
+
+#define HLOG_SINK_DECL1(__sym) extern struct hlog_sink __sym
+
+#define HLOG_SINK_DECL(__name) HLOG_SINK_DECL1(log_##__name)
+
+#define HLOG_SINK_DEFN(__sym, __name, __parent, __state, __priority) \
+ struct hlog_sink __sym = { \
+ .ls_name = __name \
+ , .ls_parent = (__parent) \
+ , .ls_state = (__state) \
+ , .ls_priority = (__priority) \
+ }; \
+ HLOG_CONSTRUCTOR(__sym)
+
+#define HLOG_SINK_MEDIUM_DEFN(__name, __parent, __priority) \
+ HLOG_SINK_DEFN(log_##__name, #__name, &log_##__parent, \
+ HLOG_SINK_S_PASS, (__priority))
+
+#define HLOG_SINK_SHORT_DEFN(__name, __parent) \
+ HLOG_SINK_MEDIUM_DEFN(__name, __parent, LOG_INFO)
+
+#define HLOG_SINK_TOP_DEFN(__name) \
+ HLOG_SINK_DEFN(log_##__name, #__name, NULL, HLOG_SINK_S_ON, \
+ LOG_INFO)
+
+#define HLOG_F_ALL (0xffffffff)
+
+HLOG_SINK_DECL(all);
+HLOG_SINK_DECL(arithmetic);
+HLOG_SINK_DECL(mem);
+HLOG_SINK_DECL(sys);
+HLOG_SINK_DECL(notice);
+HLOG_SINK_DECL(err);
+HLOG_SINK_DECL(warn);
+
+#define HLOG_LOG hlog_log
+
+#define HLOG_LAZY(__ls0, ...) \
+ do { \
+ struct hlog_sink *__ls; \
+ if ((__ls = hlog_sink_find_active((__ls0))) != NULL) \
+ hlog_always_log((__ls), __VA_ARGS__); \
+ } while (/*CONSTCOND*/0)
+
+struct hlog_sink *hlog_sink_find_active(struct hlog_sink *);
+extern void hlog_sink_register(struct hlog_sink *);
+extern struct hlog_sink *hlog_sink_lookup(const char *);
+extern int hlog_set_state(const char *, enum hlog_sink_state, int);
+extern int hlog_start(enum hlog_mode, int);
+
+void hlog_vlog(int, const char *, va_list)
+ __attribute__((__format__(__printf__,2,0)));
+
+void hlog_vwarn(const char *, va_list)
+ __attribute__((__format__(__printf__,1,0)));
+void hlog_vwarnx(const char *, va_list)
+ __attribute__((__format__(__printf__,1,0)));
+void hlog_verr(int, const char *, va_list)
+ __attribute__((__format__(__printf__,2,0))) __attribute__((__noreturn__));
+void hlog_verrx(int, const char *, va_list)
+ __attribute__((__format__(__printf__,2,0))) __attribute__((__noreturn__));
+
+void hlog_warnx(const char *, ...)
+ __attribute__((__format__(__printf__,1,2)));
+void hlog_warn(const char *, ...)
+ __attribute__((__format__(__printf__,1,2)));
+
+void hlog_err(int, const char *, ...)
+ __attribute__((__format__(__printf__,2,3))) __attribute__((__noreturn__));
+void hlog_errx(int, const char *, ...)
+ __attribute__((__format__(__printf__,2,3))) __attribute__((__noreturn__));
+
+void hlog_always_log(struct hlog_sink *, const char *, ...)
+ __attribute__((__format__(__printf__,2,3)));
+
+void hlog_log(struct hlog_sink *, const char *, ...)
+ __attribute__((__format__(__printf__,2,3)));
+
+int log_printer(void *, const char *, ...);
+
+#endif /* _HLOG_H */
Index: othersrc/external/bsd/arfe/mk/helpers.mk
diff -u /dev/null othersrc/external/bsd/arfe/mk/helpers.mk:1.1
--- /dev/null Tue Jan 26 02:22:57 2016
+++ othersrc/external/bsd/arfe/mk/helpers.mk Tue Jan 26 02:22:57 2016
@@ -0,0 +1,11 @@
+.if empty(.MAKEFLAGS:M-V*)
+.if defined(MAKEOBJDIRPREFIX) || defined(MAKEOBJDIR)
+PRINTOBJDIR= ${MAKE} -r -V .OBJDIR -f /dev/null xxx
+.else
+PRINTOBJDIR= ${MAKE} -V .OBJDIR
+.endif
+.else
+PRINTOBJDIR= echo # prevent infinite recursion
+.endif
+
+PRINTOBJDIROF= $(PRINTOBJDIR) -C
Index: othersrc/external/bsd/arfe/tt/tests/6/expected-output
diff -u /dev/null othersrc/external/bsd/arfe/tt/tests/6/expected-output:1.1
--- /dev/null Tue Jan 26 02:22:57 2016
+++ othersrc/external/bsd/arfe/tt/tests/6/expected-output Tue Jan 26 02:22:57 2016
@@ -0,0 +1,2 @@
+ 0 input bytes in 0 packets (0 multicast)
+ 29641834 output bytes in 85674 packets (4 multicast)
Index: othersrc/external/bsd/arfe/tt/tests/6/input
diff -u /dev/null othersrc/external/bsd/arfe/tt/tests/6/input:1.1
--- /dev/null Tue Jan 26 02:22:57 2016
+++ othersrc/external/bsd/arfe/tt/tests/6/input Tue Jan 26 02:22:57 2016
@@ -0,0 +1,2 @@
+ input: 0 packets, 0 bytes
+ output: 85674 packets, 29641834 bytes, 4 multicasts
Index: othersrc/external/bsd/arfe/tt/tests/6/match
diff -u /dev/null othersrc/external/bsd/arfe/tt/tests/6/match:1.1
--- /dev/null Tue Jan 26 02:22:57 2016
+++ othersrc/external/bsd/arfe/tt/tests/6/match Tue Jan 26 02:22:57 2016
@@ -0,0 +1,2 @@
+ input: 745177 packets, 455120099 bytes, 0 multicasts
+ output: 540500 packets, 121290742 bytes, 253 multicasts
Index: othersrc/external/bsd/arfe/tt/tests/6/transform
diff -u /dev/null othersrc/external/bsd/arfe/tt/tests/6/transform:1.1
--- /dev/null Tue Jan 26 02:22:57 2016
+++ othersrc/external/bsd/arfe/tt/tests/6/transform Tue Jan 26 02:22:57 2016
@@ -0,0 +1,2 @@
+ 455120099 input bytes in 745177 packets (0 multicast)
+ 121290742 output bytes in 540500 packets (253 multicast)
