from:"Ian Bruene via devel"

Re: Time to plan for 1.0

2017-08-09 Thread Ian Bruene via devel




On 08/09/2017 01:13 AM, Gary E. Miller via devel wrote:

I find leaving ntpmon running is smoking out a bunch of hard failure.


In hindsight (yeah) we should have thought of this - *I* should have 
thought of this - a long time ago.


--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Time to plan for 1.0

2017-08-08 Thread Ian Bruene via devel


This morning on #ntpsec:

07:45 <@esr> ianbruene: I knew the snmpd project would lead to much yak
 shaving.  This is actually part of the reason I viewed it 
it as a
 good training opportunity.  When you're not under time 
pressure
 and can thus afford to get the bovid tonsuring *right* - 
well,
 that is the Tibetan spelling of "learning opportunity".  
Or should

 be.

I was already on the verge of calling it, given the above I am now 
Officially Calling SNMP support as slipping past 1.0.


The pattern of recent development has been: work on ntpsnmpd for a few 
minutes, go shave a dozen yaks in agentx.py. I see no reason why this 
would change by very much before ntpsnmpd is near complete. And that is 
before counting the Yaks I already know should be shaved without any 
specific trigger from ntpsnmpd.


--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Time to plan for 1.0

2017-08-08 Thread Ian Bruene via devel



Oh, and I can confirm that the code in agentx.py (also ntpsnmpd but I 
haven't merged that) is completely separate from everything else.


agentx.py can be ripped out for 1.0 without problems, it could be 
shipped in a perfectly broken state without problems outside of being 
/incredibly/ bad form Old Chap.


--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Time to plan for 1.0

2017-08-08 Thread Ian Bruene via devel




On 08/08/2017 09:01 AM, Eric S. Raymond wrote:

Strictly speaking you can't do that.  It's the PM's decision whether
we want to drop the feature or change the schedule.  We're pretty informal
here, but you need to know where those lines of authority are, because
many other projects (especially in corporate-land) aren't.


Ah, I misunderstood the structure. Also forgot that there is something 
called a PM and it exists for a reason


I am Officially Suggesting that SNMP support be allowed to slip 1.0 
unless I can make major progress in the next couple weeks.



But, speaking in Mark's absence, I'm OK with slipping it.  It's not
like Classic had this right; their snmpd was broken by design (not
RFC-conformant) and in fact its author urged me to remove it.


NTPc having broken SNMP support was a major factor in my willingness to 
slip for rightness, I wasn't aware that it was so bad the maintainer 
urged you to remove it.



I think our support can wait for 1.1.  Possibly Mark might do an
override on this, but I doubt it.


Is there any idea on how long that would be after 1.0? I'm not saying 
anything about "plenty of time", already eating my "plenty of time till 
1.0" statement.


--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Time to plan for 1.0

2017-08-08 Thread Ian Bruene via devel




On 08/08/2017 11:16 AM, Eric S. Raymond wrote:

Thank you, that is correct procedure.  Acting in Mark's absence I
concur; we'll slip it if you don't have it done.  Mark has the
privilege to override this decision but I will be quite surprised if
he does.


Well now I'm increasing this to Official /Strong/ Suggestion: the 
packet.py testing project just got moved to the top of the list because 
of bug #341. And IIRC the code in packet.py is entangled with the comm 
channels in ways that may require a significant test jig, or refactor. 
Either way AgentX got plonked onto the backburner.



Your priorities should be, in this order, (1) Any NTPsec tracker
issues you can close, (2) Any deadline-sensitive work you are seconded
to on other projects, (3) AgentX. Just tell us when it's done.


Got it (see above).

--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Fwd: Re: Time to plan for 1.0

2017-08-07 Thread Ian Bruene via devel

Note to self: check the reply address in the future.

 Forwarded Message 
Subject:Re: Time to plan for 1.0
Date:   Mon, 7 Aug 2017 14:59:14 -0500
From:   Ian Bruene 
To: Eric S. Raymond 

On 08/07/2017 11:58 AM, Eric S. Raymond via devel wrote:

If anyone thinks my assumptions are incorrect, speak up quickly,
please.  Otherwise let's ID what we need to get done and do it.  I
actually think we ought to be fully able to ship in three weeks
(that is, around 28 August); let's try for that.

Gary, Hal, Matt, Daniel: Would all of you check in on this, please?

Only thing I know of is bug #341, which I have repeatedly forgot or
deliberately ignored because other things were higher priority. *If* the
bug still even exists, as I have heard nothing about it since back when
it was posted.

I just need to rendezvous with Gary to either mark it fixed or track it
and stomp it into the ground.

--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: possible bug: peerstats

2017-08-19 Thread Ian Bruene via devel




On 08/19/2017 08:54 AM, Achim Gratz via devel wrote:

I've updated to ntpsec-0.9.7+1104 ten days ago and just realized that
the peerstats logging has changed format: if I use the new refclock
syntax, then instead of the 127.127.. in the address
field, I now get the driver name like NMEA(0).  I had written my scripts
defensively enough to ignore these lines, so it only now dawned on me
why the associated data went missing.

In principle I'd like a logging format that uses symbolic names for all
peers (that'd solve the problem of peers getting new addresses via DHCP
or IPv6 prefix changes), but please make that configurable and
independent of the way the server / refclock gets specified in the
config.


This is a deliberate incompatibility with NTPclassic. The relevant 
sections from docs/ntpsec.txt:



* Clock identifiers in log files are normally the driver shortname
  followed by the unit number in parentheses, rather than the magic IP
  addresses formerly used.  This change affects the peerstats, rawstats,
  and clockstats files.  Reverted in the --enable-classic-mode build.


* An instance of +ntpq+ built from the NTPsec code
  querying a legacy NTP daemon will not automatically display
  peers with 127.127.127.t.u addresses as refclocks; that assumption
  has been removed from the NTPsec code as part of
  getting it fully IPv6-ready.

--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Fwd: Re: More SNMP dependency questions, also assertions. [FWD because reply error]

2017-05-01 Thread Ian Bruene via devel

 Forwarded Message 
Subject:Re: More SNMP dependency questions, also assertions.
Date:   Mon, 1 May 2017 21:08:17 -0500
From:   Ian Bruene 
To: Matthew Selsky 

On 05/01/2017 08:50 PM, Matthew Selsky via devel wrote:

Hey Ian,
Do you intend to make this an snmp agent that would run under an existing 
net-snmp daemon and communicate via the AgentX (RFC2741) protocol?  Or are you 
thinking of a stand-alone snmp daemon?

I assumed a standalone program, but I had no knowledge of other options.

--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

State of the debugging flags.

2017-05-30 Thread Ian Bruene via devel

At ESRs request I've trawled through the C sources to see how debug 
logging is handled. First, by way of summary let me present you with a 
couple bits of code:


# define DPRINTF(lvl, arg)\
do { \
if (debug >= (lvl))\
mprintf arg;\
} while (0)

#define TRACE(lvl, arg)\
do { \
if (debug >= (lvl))\
mprintf arg;\
} while (0)

#define parseprintf(LEVEL, ARGS) if (debug > LEVEL) printf ARGS

Yeah.

Debug printing is controlled by the debug variable, defined as int debug 
in lib_srbuf.c and extern int debug in declcond.h (both the one in 
include/ and ntpd/) (why isn't this unsigned?). debug==0 is no debug, 
with each level above that including more data as is typical.


The ntpd -d flag increments the debug variable by one for every time it 
appears in the argument list, and also sets nofork.


The ntpd -D n flag sets the debug variable to n, but does not set nofork.

From there the actual logging is handled through several methods: some 
functions with complicated logging requirements directly use debug to 
setup their own printing, similarly some modules with module-wide 
special requirements define their own debug logging functions.


But in most cases debug logging requires only a simple if (debug>0) 
{printf("blah")}, this is handled through four (4!) different methods 
I've seen so far:


DPRINTF, which is defined in ntpd.h
TRACE, which is defined in ntp_debug.h, and is identical to DPRINTF
parseprintf, which is defined in include/parse.h, and is used 
exclusively in libparse/
adhoc if (debug>n) statements. many of the level one statements also 
merely ask if (debug), rather than a proper if (debug > 0)


Every instance I've seen has the logging code enclosed in an #ifdef 
block for the debug compilation switch, whether directly where it is 
used, of inside of a macro. It would appear that someone began to 
replace the explicit if debugs with macros, but never completed it for 
unknown reasons.


--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Proposed argument changes to ntpq (fixing bug #319)

2017-06-07 Thread Ian Bruene via devel



Currently ntpq has -d and -D flags which function much like the ones for 
ntpd. Except that the -d flag also sets ntpq to log to a file instead of 
stderr because, um, reasons?



Proposed interface change:

-d/-D remain, but *only* affect the debug level

Add flag -l / --log-file (alt: -f, etc) turns on logging to a file with 
the default logging filename.


Add flag -L / --log-filename (alt: -F, etc) on logging to a file, with 
the user provided filename.


--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: ntpsec | ntpq: -d -d != -D2 (#319)

2017-06-08 Thread Ian Bruene via devel




On 06/08/2017 05:18 PM, Gary E. Miller via devel wrote:
So, what do you think is still unresolved? 


At this point I think we have solved it, simply by eliminating the 
alternatives. It's all part of the ntpq flag discussion.



--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Proposed argument changes to ntpq (fixing bug #319)

2017-06-08 Thread Ian Bruene via devel




On 06/08/2017 05:29 PM, Gary E. Miller via devel wrote:
NTPsec does not use Python's getopt(). It uses argparse(). 


So the real alternatives here are:

1. Have the dual -l/-L flags

2. Convert ntpq from getopt to argparse

--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

New state of the debug flags

2017-06-06 Thread Ian Bruene via devel



After consolidating the relevant macros into ntp_debug.h and cleaning up 
all of the simple if(debug)s I have done another grep through the 
codebase. It appears that there aren't anymore cases where it is worth 
creating a macro for debugging. What's left is either blocks of custom 
computation, code triggered by different symbols, or code not repeated 
enough to be worth changing.


--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

argparse vs getopt

2017-06-09 Thread Ian Bruene via devel

First: I am not considering performance here *whatsoever*, even if there 
were a meaningful difference, which I doubt, option parsing happens once 
during program startup, and ntpq doesn't need high speed anyway.


Advantages of getopt
getopt is simpler, it only needs argv + some definitions fed into a 
single function and you get parsed options out the other end.
The definitions are themselves simpler; just a string of the short 
arguments, and a list of strings for the long arguments.

Because of that, the "setup" such as it is, is very compact.
getopt results are simpler, just a list of (option, value) pairs 
that are easy to loop through.
Since getopt does not handle usage notes the usage string will be 
defined in one place, and formatted exactly as the author intends.
Generally speaking getopt handles basic option parsing and no more. 
This lack of options means there is no need to tell it which bells, 
whistles, and gongs should or should not be used.


Advantages of argparse
All of the relevant information about an option is defined in the 
same place, this includes both short and long forms, casting, 
repetition, and usage string. This is a clear win for self-documentation 
despite being more verbose.
argparse can automatically create the usage note from the 
information given to it in the definitions.

The previous features result in a Single Point of Truth.
While argparse's setup is far longer it is visually simpler than 
the big bag of bytes that getopt wants.

argparse can cast and sanity check inputs where relevant.
argparse can provide default values where relevant.
argparse returns its results in a form that is more complicated, 
but does not require a loop to find and assign from.
argparse supports options with optional values. The whole reason 
this thing started.



The general tradeoff between getopt vs argparse would appear to be a 
matter of option complexity: programs with few, simple options should 
use getopt. Programs which have many options, or options with complex 
argument or exclusivity requirements should use argparse as they will 
have lower overall complexity, and will also be easier to read.


TL;DR: if the program is xkcd 1168 compliant use argparse.

As it currently stands I believe that ntpq is small enough to be below 
the complexity crossover point. Of the 14 currently existing options 6 
are switches, 2 are knobs that take ints, 2 are for the auth system one 
taking an int the other a filename, 2 are commands, and 2 are debugs. No 
fancy parsing or exclusion is required, and the total number is low 
enough that there is little need to pull hair out when tinkering with 
the system. Under those circumstances I do not believe the transition 
cost is worth it, unless consistency across the python toolchain is a goal.


However.

The reason this subject is being discussed in the first place is because 
a currently existing snafu with the debug options would be best solved 
by adding separate options to control logging to a file. With getopt 
this requires 2 additional options because it doesn't support optional 
arguments. There is also the possibility of adding an option in the 
future to display the raw packets that are sent and received.


Given /these/ circumstances I believe the conversion to argparse is 
justifiable, with ntpq just coming over the threshold of benefiting from 
the SPOT and self-documenting aspects of argparse more than it loses 
from the increased setup costs. I am weighting the self-doc relatively 
high in this case because ntpq is part of NTPsec.


--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: argparse vs getopt

2017-06-14 Thread Ian Bruene via devel




On 06/14/2017 01:39 AM, Matt Selsky via devel wrote:

Only new NTPsec programs use argparse. Everything that's replacing NTP Classic 
C code uses getopt, so that we can support python2.6 (RHEL 6 default python)


Well that settles that. I'll get the getopt version up and running.

--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: argparse vs getopt

2017-06-14 Thread Ian Bruene via devel




On 06/14/2017 02:30 PM, Eric S. Raymond wrote:

Yeah, I thought I recalled that this was an issue.


But wait! There's more!


argparse works just fine on Python 2.6: pip install argparse



So which is it? Is requiring argparse installation acceptable?

The conversion tradeoff was marginal enough that I'm inclined to stay
with getopt at this point even if it is acceptable.

--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: argparse vs getopt

2017-06-14 Thread Ian Bruene via devel



Think Kobayashi Maru, and you are Captain. 



I concur.  Time admins would not thank us for the additional dependency.


Nice timing; in testing the getopt version I think there is an 
assumption that needs to be examined.


Does there need to be a default logfile name in the first place?

The only time it is needed is when the user specifies logging to a file. 
Implementing a default means that ntpq needs an entire option, and for 
what? Debug logging to a file is an inherently unusual operation. Either 
the user wants the file for immediate use (no default needed), or has an 
alias setup for automatic logging, and then any default should be in 
/var/log/ or whatever, not the working directory (and the alias can 
specify that).


Oh and Gary, the solution is to be flying the planet eater so you can 
eat your enemies for fuel. And to be careful to not let anyone ram a 
nuke down your throat.


--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: argparse vs getopt

2017-06-14 Thread Ian Bruene via devel


It. Is. Done.

--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: argparse vs getopt

2017-06-15 Thread Ian Bruene via devel




On 06/15/2017 08:24 AM, Eric S. Raymond wrote:

Good argument.  The only question is whether the semantics you propose
would create a confusing difference from ntpd.


The only difference between ntpq -l and ntpd -l is that ntpd has a 
default in the system logfiles. ntpq can't have optional values for 
horses already beaten to death, and there is no logical default beyond 
the current working directory.


*ding* Oh, I should have named ntpq's long version of -l the same as the 
one in ntpd. brb


--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: More SNMP dependency questions, also assertions.

2017-04-30 Thread Ian Bruene via devel




assert: if both exist, they should be bound at the same time

If yu bind to the IPv6, you get the IPv4 for free.  No need to bind both.


TIL


assert: the port(s) should be choosable

Yes, but default to the snmp port in /etc/services.


Knew it had a specific default, TIL /etc/services


assert: security should be optional

I would want default, but that that can't work without a default passwowrd,
which is worse...


ACK


python-daemon is an implementation of PEP 3143. According to the PEP
and other
info getting a daemon right is finicky, python-daemon exists to
handle that.

Daemonizing is at most a dozen lines of code.  Play with it, but
you will likely decide that it is too trivial to add as a dependency.


Ah, ok. The info I saw made it sound as if daemonization was something 
even masters
found tricky. Conveniently there is a library and PEP on how to do it 
right. :-)


--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: NetBSD 6.1.5 doesn't have ldexpl in math.h

2017-09-15 Thread Ian Bruene via devel




On 09/14/2017 11:46 PM, Eric S. Raymond via devel wrote:

A fair point.  But...on the other hand, a major platform.  Not by our
criterion, which is more or less "Are flocks of these going to be
running in $J_RANDOM_HUMONGOUS_DATACENTER?"  Part of our strategy is
to optimize for the toughest, highest-end users on the newest
hardware.  Classic can keep the hobbyists, we're after the people who
can and sometimes actually do write big donation checks.  We want them
to trust us and love us and give us money and lend us engineers.


As I understood it part of the rationale for NTPsec was the yawning 
security chasms in NTPclassic. Shouldn't wide adoption therefore be 
highly desirable?


Possible answer: Aunt Tillie is not going to hunt down NTPsec and 
install it, we can't get any real foothold in the diffuse installed 
base. But NTPsec *can* get into new OS releases and big server farms.



Your attribution to Bruce suggests that you don't know where he picked up the
term, and you are obviously the kind of person who likes to know such things.
So, "Externality" is a term of art in economics:

https://en.wikipedia.org/wiki/Externality


@Fred Wright

Along this line of thought you should look up Coase's Therom; there is a 
tremendous amount of generative value in understanding it.


--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Testing float-valued functions

2017-10-06 Thread Ian Bruene via devel




On 10/06/2017 10:15 AM, Eric S. Raymond wrote:


This is why the minimal-change alternative is worth considering at this
point in our release cycle.  It means you don't have to do that research
project.


Done, functions that can be simply changed to assertAlmostEqual have 
been, problematic ones have comments documenting the potential problem.



My advice is to keep the out-of-domain tests, but mark them as low-value
and disposable if passing them would incur additional complexity costs.


Done.

--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Testing float-valued functions

2017-10-06 Thread Ian Bruene via devel




On 10/06/2017 06:52 AM, Eric S. Raymond wrote:

In fact, all this test code is subtly wrong, and it is just blind luck
nothing went sproing sooner.  Any of those assertions could have gone
toes-up at any time. The tests in posixize() are wrong, too.

The problem here is that float representation is not exact, and the
"same" float calculation on different platforms may yield slightly
different results.  Thus comparing floats to floats is flaky,
and comparing ints to floats is flaky.  This is true no matter
what language you are using; it's an intrinsic property of the
float data representation.

[...]
Ian, your assignment is to fix this and verify the fix.  Please do
this as immediately as possible.


I changed ntp_to_posix back to what it should be, and fixed what tests I 
can fix immediately.



A question for you to ponder: should you replace all assertEqual calls
or only those on which we have seen failures?  If the latter, what
else do you need to do?

Be prepared to discuss the tradeoffs and justify your answer.


For ntp_to_posix and posixize yes, all of the float tests should be 
changed. Without changing them J. Random System might throw a fit at any 
time. For ntp_to_posix it is more complicated; it takes a float, so 
representation issues will arise. However it returns an integer, so a 
simple assertAlmostEqual() won't cut it. This will probably require some 
custom test code.


Today I will do a run through the tests to find other float-returning 
testees that need adjustment.


A related issue is that I am unsure of the value of the Timeless Void 
tests. Those are testing values outside of what NTP represents, and in 
theory are the same kind of unnecessary test as force feeding a list to 
a function that expects a bool. On the other hand one of those 
"unnecessary" tests was the one that caught the bug.


--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Name of path for installed libraries: ntp or ntpsec

2017-09-26 Thread Ian Bruene via devel




On 09/26/2017 01:21 PM, Hal Murray via devel wrote:

Should we be using ntpsec rather than ntp?


If so we need to know *now*. Because every python file will need its 
imports changed.


--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Python 3 and 1.0

2017-09-26 Thread Ian Bruene via devel




On 09/26/2017 03:19 PM, Eric S. Raymond wrote:

Can you get enough verificaion in a week?  We may have to push back the release
for other reasons.


I can hammer on it, if nothing serious shows up it should be fine.

Post-1.0 I'd like to take a systematic look at how data is flowing 
through the code and how it is being represented by the different 
versions of python. I can't tell for sure yet what is and isn't a kluge. 
I know for example that my fix earlier today was a band-aid due to time 
pressure and not fully understanding the data flow. I do *not* like not 
knowing how many of these there are.


--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Post-1.0 refactoring and unit documentation; a request

2017-09-24 Thread Ian Bruene via devel



A few months ago when I added unit display to the python tools I created 
the devel/units file to document what the assorted Important Variables 
in NTPsec represent, and more importantly how that representation 
changes as the data moves through the programs. Because of my lack of 
knowledge of NTP and inexperience with C, this file is still very 
incomplete. It mostly contains random information that I could glean 
from obvious spots in the codebase.


I am requesting that post-1.0 if you start refactoring or testing the 
codebase and come across an Important Variable, please add it and what 
unit it represents to devel/units. There is no need for complicated 
formatting as of yet, merely appending the information to the file will 
be sufficient, and I will massage it into a coherent picture.


It also occurs to me that there may be other Important Invariants that 
need to be documented, but I do not know what those might be.


--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Python 3 and 1.0

2017-09-26 Thread Ian Bruene via devel




On 09/26/2017 05:28 PM, Fred Wright via devel wrote:

[error snipped]
I have no idea whether this affects anything real or whether it's just a
test artifact.  If the latter, it's clearly not a release blocker.


This error does not effect the program itself. It is part of a test jig 
which is spliced into the code when necessary.



Yeah, I'm familiar with that document. :-) The point is that one should
try to understand what's going on and type things consistently, rather
than just ramdomly converting all over the place.  And since the Python
ntpq is a *new* program, it has less excuse for getting this wrong.


Well there it gets tricky... ntpq is not strictly a new program: it was 
translated from C, and bears the scars of that. As the tests get better 
it will be easier to refactor. Also, these problems are concentrated not 
in ntpq, but packet.py.


--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Python 3 and 1.0

2017-09-26 Thread Ian Bruene via devel




On 09/26/2017 04:20 PM, Fred Wright via devel wrote:

I also notcied that test_agentx.py doesn't work with Python 3, but my
impression is that the agentx stuff is still a WIP, anyway.


This is true.


Indeed.  When I started looking at the ntpq bug, I noticed that there
seemed to be some inconsistencies in whether 'response' was expected to be
str or bytes.  It doesn't matter in Python 2, of course.  But tossing in
enough polystr() and polybytes() calls to make the exceptions go away
isn't necessarily the best approach to making reliable code. :-)


Right. Hence why I need to figure out the best way to make this work 
/cleanly/.


--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Python 3 and 1.0

2017-09-26 Thread Ian Bruene via devel




On 09/26/2017 05:02 PM, Eric S. Raymond via devel wrote:

Huh? If so, why has this not shown up in the results from the FreeBSD buildbot.


Two reasons:

1. python tests still not run by the build script

2. subsequent reports are inconsistent on whether FreeBSD has a problem 
or Fred's system is wonky.


--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: All hands - we need to test Fred's build changes pronto

2017-09-26 Thread Ian Bruene via devel


I pushed a fix for both of these.

On 09/26/2017 11:48 AM, Ian Bruene wrote:
python3 build: fails, can't find ntp module, after installing p3 
version it runs but crashes with a type conversion error (I'll get on 
this right away)


python3.6 build: ditto, but also crashes with a "ModuleNotFoundError: 
No module named 'apt_pkg'" error.


--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Python 3 and 1.0

2017-09-26 Thread Ian Bruene via devel



The python 3 build appears to work. However it has a unicode bug in ntpq 
(but not ntpmon! Yay consistency!), and I can not say that I *trust* any 
of it.


This is partially my fault, as I failed to test the software in Py3 as 
much as I should have. As an excuse I will note that I fixed several py3 
bugs in the last few weeks, and part of the reason for the lack of 
testing has been the higher friction in getting the p3 build to work.


Under these circumstances I strongly suggest that there be a note in the 
README to the effect that python 3 compatibility is not guaranteed.


--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Nailing an elusive unicode bug, and a heads up

2017-10-02 Thread Ian Bruene via devel



Several months ago when I added the unit display feature there was a bug 
that caused ntpq to crash with a unicode error for no apparent reason. 
The crash was never replicated, but I added some debugging statements in 
hope of catching it. Someday.


In the last week during the release delay Gary and I stumbled on a 
little known feature of Python where the encoding of the standard 
streams can be changed by an environment variable. Tests unearthed a 
number of unicode bugs due to this in both ntpq and ntpmon, ntpq was 
fixed to detect via sys.stdout.encoding if it was talking over UTF-8 and 
if not to use a wrapper to force UTF-8 output. ntpmon unfortunately 
could not be fixed in this way; it uses the curses library which 
interfered with this solution. ntpmon now downgrades to a non-unicode 
version if it encounters this problem.


At the end of last week I got another bug report for ntpq from jasonium 
which matched the old never replicated bug. Today I got the full 
traceback from him and discovered that sys.stdout.encoding sometimes 
lies, sometimes claiming that it is UTF-8 when it is not. ntpq now 
always forces UTF-8 regardless of what python claims it is.


Be advised: sys.stdwhatever.encoding can be fake news.

--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Python 3 and 1.0

2017-09-27 Thread Ian Bruene via devel



Since my initial complaint about Py3 compatibility some bugs have been 
fixed, agentx tests work, and I've poked at it with a stick.


Panic-mode rescinded.

--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Attn: Anyone familiar with 32 vs 64 bit time_t issues, or who can make policy decisions.

2017-10-08 Thread Ian Bruene via devel



Please take a look at bug #404 at the earliest opportunity. I hate to 
raise such an alarm over a test bug but we are running out of hours 
before release.


Relevant problem is my final comment on the bug thread.

--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Let's get back to work

2017-10-17 Thread Ian Bruene via devel




On 10/17/2017 05:37 AM, Eric S. Raymond via devel wrote:

Other possibilities welcomed.


Someone who understands waf: make the build run the python tests.

Note to whoever does it: you need to run them directly, python discover 
requires pip.


--
In the end; what separates a Man, from a Slave? Money? Power?
No. A Man Chooses, a Slave Obeys. -- Andrew Ryan

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Fixing the build to run python tests

2017-11-29 Thread Ian Bruene via devel




On 11/29/2017 04:17 PM, Hal Murray via devel wrote:

That link lets you test python code using the new libraries.

In a previous version of that code, the link was in ntpclients and you could
run things by cd-ing there.  Now the link is in $build/main/ntpclients so you
have to cd there.  Are you doing that?


This problem is only with the tests which need to access the python ntp 
modules, not the sutff in ntpclients/. Once the build is complete 
everything works just fine. And this is happening within the build 
script: no cd'ing allowed as far as I know.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Fixing the build to run python tests

2017-11-29 Thread Ian Bruene via devel



First issue (blocker)

I have succeeded in getting the unit testing scripts for python code to 
run as part of the build on my machine, however when uploaded to GitLab 
the pipelines fail with an import error in the test scripts. The test 
modules cannot import the library modules they are meant to test.


The cause of this issue is that the python libraries are stored in 
pylib/ but installed (and referenced in the code) as ntp/. Previously 
this caused import problems that were solved when someone added code to 
create a symlink from ntp/ to pylib/ at the end of the build process. 
When run by the pipeline this symlink is not in place, resulting in the 
error. When I delete the build/ folder from my repo and run the build it 
errors out with a different import error (probably gets farther because 
I have an installed copy it can piggyback off of), but then a subsequent 
clean configure build succeeds.


Adding the pytest_paths argument to the build script does not - and 
cannot - work because of the "pylib" vs "ntp" difference.


I attempted to snip the code from afterparty() that creates the symlinks 
and stick it in build() before the tests are run. It errored in ways I 
do not understand.


Though I understand the build system far better than I did a week ago I 
am still mostly poking it while trying to not to blow it up. Which leads 
nicely into the next problem...


Second issue (non-blocker, but ugly)

In order to run unit tests on python code the "pytest" waf module is 
required. Attempting to load it normally failed, leading me to believe 
that it was not included in the non-human-readable waf code. After 
manually downloading the relevant file from the waf repo I tried to 
place it in the directory where it should go in theory, resulting in 
failed imports for everything else that has that import path (but is 
really imported from the archive). I also tried to place in in 
wafhelpers, which failed. Currently in the MR for the build changes 
pytest.py sits in the top level ntpsec/ directory where it works, but 
this kind of sloppiness is not how the build should be arranged.


I do not know how to "recompile" waf to include different modules, I 
don't know the beginnings of what I don't know, and I'm not sure that I 
would do it if I did know. Someone with Official Build Wrangling status 
needs to take a look at this.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Pulling - or clawing - data out of mode 6

2017-12-01 Thread Ian Bruene via devel




On 12/01/2017 08:44 PM, Hal Murray via devel wrote:

The "uptime" variable is used to by snmp clients to do "count per time"
calculations, and also to notice how long after boot that that that daemon
started, or if its restarting.   If you need to, insert into your subagent
code when it first runs to grab the system clock value and store it, and
then just sent back the diffence between "now" and that value whenever that
variable is queried

ntpd has an uptime timer.  You can get it via ntpq -c sysstats
I don't know if you can get it without all the other stuff.

The counters in sysstats are since reset (second line).  They get reset when
written out if sysstats logging is enabled.  That's every hour.


Oh, I forgot about this email. I've determined how to implement uptime, 
but the only logical way I saw was to break the spec because the spec is 
wrong.


See ntpclients/ntpsnmpd.py line 444 in current HEAD.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: version stuff, autorevision

2017-12-12 Thread Ian Bruene via devel




On 12/12/2017 08:04 PM, Hal Murray via devel wrote:

Something broke in my setup and I can't figure out what's going on.


Er, whoops! I broke that as part of my fix for improperly generated 
files. Should be fixed now.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Pulling - or clawing - data out of mode 6

2017-11-20 Thread Ian Bruene via devel



I'm trying to fill out the MIB for ntpsnmpd and have several items that 
I do not know how to get data for. This also effects ntpq as so far my 
primary method of discovering what to poke mode 6 with to get what I 
need has been to see how ntpq does it. A side effect of ntpsnmpd will 
probably be to expand ntpq's capabilities, or even mode 6's.


Time Resolution (as distinct from time *precision*, which I can get from 
the "precision" variable).


Time distance: From the MIB: "The distance from this NTP entity to the 
root time reference (stratum 0) source". The closest thing I can find 
that sounds like match is the synchd() method in packet.py/SyncPacket. 
It is only used by ntpdig.


Current mode: There is a "peermode" variable, but I don't think it is 
giving the information that the MIB wants. "hostmode" or "hmode" do not 
exist.


Active reference source ID: pretty sure this means which one is the 
syspeer. I can get the address with "peeradr", but not the associd. This 
could be brute forced by getting all of the peers and finding the right 
one


Uptime: well there is an uptime variable. But I do not know the format 
(it *looks* like seconds since start), and the MIB wants a very specific 
and unusual format.


Date of next leap second: from what I can gather from the 2-bit leap 
second code this is predicted a day in advance?


Next leap second direction: so long as the 2-bit code is a usable source 
this is a simple one.


Heartbeat interval.

--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Of MIB trees and universes with a time dimension

2017-11-05 Thread Ian Bruene via devel



In my report during last night's mailing list storm I mentioned needing 
to change the architecture of ntpsnmpd slightly. This email is a 
description of how it works now for 
updation/clarification/documentation/sanity checkification purposes.


In ntpsnmpd there is a class which manages the MIB tree (and currently 
data retrieval as well), the tree itself takes the form of a series of 
nested dicts with each dict-key forming part of an OID name. Previously 
each node consisted of a tuple with the callback for read/write, and the 
nested dict for that branch of the tree. I defined it as a tree because 
it made some things easier to define, and reduces the chance of 
accidentally forgetting an entry or mis-ordering them, during init the 
manager class would then flatten the tree down to a list for ease of 
search. This worked fine so long as the tree never changed, and since I 
didn't understand how sequences are implemented in MIB trees I worked 
with that assumption. That assumption was false.


Now each node consists of a dict with the keys "static", "callback", 
"subids". If "static"==True it behaves as before: "subids" contains the 
child nodes of the tree, and "callback" contains the data read/write 
function. However if "static"==False "subids" contains a function which 
is called by the new walkMIBTree generator to build that part of the 
tree on the fly (which is more elegant even for static trees than the 
code it replaced). From the perspective of the calling code everything 
works as before: call functions to get an OID, and it returns whatever 
one is necessary.


Since most parts of the MIB are static it is still useful to have the 
Dicts of Doom, and then splice in the dynamic entries exactly when needed.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Questions on SNMP TestSet validation

2017-11-09 Thread Ian Bruene via devel



To process the TestSetPDU the sub-agent must validate each varbind 
through a series of steps detailed here 
https://tools.ietf.org/html/rfc1905#section-4.2.5


There are a number of steps that I do not understand, my 
summary/questions about the steps follow:


1 Denied access because not in MIB view (I think the master gets this?) 
[noAccess]


2 Nothing shares the OID prefix (what counts?) and can be 
created/changed regardless of value (does the master get this?) 
[notWritable]


3 Value type is wrong for all variables with same OID prefix (what 
counts? Doesn't matter: the master deals with it) [wrongType]


4 Value length is wrong (share OID prefix, etc. Master has this) 
[wrongLength]


5 Value encoding does not match MIB (huh? Encoding? I don't remember 
this, and it can't be referring to the value type as that is covered 
elsewhere), #NotAll implementations use this [wrongEncoding]


6 Value could never be assigned to the variable (I think this is for 
values outside a specified range?) [wrongValue]


7 Variable does not exist and could never be created [noCreation]

8 Variable does not exist and cannot be created under current 
circumstances [inconsistentName]


9 Variable exists but cannot ever be modified [notWritable]

10 Variable exists but cannot be modified under current circumstances 
[inconsistentValue]


11 During previous checks resources were not available [resourceUnavailable]

12 Some other error happened [genErr]

13 SUCCESS!

Primary Question: What counts as an OID prefix that is shared? The 
answer that seems right without being ridiculous (anything in an OID's 
path) is for things like SEQUENCE blah blah where there is a root path 
with a bunch of dynamic elements. Maybe?


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Getting things moving again

2017-11-04 Thread Ian Bruene via devel




On 11/04/2017 04:54 PM, Eric S. Raymond via devel wrote:

* I would like us to plan on a short-cycle 1.1, to land early January,
   with SNMP support as the big new feature. Ian: is this a realistic
   timeframe?


Current tasks that I see are:

* Filling out the MIB.

* Implementing the rest of the packet types.

* "UI", aka. allowing the daemon to use different communication channels 
with the master agent, expanded logging, etc.


I believe it is realistic.


   Is there any support you're not getting that you'll need
   to get this done.


The first two are dependent on my understanding of how SNMP/AgentX 
works, the potential blocker I previously had *just* clicked yesterday 
(how sequences are implemented). This means I need to rethink the 
architecture slightly, but I'm pretty sure I know how to make that work.


Aside from that the major issues are my inexperience in implementing 
this type of software (something that solves itself as I look for 
information and do tests against the master agent), and the mismatches 
between what the NTPv4-MIB wants and what data I can get from mode 6. 
The size of the latter problem is currently unknown.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: upcoming 1.0.1 release this weekend

2017-12-05 Thread Ian Bruene via devel




On 12/05/2017 02:33 PM, Mark Atwood, Project Manager via devel wrote:
Please make sure that whatever you have pushed to master is ready for 
release, andor let me know if there is any good reason not to do a 
release this weekend.


Only thing I have in master that doesn't match the requirement is 
ntpsnmpd, which will be marked as alpha status. It would be possible to 
comment out its slot in the build script so that it is not installed.


But we are *still* having build issues, with #414 starting to look more 
like a rabbit hole with each passing second.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: upcoming 1.0.1 release this weekend

2017-12-05 Thread Ian Bruene via devel




On 12/05/2017 06:47 PM, Eric S. Raymond via devel wrote:

I'm OK with this plan - codebase seems to be in pretty good shape -
except it might make the cooldown period after Ian lands his
install-path fix a little short. Can we hold off three days or so?


I have something already. WIP MR !600.

Unfortunately it *only* treats the symptoms because I do not know what 
is really going on. And it might break in one of any number of ways at 
the drop of a hat.


Then there is still #417, which I was able to clarify as not being part 
of the install problems. It is caused by the intermittent problems with 
version.py.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Testing

2017-12-07 Thread Ian Bruene via devel




On 12/07/2017 07:43 AM, Eric S. Raymond via devel wrote:

How important is your individual way of doing things?  Would you be willing
to tolerate some inconvenience if that made the rest of us more productive?

In principle, yes.  I'd need to be persuaded that the net was positive -
that the rest of you got sped up more than I got slowed down.  Past
experience makes me skeptical of such claims.


The perspective of a noob might be helpful here: Before I was allowed to 
push to master I routinely had to stop work for hours (a couple times >1 
day) because I needed to make a commit and there was no one around to 
merge it. I am not aware of any change in my bug production rate around 
the increased permissions that isn't directly attributable to generally 
increased experience.


I am unsure of how "the rest of us" will be more productive if even the 
noobs are randomly ground to a halt waiting.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Bite of the Buildbugs!

2017-12-11 Thread Ian Bruene via devel



On 12/10/2017 04:26 PM, Ian Bruene wrote:

On 12/10/2017 10:52 AM, Eric S. Raymond wrote:

Do you understand the problem well enough that you could specify an upstream 
fix?
I'm not yet certain whether python or the distributions have 
jurisdiction here. Earlier comments from rlaager suggest that it is 
the distributions. Working on getting an error specification...


After reviewing the bits of data among the arguments again, yes, python 
paths are controlled by the distribution.


Breakage I know of well enough to specify:

* Fedora does not have /usr/local/ in its path

* Ubuntu 17.04 (mine) does not include /usr/local/lib/python3 is its 
path. Unfortunately python tells it to install there, instead of 
python3.5 / 3.6 / etc


So how does this work? Just hunt down each distribution that has a path 
problem and file a bug report?


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Bite of the Buildbugs!

2017-12-10 Thread Ian Bruene via devel



After reading over the discussion regarding the recent /issues/, I have 
come to a side: Revert Fred's fix and throughly document the import 
breakage.


Reasoning:

The standard method means that on some systems the ntp module can't be 
seen by python without modifying PYTHONPATH.


The fix results in ntp imports always working, but on those systems that 
would have had problems it randomly shunts files where they have no 
business being.


The import error is entirely due to upstream problems.

Any attempt to directly hack the path to be correct will result in 
brittle code that could blow up at any time on who knows what system. 
Best case is it simply doesn't install, worst it trashes something.


Randomly shunting files to places that are non-standard - when the user 
did not request that - is Very Bad. I would consider it bad from a 
random unimportant library or tool, but NTPsec has a higher than average 
need to be well-behaved. The fix is unacceptable. Hacking the paths is 
even worse: the brittleness is everything NTPsec was created to eliminate.


In contrast PYTHONPATH issues are well behaved on our part, does not 
have a chance of screwing  up someone else's system, and if someone 
complains (as they should) we can point them upstream to put added 
backpressure on the real bug.


Since Aunt Tillie is not NTPsec's market adding something to PYTHONPATH 
is not a showstopper, whereas (correct me if I'm wrong) I don't think 
any sysadmin for an Important Server worth their salt will hear "this 
install might decide to put part of its code in a random system 
directory" and do anything other than run rm -rf ntpsec.


A discussion about whether to have an option to auto-hack the PYTHONPATH 
might be fruitful however.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Bite of the Buildbugs!

2017-12-10 Thread Ian Bruene via devel




On 12/10/2017 10:52 AM, Eric S. Raymond wrote:

Ugly, but simple.  I'd like to hear counterargument from Gary and Fred before
we make a final decision. Keep it succint, guys.


Agreed. My main concern is that trying to be clever here has many ways 
to go wrong, and few ways to detect them before it gets to users.



Do you understand the problem well enough that you could specify an upstream 
fix?


I'm not yet certain whether python or the distributions have 
jurisdiction here. Earlier comments from rlaager suggest that it is the 
distributions. Working on getting an error specification...


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Preparing for a point release

2017-12-05 Thread Ian Bruene via devel




On 12/05/2017 08:24 AM, Eric S. Raymond via devel wrote:

* "Install failure on gentoo Stable" (#417) and "Python site packages
installed in wrong location" (#414). Ian has a WIP patch to fix this;
it needs to be finished and landed.


I currently do not have any handle on #417. I know where the issue is 
most likely to be, but do not know how to truly fix the problem short of 
whipping up a completely new build-path-finder (say goodbye to a 
ntpsnmpd in an early January 1.1 if this happens).


#414 is a simple - if hacky - fix. The problem is figuring out how to 
trigger the fix at the right times and no others. The other day rlaager 
made a suggestion:


18:21 < rlaager> ianbruene: You could switch based on $prefix == '/usr'

The problem with this is that there are four possible situations, not 2:

Distro Install: /usr/lib/python/dist-packages

Distro Install, pip: /usr/lib/python/site-packages

User Install, pip: /usr/local/python/dist-packages

User Install: /usr/local/python/site-packages

Distinguishing between Distro/User is easy enough; rlaager's suggestion 
works. But I don't know how to detect the other switch.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Preparing for a point release

2017-12-06 Thread Ian Bruene via devel




On 12/05/2017 11:57 PM, Richard Laager via devel wrote:

 From my reading of that wiki page, the distro-packaged Python uses
dist-packages whenever stock Python would use site-packages. This way,
if you install the distro Python package *and* Python from source, you
can install modules for each and they don't conflict. Modules for the
distro-packaged Python go in dist-packages, and modules for the
source-built Python go in site-packages.

Debian, distro Python, prefix = /usr (e.g. the ntpsec package):
 /usr/lib/python/dist-packages

Debian, source Python, prefix = /usr (e.g. not a great idea):
 /usr/lib/python/site-packages

Debian, distro Python, prefix = /usr/local (e.g. ntpsec from source):
 /usr/local/lib/python/dist-packages

Debian, source Python, prefix = /usr/local (e.g. both from source):
 /usr/local/lib/python/site-packages

Other distros, assuming they don't patch in dist-packages, have only
site-packages.

non-Debian, prefix=/usr (e.g. ntpsec package):
 /usr/lib/python/site-packages

non-Debian, prefix=/usr/local (e.g. ntpsec from source):
 /usr/local/lib/python/site-packages


So if I am understanding this correctly I can wipe the dist/site fix as 
it was doing the Right Thing already, for distribution specific values 
of Right Thing.



 From ianbrunene later on IRC:
   import distutils.sysconfig
   print(distutils.sysconfig.get_python_lib(
   standard_lib=0, prefix='/usr/local'));

Instead of the hard-coded '/usr/local', pass in whatever --prefix was
passed to waf.


Yes, waf already deals with the prefix, with a default of /usr/local. I 
was trying to trace where the breakage came from.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Preparing for a point release

2017-12-06 Thread Ian Bruene via devel




On 12/06/2017 09:11 AM, Richard Laager via devel wrote:

Probably? The behavior was already correct for the distro package. Was
anything else broken?


For installs the only remaining problem is that for unknown reasons it 
sometimes doesn't follow the PREFIX when installing the python libs.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: ntpsnmpd

2017-10-29 Thread Ian Bruene via devel




On 10/29/2017 02:13 PM, Eric S. Raymond wrote:

Eric S. Raymond via devel :

So what's missing?  The Mode 6 transactions to get the read data?

Er, I meant *real* data.


Currently all of mode 6, much of the AgentX protocol.

Reason for merging now is that ntpsnmpd has hit the level of basic 
functionality and Gary has wanted to poke at it.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

ntpsnmpd

2017-10-29 Thread Ian Bruene via devel



I have pushed the prototype SNMP AgentX daemon to the repo. In it's 
current state it implements the basic data retrieval packets (get, 
getnext, getbulk), returning dummy values for most items. At this point 
others banging on it is probably more useful than preventing commit clutter.


I make no guarantees that the viewer will not suffer from ocular 
bleeding when viewing the code. It is still very much a prototype.


Documentation will follow shortly.

--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: warning:Mime-Version: 1.0

2018-06-18 Thread Ian Bruene via devel




On 06/18/2018 03:37 PM, Eric S. Raymond via devel wrote:

Ah, now that's the kind of error pattern I *expect* from Bison parsers.
The underlying problem is that the C in Bison parser skeletons is
really archaic. It dates from times when not even the value of
procedural encapsulation was fully understood - thus all those ugly
globals hanging out in front of God and everybody.

It's not actually in the least difficult to design a skeleton that
gets this right.  I did it once.  The point is that that warning is nothing
we're doing wrong, it's GCC correctly noticing that the skeleton code kinda
sucks, and we probably *would* have to build a custom skeleton to
fix it.  I have seen this movie before.


So why haven't the skeletons been fixed in all this time? The only 
reasons I can see are Write Once Forget Forever, and unwillingness to 
use more modern features that not all projects are able to use (which 
could be an option switch).


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


I work for the Internet Civil Engineering Institute , 
help us save the Internet from Entropy!


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Initial test coverage report

2018-06-19 Thread Ian Bruene via devel



I had a look through libntp/ and tests/libntp/ today.

Of the 35 files that are relevant for testing

* 4 are AFAICT fully tested

* 11 have partial tests

* 20 do not even have a test file

There is undoubtedly some error in these figures as the test directory 
has some files in it that do not match up with anything in libntp/. Also 
at least one test file is misnamed.


I will go through these and check / patch everything to be where it 
should be, then start filling out the existing test files, etc.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


I work for the Internet Civil Engineering Institute , 
help us save the Internet from Entropy!


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Why admin's do not trust daemons to do their own packet filtering (was Re: Resuming the great cleanup)

2018-05-29 Thread Ian Bruene via devel




On 05/29/2018 02:15 PM, Eric S. Raymond via devel wrote:

I'm inclined to think dropping this would be a good thing.  There's a
lot of code complexity behind that, and that bit abour interface
commands being inoperative if you choose the wrong command-line option
raises my shoot-self-in-foot alarm.


I've skimmed through some of the code associated with these features 
during deglobalization. It /needs/ to be cleaned up one way or another. 
Cleaning it with a scythe is all the better.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


I work for the Internet Civil Engineering Institute , 
help us save the Internet from Entropy!


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Resuming the great cleanup

2018-05-27 Thread Ian Bruene via devel




On 05/27/2018 11:31 AM, Eric S. Raymond via devel wrote:

* GOPREP: Clear the path to moving the codebase to Go. We haven't committed
to doing this yet, but the odds on that happening someday look high enough
that I think it is good to already be factoring it into our planning.


Though it is known to some I have not mentioned this publicly until now; 
the De-globalization / Structification project's goal is to make the 
codebase easier to port over to Go if the time comes. The project has 
spawned a check of when variables are getting initialized, which also 
helps with GOPREP.



4. Returning to code simplification, every simplification helps with
GOPREP.  This one more so than most because it will dramatically
reduce the spread of platform-dependent code paths we have to map to
Go if and when it comes time to do that.

[...]


Reasons for GOPREP:

Go means (1) no more buffer-overrun attacks, ever, (2) no more memory-leak
issues, ever, (3) *vast* code simplification (LOC might easily drop 50%), (4)
greatly improved maintainability of what remains.


There is also a benefit in that by cutting out lots of easy-to-screw-up 
C boilerplate code it dramatically lowers the barrier to entry for new 
developers to come in without being in terror of breaking something.



GOPREP:  Aside from the considerable labor of code translation, there is
only one problem blocking a move to Go.  That is how GC stop-the-world
pauses might stall refclock reports.


[...]


= DEPENDENCIES =

The logical order to do these things in is: SINGLESOCK first.  Then
EVENTS, if we choose to do it.  Then NTS.  GOPREP isn't a task to be
scheduled (at least not yet) but a set of issues to keep an eye on.


Fortunately most (all?) of what GOPREP needs before the translation 
proper begins is a win even if GOMOVE never happens.



--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


I work for the Internet Civil Engineering Institute , 
help us save the Internet from Entropy!


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Why admin's do not trust daemons to do their own packet filtering (was Re: Resuming the great cleanup)

2018-05-29 Thread Ian Bruene via devel




On 05/29/2018 03:54 PM, Eric S. Raymond wrote:

Ian Bruene via devel :

I've skimmed through some of the code associated with these features during
deglobalization. It /needs/ to be cleaned up one way or another. Cleaning it
with a scythe is all the better.

Hmmm.  You may have talked yourself into a job, apprentice.  Because you
are absolutely right, and more motivated than I knew.


I may have over-exaggerated slightly. but see below.


I was going to do SINGLESOCKET myself, but now that I think about it
that would be almost the next logical C task for you.  I say "almost"
because it's more challenging than I would have picked in an ideal
world; OTOH you haven't screwed up anything I've thrown at you yet.
(Which fact is pretty impressive, by the way.)

Are you up for trying?  "No" is an acceptable answer - I'm not certain
you're ready for this myself.


Yes, though I can't guarantee I won't be screaming for help at 10% 
completion. Also it won't start till after SELF as this does not need to 
be happening right before a release, and Hathi / SELF-prep must have 
priority.



It would be what Marines call "good training" (evil laughter).


I can see the pile of broken industrial grade snark detectors already.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


I work for the Internet Civil Engineering Institute <https://icei.org/>, 
help us save the Internet from Entropy!


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: SINGLESOCK - How much to strip away?

2018-05-31 Thread Ian Bruene via devel




On 05/30/2018 02:45 PM, Hal Murray via devel wrote:

It can be done in two steps.  First is to dump the work-queue but still make
each packet get a buffer from the free queue and go back there.  The second
is to remove the free queue.


Looking at the code, I believe this is the proper action plan for stage 
0 (and probably what the commits will look like):


1. Stop using work queue. Handlers are called directly by the receivers.

2. Remove work queue checking by mainloop()

3. Remove work queue.

4. Stop using free queue. Receivers will simply use local structs.

5. Remove free queue.

Stage 1 would then involve refactoring the recvbuf struct to no longer 
include the parts used by the queue system.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


I work for the Internet Civil Engineering Institute , 
help us save the Internet from Entropy!


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Preliminary report on variable initialization, and when does code freeze happen?

2018-05-27 Thread Ian Bruene via devel



I have done a scan over the variables that I have already structified in 
recent weeks to see where they get initialized.


Most of the variables I have checked get initialized either at 
definition time, or in some sort of init_foo() function. I found three 
variables that are never initialized:


* sys_vars.sys_rootdist

* mon_data.mru_entries

* mon_data.mru_peakentries

sys_rootdist was only recently added to the code, its entire purpose is 
to funnel data to mode 6. I believe this could be safely initialized to 
0 with issue.


mru_entries is incremented and decremented, but is only ever assigned in 
mon_stop(). As I understand it it is only through dumb luck that this is 
ever correct. I believe it can be initialized to 0 without issue.


mru_peakentries is a counter, much like sys_rootdist I believe it can be 
initialized to 0 without issue.


I can fix all three of these now. However I do not know when the pre-1.2 
code freeze is happening and do not wish to make any further changes to 
the C code until after SELF except as necessary.


There is also the larger question of do we want variables to be 
initialized at definition or within an init_foo()? The code is not 
consistent on this and appears to be in the middle of a transition in 
some direction. I assume it is in the direction of initialization 
functions, but I do not know.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


I work for the Internet Civil Engineering Institute , 
help us save the Internet from Entropy!


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Resuming the great cleanup

2018-05-27 Thread Ian Bruene via devel




On 05/27/2018 01:56 PM, Hal Murray via devel wrote:

How thread friendly is GO?


Very. Easy concurrency in network software is Go's major raison d'être.

--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


I work for the Internet Civil Engineering Institute , 
help us save the Internet from Entropy!


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Attn: Install path debaters

2018-01-04 Thread Ian Bruene via devel




On 01/04/2018 12:54 PM, Gary E. Miller via devel wrote:

[*SNIP* long description of why path embedding is Not Done]
RGDS
GARY


Ah. This was rattling around in the back of my head but I had forgotten 
the details. !615's fix can be removed from consideration.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Attn: Install path debaters

2018-01-04 Thread Ian Bruene via devel



Oy Gary!

I think on a a couple of your responses we may be talking about 
different things. However that is moot at this point, as it is clear 
that we have our last solution standing: rip out the "fix" that started 
this whole debate and revert to the old method.



--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Attn: Install path debaters

2018-01-04 Thread Ian Bruene via devel




On 01/04/2018 01:21 PM, Gary E. Miller via devel wrote:

What are these other issues?

The FHS, Gentoo, and AFAIK all distros, do not include /usr/local/XX
in any enviroment PATHs.


Ubuntu does. Did people just not usually use /usr/local/ much in the 
Eldar Days? That would explain it not being part of FHS but distros 
moving towards including it.



So, when I install NTPsec in /usr/local, I need to be sure I
have added /usr/local/XX to at least:
PATH
MANPATH
PYTHONDIR

OTher things installed in /usr/local may also require adding /usr/local
to:
INFOPATH
 PKG_CONFIG_PATH
LD_LIBRARY_PATH

And there are more not wide used.

'Fixing' just PYTHONPATH, and ignoring the others is touching only
part of the problem.


Ooookaaayyy. That is a much bigger problem. If what you are 
describing is true how is the build working /at all/ on non-Debian systems?



If anything is going into /usr by default, that is new, and very, very
bad.  That conflicts with FHS and the policy of every distro I know of.


Not by default, but if the provided paths don't show up in sys.path it 
does. And this is not a new problem, you came across it some time ago, 
but no fix has been decided on as of yet.



Yes, and also the binaries, man pages, and other things.  This is
by design, dating back to UNIX tradition in the 1970's, still embedded
firmly in the FHS, etc.


Why then did the documentation only talk about adding to PYTHONPATH?


      bad: apparently breaks inter-version seals between different
copies of NTP. But this is true of any distro with /usr/local/
      good: it doesn't bypass python versioning <--- This is a Huge

Uh, no.  Until the user sets his PATH, MANPATH, INFOPATH, PYTHONPATH, etc.
the traditional way does NOT break the NPT in /usr.


Not a particularly relevant detail; if it is used to breaks the seal. 
One might also say that if it isn't built it won't have bugs.



      neutral: I'll bet that this doesn't solve the specific variant
of the problem that I've encountered (a weird variant)

You only have a problem because you have not properly configured your
many PATH variables yet.


IMHO if we end up defaulting to the old method we should suggest the
user create a .pth file instead of PYTHONPATH.

I suggest we give him both options.  .pth file is not an option for
many.


PYTHONPATH is a mess for this kind of thing.

Really???  So PATH, MANPATH, INFOPATH, LD_CONFIG_PATH, etc. are somehow
easy for you, but PYTHONPATH is not easy  This is ONE LINE in ONE
FILE.


Ok, now I have to describe the bug I encountered (yes, already filled a 
report upstream). On my system any version of python 3 gives a 
/usr/local/ install path for version "python3". But it only adds paths 
for "python3." under /usr/local/.


This is what I mean when I say that PYTHONPATH is a mess for this kind 
of thing. You can't tell it to add a path for certain versions of python 
only. You can tell it to use a specific path, or you can tell it to use 
a path with the standard subdirectories which the sys.path builder 
script will then add.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Request for data / ntpsnmpd report

2018-01-09 Thread Ian Bruene via devel




On 01/09/2018 11:42 AM, Eric S. Raymond wrote:

Third way: there's a reference to get_resolution() ntp_systime.c at
line 147 ("After default_get_precision() has set a nonzero sys_fuzz")
that implies two things; (1) get_resolution() used to be called within
ntpd, and (2) its role has been taken over by the system variable
sys_fuzz, which is exposed through Mode 6.  We could return that,
scaling it as the RFC requires.

I think the third way is probably best - returning sys_fuzz.


I'll look there.


* Time Distance

This could refer to one of three concepts: "root distance", "root
dispersion" "synchronization distance".  I am not entirely sure these
are all different things (!) as the documentation around them is
remarkably obscure - I refactored the code without fully understanding it.

I *think* "synchronization distance" is about the hop to the peer that
shipped the most recent packet accepted, while "root distance" and
"root dispersion" are the same statistic cumulated back to the root.
Hal or Daniel might know more than I do here.

If my belief is correct, you can fill this in with the mode 6
"rootdisp" system variable.  But I'd like a sanity check on this. If
you're feeling brave, grep for these terms in the documentation and
develop your own theory.


It isn't rootdisp, that is already covered by ntpEntStatusDispersion. 
Will try to track the other(s).



* Validation for the TestSet sequence. According to the RFC this is one of
the most complicated parts of an agent to get right. Currently it is setup
to be failsafe, which we can get away with for now because the NTP MIB only
has two writable entries, and simple ones at that.

I don't understand this at all.  Where is it in the RFC?


Ah, I left that ambiguous. RFC 2741, which defines the AgentX standard.

Quick explanation: To write a variable the master first sends a TestSet 
packet to queue and validate the changes. That is followed by one or 
more of Commit/Undo/CleanupSet. The RFC defines several steps that the 
TestSet handler needs to check in order to validate what it is trying to do.


(This question was directed at anyone with AgentX experience)


I have no idea how to check for this either.


Ok.

--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Attn: Install path debaters

2018-01-03 Thread Ian Bruene via devel



We are on track to merging the solution in !615, if you have objections 
please state them *soon*, together with a patch that fixes the problem. 
We are rapidly approaching the planned mid-January 1.1 date.


To snip one large set of objections in the bud: Yes, this solution is 
/hideous/. Unfortunately the nature of the bug means that we get to pick 
two of {correct install path | can import without shenanigans in the 
user's environment | pretty}. This particular node in tradeoff space 
also means that the ugliness is directly in our code where it can be 
paired with a big fat explanation comment.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Attn: Install path debaters

2018-01-05 Thread Ian Bruene via devel




On 01/04/2018 10:48 PM, Richard Laager via devel wrote:

Can you submit an actual merge request for review?


Currently waiting for the pipeline to finish on !641.

This changes the build back to how it used to work, it builds and 
installs on my system, it has passed the build phase of the pipeline so far.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Concluding the install path debate

2018-01-05 Thread Ian Bruene via devel



This is a summary of the last few months of intermittent arguing over 
the build system. I am glossing over most details as the specifics are 
not important to the summary, and I'm too lazy to track down every post 
on the matter.


Just before the 1.0 release Fred Wright submitted a patch to solve the 
problem of waf installing python libraries in places that did not show 
up in sys.path. This fix worked by checking sys.path to see if the 
install path that python's distutils package returned was added, and if 
not to install somewhere that was in sys.path at the time. The code had 
a known issue with paths that didn't yet exist but would after the 
install not being in sys.path. In practice the patch resulted in a 
default install (/usr/local/) on some systems installing in /usr/, 
violating the FHS in the process.


This spawned frequent intermittent arguments with the broad sides being 
{better to have the import Just Work}, {violating FHS is unacceptable}. 
Various other solutions were proposed or attempted, each one usually 
breaking something else (often in a brittle way), each time this 
happened it drew another person into the argument, peaking at 5 I 
believe. Today (Jan 05, 2018) I ripped out fix_python_config.py, and 
restored the warnings about PYTHONPATH. Here is my reasoning:


* The fix_python_config.py solution stomps on privileged directories at 
times which cannot be predicted by the code. Even if the FHS did not 
exist this would be very Bad Form Ol' Chap.


* The old way (now the current way) means that on some distributions the 
user has to fiddle with their environment to add a .pth file or 
PYTHONPATH. This is not good either, but it has zero chance of damaging 
anything, anyone installing NTPsec is highly unlikely to have difficulty 
with it, and the cherry on top is that serious issues are upstream's 
problem to fix.


* .pth files are apparently not a solution that works generally, as well 
as requiring modification of /usr/, thus violating FHS.


* Embedding the paths directly into the python programs works great, in 
all situations. Right up until it doesn't. One problem is that the 
Gentoo installation workflow is based around moving things around 
temporary directories several times during the process. My initial 
instinct upon seeing this fix was that it was horrible and brittle, but 
I didn't think of the exact reason why and all the other solutions 
seemed to be other forms of terrible so I went with it. I should have 
trusted my instinct here.


TL;DR: we have gone back to the old way. If someone wants a better 
solution feel free to propose it, *after* showing that it will not break 
FHS, break if the user's assumptions are slightly different from ours, 
or break because a feature is only available on a handful of systems.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Python test coverage readout

2018-01-05 Thread Ian Bruene via devel



How can I get a detailed report on the new python test coverage check? 
And how robust is it: does it only count the percentage of functions 
tested, or can it tell what parts of a function are being exercised?


91% is higher than I expected.

--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Attn: Install path debaters

2018-01-04 Thread Ian Bruene via devel




On 01/04/2018 11:44 AM, Richard Laager via devel wrote:

I'm not convinced it's actually bad form. Can you elaborate on why you
see this as hideous?


My understanding is that embedding paths into code like this is 
something that Shouldn't Be Done unless absolutely necessary. It also 
adds the complication of the build system having to edit the code. I may 
need to recalibrate my understanding.


The need to copy the code across several files because it can't be 
imported doesn't help either, but that is a minor point; there are 
larger duplicates. I am fine with this solution if people can agree on it.



There is a reason that build systems from autoconf to waf support
embedding paths, in the style of @PREFIX@ or @SYSCONFDIR@: it is the
best solution for certain problems. I don't think that embedding the
path to look for *this project's* module is any worse than an
application embedding search paths for its own plugins, images, sounds, etc.


I definitely need to recalibrate.


Python's default sys.path is itself built from the embedded copy of
Python's prefix.


I was aware of this from chasing down the bug on my system. Didn't 
consider it relevant to !615 because the sys.path builder is part of the 
distribution code.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Request for data / ntpsnmpd report

2018-01-09 Thread Ian Bruene via devel




On 01/09/2018 09:58 AM, Jason Azze via devel wrote:

At the risk of sounding like a drop-out from a Scrum Master training
camp, could you explain briefly what the "story" is for this tool?

I use SNMP every day to monitor the health of lots of servers and
services, but, to be honest, I haven't been able to follow what you're
trying to achieve with ntpsnmpd.


Good question. My understanding (what was mentioned back when I was 
pointed at the project months ago) is that it is to provide another 
monitoring system for people using SNMP. Essentially stuffing mode 6 
through another pipe.



If I run it on a time server, will I (in my capacity as a sysadmin
running time servers) be able to aim my monitoring system at it make
queries about the state of ntpd?


Yes.


Just to clarify, I'm not criticizing your effort or purpose. I just
literally don't know what the goal is even though I feel like I've
been following along.


I've been focusing on the implementation, no idea what the 30,000ft view 
looks like. I had never even heard of SNMP before this. That no doubt 
fills you with confidence :D.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: What do I do about this? (Pipeline failed - can't find bison)

2018-01-08 Thread Ian Bruene via devel




On 01/08/2018 05:30 AM, Hal Murray via devel wrote:

This smells like a Gitlab glitch, probably transient.

Is there a simple way to say "Please try again?" (without adding clutter to
git's log)


Click the pipeline icon, there will be options from there.

--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Proposal: HAZARD tag

2018-01-17 Thread Ian Bruene via devel



First proposal: For cases where a piece of code needs to embed brittle 
assumptions, in addition to the comment block explaining said 
assumptions it should also include a HAZARD tag with a one line summary 
(not unlike a git summary line). While this standard will only help to 
catch instances of bitrot that are marked, it will make finding those 
cases far easier.


Example:

# HAZARD: assumes this and that
# more detailed explanation
# follows here
[code doing brittle stuff]

Second proposal: As part of the pre-release checklist someone should 
grep the entire codebase for the HAZARD tag and post the list of 
instances to the devlist. Each one must be either signed off on by a 
core developer, or checked for bitrot. Signing off would be the norm, 
used in cases where it is known that the ground has not changed since 
last release and at least one core developer knows/remembers enough 
about the territory that is doesn't need a manual check.


Disadvantages:
    * extra work for release
    * more "paperwork", even if only in the form of devlist traffic

Advantages:
    * known sites with high bitrot potential are regularly checked
    * exerts pressure to fix those sites
    * NTPsec has robustness requirements that make the tradeoff of 
having another checklist more valuable than it would otherwise be


Potential failure mode: everyone signs off out of habit / not caring 
without ever checking anything.


I judge "not caring" a very low probability with this team. Anyone who 
is onboarded is also likely to be assimilated into / have a preexisting 
sense of duty on such matters.


Habit is a more likely problem, but I believe that the proper solution 
is the focus on the "exerts pressure to fix those sites" part of the 
proposal. This type of bad habit is most likely to form where there are 
many items to check.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Request for data / ntpsnmpd report

2018-01-25 Thread Ian Bruene via devel



I finally tracked down where root_distance is created. ntpd/ntp_proto.c 
line 2130 function root_distance.


However it is a function internal to ntp_proto.c, not mentioned in 
header files. And as far as I can tell the values generated from it are 
only ever used inside the functions in that file, not saved.


ntpsnmpd requires this data. As I am unfamiliar with the C code, and 
this is a particularly delicate portion I am unwilling to poke it 
without confirming with others first. Solutions I can see to get the 
data out of ntp_proto.c are:


* save the data. Requires changes to the peer struct. Could easily 
become stale (I do not know how often the peer filtering happens).


* expose the root_distance function. I am unsure of the total extent of 
the needed changes, but they will be scattered across several files.



--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Request for data / ntpsnmpd report

2018-01-24 Thread Ian Bruene via devel



Accidentally replied to ESR directly instead of the list

Update on previous status.

On 01/09/2018 11:42 AM, Eric S. Raymond wrote:

Heads up, Hal!  I'd like your opinopn on these.

Ian Bruene via devel<devel@ntpsec.org>:

* Time Resolution (not to be confused with Time /Precision/, which is one of
the first entries I implemented)

[...]
I think the third way is probably best - returning sys_fuzz.


I believe I have nailed this one down. The mode 6 variable name is "fuzz".


* Time Distance

ntpEntTimeDistance OBJECT-TYPE
 SYNTAX  DisplayString
 MAX-ACCESS  read-only
 STATUS  current
 DESCRIPTION
 "The distance from this NTP entity to the root time reference
 (stratum 0) source including the unit, e.g., '13.243 ms'."
 ::= { ntpEntInfo  7 }


This could refer to one of three concepts: "root distance", "root
dispersion" "synchronization distance".  I am not entirely sure these
are all different things (!) as the documentation around them is
remarkably obscure - I refactored the code without fully understanding it.

I *think* "synchronization distance" is about the hop to the peer that
shipped the most recent packet accepted, while "root distance" and
"root dispersion" are the same statistic cumulated back to the root.
Hal or Daniel might know more than I do here.

If my belief is correct, you can fill this in with the mode 6
"rootdisp" system variable.  But I'd like a sanity check on this. If
you're feeling brave, grep for these terms in the documentation and
develop your own theory.


Doing some grepping reveals that "root distance" == "synchronization 
distance" (see /docs/select.txt), and I had already established that 
"root dispersion" was something else. I am currently trying to pin down 
what variable to ask mode 6 for , it looks like it might be 
"root_delay", but I am unsure. (yay for consistent naming!)


It could be root_delay + something_else.


ntpEntNotifConfigChanged NOTIFICATION-TYPE
 OBJECTS { ntpEntStatusDateTime, ntpEntNotifMessage }
 STATUS  current
 DESCRIPTION
 "The notification to be sent when the NTP configuration has
  changed, e.g., when the system connected to the Internet and
  was assigned a new IP address by the ISPs DHCP server."
 ::= { ntpEntNotifications 6 }

I have no idea how to check for this either.


Still no clue here. The way the MIB is worded would exclude just 
slurping up the config file and monitoring for changes.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Request for data / ntpsnmpd report

2018-02-11 Thread Ian Bruene via devel




On 02/01/2018 11:22 AM, Mark Atwood, Project Manager wrote:
The SNMP MIB RFCs are notorious for including magic blue sky values 
and measurements that nobody knows how to measure and that are not 
well defined.


For things that don't make enough sense, it's ok to not implement that 
particular snmp variable.


As for the "Check if ntpd configuration changed", that appears to me 
to be a good example of "poorly defined". "When did this host get a 
new IP number" is better retrieved from one of the system MIBs.   It's 
okay to not implement it.


Good to know, I will strike the config-change trap from the 
implementation until a better solution is found.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

ntpEntStatPktModeTable.... what is it? (NTPv4-MIB)

2018-02-13 Thread Ian Bruene via devel



Returning to work on ntpsnmpd after a hiatus I looked through the MIB 
entries and discovered that I had accidentally skipped one of the tables.


The table is described as "The number of packets sent and received by 
packet mode. One entry per packet mode.".


It has 3 fields per mode: ntpEntStatPktMode, ntpEntStatPktSent, 
ntpEntStatPktReceived.


The first of those is listed as MAX-ACCESS == not-accessible. Why? / 
What purpose does it serve to have it there?


From the possible values of ntpEntStatPktMode it would appear that the 
"modes" this table is talking about are not the normal NTP communication 
modes like mode6.


Query: should the "modes" listed under ntpEntStatPktMode be used as the 
field indexes for the table entries?


And I'm not entirely certain what data it wants anyway. I don't think 
there are this many non-error packet counts in the entirety of mode 6, 
let alone ones specific to ntpd and not its peers.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: ntpEntStatPktModeTable.... what is it? (NTPv4-MIB)

2018-02-14 Thread Ian Bruene via devel




On 02/13/2018 04:23 PM, Hal Murray wrote:

devel@ntpsec.org said:

  From the possible values of ntpEntStatPktMode it would appear that the
"modes" this table is talking about are not the normal NTP communication
modes like mode6.

What are the possibilies?


From the MIB:

ntpEntStatPktMode OBJECT-TYPE
    SYNTAX  INTEGER {
    symetricactive(1),
    symetricpassive(2),
    client(3),
    server(4),
    broadcastserver(5),
    broadcastclient(6)
    }
    MAX-ACCESS  not-accessible
    STATUS  current
    DESCRIPTION
    "The NTP packet mode."
    ::= { ntpEntStatPktModeEntry 1 }




--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: prep for 1.0.1

2018-02-20 Thread Ian Bruene via devel




On 02/20/2018 08:41 PM, Mark Atwood via devel wrote:

Hi!

A few months ago, I announced prep for a 1.0.1 release.  Turns out, it never 
actually happened.

So, I'm declaring an intention for the 1.0.1 release the weekend after next, 
about March 3rd.

As you work, consider stability, and avoid introducing instability.   And let 
us know if it shouldn't happen.

Thanks, everyone!


ntpsnmpd is reaching the point of not being completely insane to use it. 
It could use more people whacking it with hammers or reviewing the code.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: prep for 1.0.1

2018-02-20 Thread Ian Bruene via devel




On 02/20/2018 09:19 PM, Eric S. Raymond via devel wrote:

I'll get on the tracker and swat a bunch of small issues I see.

The big deal is whether we have closure on the Python installation mess.


The Python installation works the way it did before that last minute 
'fix' before 1.0. So the "mess" no longer exists.


There are some tracker issues involving the installation paths, but not 
the lunacy of before.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

ntpsnmpd beta

2018-02-16 Thread Ian Bruene via devel



MIB implementation complete except for where explicitly not implemented.

No known bugs extant, mostly due to lack of swarm attack.

Packet handling implemented for all required PDU types. But not all in 
general.


Can has testers?

--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: ntpsnmpd beta

2018-02-16 Thread Ian Bruene via devel



Query: what file can/should I use for config data that ntpsnmpd needs to 
be able to change on the fly?


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: ntpsnmpd beta

2018-02-16 Thread Ian Bruene via devel




On 02/16/2018 01:09 PM, Eric S. Raymond wrote:

Ian Bruene via devel <devel@ntpsec.org>:

Query: what file can/should I use for config data that ntpsnmpd needs to be
able to change on the fly?

Does a human ever set this data?  If not, there's a (weak) convetion of
putting the file in /var/run.


There are two sets of configuration data: the usual server and logging 
settings, and the flags which control which notifications are active.


Server settings do not require changes while the daemon is running.

Notification flags can be set from the SNMP side, and as such do require 
updating from the program itself.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Fwd: Re: ntpsnmpd beta

2018-02-16 Thread Ian Bruene via devel

Address mixup landed this in my inbox instead of the devlist. Forwarding 
so that it becomes part of the discussion.

 Forwarded Message 
Subject:Re: ntpsnmpd beta
Date:   Fri, 16 Feb 2018 09:30:19 -0800
From:   Gary E. Miller <g...@rellim.com>
Organization:   Rellim
To: Ian Bruene <ianbru...@gmail.com>

Yo Ian!

On Fri, 16 Feb 2018 11:03:36 -0600
Ian Bruene via devel <devel@ntpsec.org> wrote:

Query: what file can/should I use for config data that ntpsnmpd needs
to be able to change on the fly?

You think it will be that simple?  :-)

I'd like to see it all in with the ntpd configs, but that would cause
conflict with older ntpd.

These days there is no one config file.  Looking at NTP Classic, you
have /etc/ntp.conf and all files in /etc/ntp.d/

I like that idea, the /etc/ntp.conf comes from the distro and then local
mods that modify it go in /etc/ntp.d/

So maybe just change ntp.conf to ntpsnmp.conf and ntp.d to ntpsnmpd.d

I also see some program starting to put configs in /usr/share, but other
than for samples, that seems wrong to me.

RGDS
GARY
---
Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
g...@rellim.com  Tel:+1 541 382 8588

Veritas liberabit vos. -- Quid est veritas?
"If you can’t measure it, you can’t improve it." - Lord Kelvin

Attached Message Part
Description: PGP signature
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: ntpsnmpd beta

2018-02-16 Thread Ian Bruene via devel




On 02/16/2018 02:17 PM, Hal Murray via devel wrote:
Is there a client for ntpsnmpd? I'd expect the client side to be part 
of a snmp package. Do we need samples for that? How many snmp packages 
are there? 


ntpsnmpd only talks to the snmpd daemon. Various sorts of clients, 
including the tools that come with SNMP (snmpwalf, snmpget, etc) then 
talk to snmpd and it relays requests as needed.


AFAIK any linux system will be using NET-SNMP, not sure what other 
systems use. NET-SNMP dropped off the face of the internet a few days 
ago, but ICEI is spinning up a rescue effort.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Future projects (post release)

2018-02-21 Thread Ian Bruene via devel



Future project: refactoring ntpd's system variables into a struct.

I've been looking at the code around mode 6 generation and discovered 
that in some areas it's still globals all the way down. Translating 
these globals will make future refactoring/translating easier.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: 1.0.1 and ntpsnmpd

2018-02-25 Thread Ian Bruene via devel




On 02/25/2018 04:39 PM, Hal Murray via devel wrote:

devel@ntpsec.org said:

The only real blocker that I can see at this time is the need for broad
testing. [reiteration of me requesting testers / reviewers goes here.]

Is there a HOWTO that tells me how to set things up?


I'll get to work on that.


Actually, I need something before that.  Why is it interesting?  What will it
tell me?  Why would I want to use it?


It will tell you the same information as mode 6 (more or less), but 
through a different channel. Beyond that for the general question of 
"Why SNMP?" I'm not sure I can tell you much more that a few minutes of 
quality time with a wiki could.


For the specific relevance to NTPsec see this devlist mail: 
https://lists.ntpsec.org/pipermail/devel/2018-January/005760.html



What will it cost me?  (I know next to nothing about SNMP.)


You need to be running an SNMP daemon and an NTP daemon.

--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

1.0.1 and ntpsnmpd

2018-02-25 Thread Ian Bruene via devel



Is ntpsnmpd having it's Official Release with 1.0.1, or are we punting 
to the next release?


Right now I do not believe that it would be crazy, but there are still 
options that need work, and logging is not systematic and consistent 
yet. The missing options I've thought of are all simple to add; they are 
not blockers. I will probably have them ready by 3/3 even without a 
specific attempt to finish them quickly.


The only real blocker that I can see at this time is the need for broad 
testing. [reiteration of me requesting testers / reviewers goes here.]


Of course nothing brings out bugs like a release.

--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: 1.0.1 and ntpsnmpd

2018-02-25 Thread Ian Bruene via devel




On 02/25/2018 04:43 PM, Eric S. Raymond via devel wrote:

Gary E. Miller via devel <devel@ntpsec.org>:

On Sun, 25 Feb 2018 16:02:00 -0600
Ian Bruene via devel <devel@ntpsec.org> wrote:


[...]

OTOH, people will not test it until it is easy to test.  So I'd suggest
putting it in 1.0.1, and mark "experimental".

[...]  By marking it "experimental" we can mitigate the
reputation risk if it does.


This seems the best option.


So...you get to tell us if it's ready, and commit the change to
include in the tarball if it is.  Don't sweat the decision, you
have more important things to spend your think time on.


Well I just pushed a commit that takes care of the most finicky option. 
So I will be marking it for ship once I do some more cleanup and figure 
out how to include it.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

SOS: SNMP/AgentX traps

2017-12-27 Thread Ian Bruene via devel



I have started implementing notification support in ntpsnmpd, however I 
have been unable to get it working so far. I do not believe there is an 
error in the packet encoding, but there must be a problem somewhere in 
my code as the master agent only ever returns responses with error 268 
(processing error).


I have tested my SNMP configuration to see if other traps work; they do. 
And if information on SNMP is lacking in the first place, information on 
sub-agent trap implementation is downright non-existent. I have not been 
able to look at the net-snmp code enough to figure out what it is doing 
yet either.


Representative example of the Notify packets I am sending (class and 
string versions):


NotifyPDU(bigEndian=True, context='public', packetID=10, pduType=12, 
sessionID=15, transactionID=3, varbinds=[Varbind(vtype=6, 
oid=OID((1, 3, 6, 1, 6, 3, 1, 1, 4, 1), False), payload=OID((1, 3, 6, 1, 
2, 1, 197, 0, 8), False)), Varbind(vtype=66, oid=OID((1, 2, 6, 1, 2, 1, 
197, 0, 1, 4, 1), False), payload=10)])


'\x01\x0c\x18\x00\x00\x00\x00\x0f\x00\x00u0\x00\x01\x86\xa0\x00\x00\x00t\x00\x00\x00\x06public\x00\x00\x00\x06\x00\x00\x05\x06\x00\x00\x00\x00\x00\x03\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x04\x00\x00\x00\x01\x04\x02\x00\x00\x00\x00\x00\x01\x00\x00\x00\xc5\x00\x00\x00\x00\x00\x00\x00\x08\x00B\x00\x00\x0b\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x02\x00\x00\x00\x06\x00\x00\x00\x01\x00\x00\x00\x02\x00\x00\x00\x01\x00\x00\x00\xc5\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x04\x00\x00\x00\x01\x00\x00\x00\n' 



The response I get back:

ResponsePDU(bigEndian=True, packetID=10, pduType=18, resError=268, 
resIndex=0, sessionID=15, sysUptime=886854, transactionID=3, 
varbinds=(Varbind(vtype=6, oid=OID((1, 3, 6, 1, 6, 3, 1, 1, 4, 1), 
False), payload=OID((1, 3, 6, 1, 2, 1, 197, 0, 8), True)), 
Varbind(vtype=66, oid=OID((1, 2, 6, 1, 2, 1, 197, 0, 1, 4, 1), False), 
payload=10)))


'\x01\x12\x10\x00\x00\x00\x00\x0f\x00\x00u0\x00\x01\x86\xa0\x00\x00\x00p\x00\r\x88F\x01\x0c\x00\x00\x00\x06\x00\x00\x05\x06\x00\x00\x00\x00\x00\x03\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x04\x00\x00\x00\x01\x04\x02\x01\x00\x00\x00\x00\x01\x00\x00\x00\xc5\x00\x00\x00\x00\x00\x00\x00\x08\x00B\x00\x00\x0b\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x02\x00\x00\x00\x06\x00\x00\x00\x01\x00\x00\x00\x02\x00\x00\x00\x01\x00\x00\x00\xc5\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x04\x00\x00\x00\x01\x00\x00\x00\n'

--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Attn: Install path debaters

2018-01-03 Thread Ian Bruene via devel





Uh, news to me that any solution was agreed to.  Last I heard this
group was in no way on the same page.

Rather than having me misread your code, can you put a plain summary here?


It's rlaager's code, the bash sys.path in each program one.


Not sure how that makes me feel better.  Exactly the opposite.


Agreed.

But since September everyone has been locked in a loop:

10 Someone notices that things are wrong and mentions / patches it

20 Everyone argues over the N ways to solve it for about a week

30 During the argument someone comes up with method N+1

40 An impasse is reached, everyone walks away in frustration for a few 
weeks. Some forget half of the previous conclusions.


50 GOTO 10

The solutions so far have been:

1. Violate FHS. If this is kosher then it is also kosher to mangle the 
user's PYTHONPATH and we should do that instead.


2. Old system of modules randomly inaccessible. (I would be tolerant of 
this)


3. .pth files. I'm not clear about why these are horribly broken, but 
they must be or they would have not been shot down as soon as they were 
mentioned.


4. The nightmare of !615

5. Have I forgotten something?


And yet, other projects do not have this problem???


This is not helpful. Someone needs to give a concrete example of another 
project that achieves this (and where the solution is compatible with 
waf) so we can see what they are doing.



Care to shaare that ccomment so I do not need to dig???


Not yet written. It would summarize the issues we have had and why we 
picked the particular solution.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: 1.0.1 and ntpsnmpd

2018-02-26 Thread Ian Bruene via devel



ntpsnmpd is now fully part of the build. Manpage installs properly. 
make-tarball includes it (mostly because it slurps up everything).


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Initial test coverage report

2018-06-20 Thread Ian Bruene via devel



I said:

On 06/19/2018 07:02 PM, Ian Bruene wrote:


Also at least one test file is misnamed.

I will go through these and check / patch everything to be where it 
should be


I need to sanity check this first: Am I correct in thinking that tests 
should be arranged as 1 file of tests corresponds to 1 file of code, 
rather than the test files corresponding to this or that concept which 
is scattered over various source files?


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


I work for the Internet Civil Engineering Institute , 
help us save the Internet from Entropy!


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Initial test coverage report

2018-06-21 Thread Ian Bruene via devel




On 06/20/2018 01:52 PM, Matthew Selsky wrote:

Is there a way to have Unity spit out information on test coverage percentage?  
If so, tell me how and I'll take a look at wiring it into our gitlab CI.


Unfortunately I know almost nothing about Unity.

--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


I work for the Internet Civil Engineering Institute , 
help us save the Internet from Entropy!


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Preparing for upcoming release

2018-08-13 Thread Ian Bruene via devel

Forwarded because wrong reply target

 Forwarded Message 
Subject:Re: Preparing for upcoming release
Date:   Mon, 13 Aug 2018 14:36:27 -0500
From:   Ian Bruene 
To: Eric S. Raymond 

On 08/13/2018 02:34 PM, Eric S. Raymond via devel wrote:

So, core devs: please scan the current issue list.  If you see
anything you consider a blocker, speak up.

Priority testing target: make sure the new installation target 
(PYTHONARCHDIR instead of PYTHONDIR) works on as many systems as possible.

--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan

/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb

I work for the Internet Civil Engineering Institute , 
help us save the Internet from Entropy!

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: ANNOUNCE: NTPsec version 1.1.2

2018-08-29 Thread Ian Bruene via devel



Fwd due to Reply/Reply all mistake.


On 08/29/2018 01:26 PM, Eric S. Raymond via devel wrote:

Those projects do tend to blur tigether in my mind.
You were planning to do quue removal at one point.  Is that still on
your list?


I've poked at both of those over the last couple months, didn't get much 
more than research before $RANDOMNESS followed by focusing on expanding 
unit tests.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


I work for the Internet Civil Engineering Institute , 
help us save the Internet from Entropy!


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: Trying again: prep for 1.0.1

2018-03-09 Thread Ian Bruene via devel




On 03/09/2018 12:43 PM, Mark Atwood, Project Manager via devel wrote:
Ok, trying again.  We held the 1.0.1 release for a fix for a problem 
that Hal discovered and fixed.  Thank you, Hal!


Since we have a CVE fix in this release, and also a "make it work 
better on AWS AMIs" fix in, I do want to get this release out soonest.


Please chime in, is there any reason to not release?
I'm targeting Tuesday the 13th of March.

..m


There was a showstopping ntpsnmpd bug from Jason Azze on the 6th. I 
pushed a fix that I think deals with the problem but I have not heard 
back yet; pinging him now.


What kind of special labeling does ntpsnmpd require? Is putting 
"experimental" in the documentation sufficient? Does it need to give a 
warning on launch?


In theory the logging system isn't really up to snuff. However this is 
not important enough to block a release, and I'm waiting till after the 
release to do some major refactoring which will change logging 
requirements anyway.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: 1.0.1 and ntpsnmpd

2018-03-15 Thread Ian Bruene via devel




On 03/15/2018 02:35 PM, Jason Azze via devel wrote:
Sanjeev, was this template created in response to your bounty? I 
finally worked through getting ntpsnmpd up and talking to AgentX on my 
test machine, but all of my Cacti graphs from netniV's template come 
up NaN.


Ian, could you recommend an snmpwalk command or something similar that 
will help answer the question: "How do I know I've got ntpsnmpd working?"


Command I've been using for this:

snmpwalk -v 2c -c public localhost | grep NTP

The most likely problem (given that I don't know cacti) is that after he 
got cacti working I discovered that scalar SNMP values need to go in 
.0 rather than just . I fixed that issue, but cacti 
may not automatically check it.


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

1 2 >

1 - 100 of 149 matches

Mail list logo