Re: Unstable tests

[Keith Mendoza]

Not ideal; but I guess it'll have to do for now.

On Sat, Apr 14, 2018, at 3:56 PM, Bart Van Assche wrote:
> On 04/05/18 16:38, Lee wrote:
> > On 4/5/18, Keith Mendoza  wrote:
> >> So again, at what point do we stop adding code to net-snmp because
> >> ISP's are messing around as if they're doing us a favor by letting us
> >> use their services?
> > 
> > Since people don't read the docs, how about adding a test to see if
> > dns is borked; if it is link to a faq entry for possible ways to fix
> > it.
> > 
> > verizon 'opt out of dns assistance' link:
> >
> > https://www.verizon.com/support/residential/internet/home-network/settings/opt-out-of-dns-assist
> 
> How about the patch below? With this patch applied Net-SNMP developers can
> make the tests pass easily by setting the NETSNMP_DNS_WORKAROUND environment
> variable before running the tests and normal operation of snmpd is not
> affected.
> 
> Thanks,
> 
> Bart.
> 
> 
> 
> Subject: [PATCH] snmplib: Avoid that test T070com2sec_simple fails due 
> to DNS filtering
> 
> ---
>  snmplib/system.c | 19 +++
>  1 file changed, 19 insertions(+)
> 
> diff --git a/snmplib/system.c b/snmplib/system.c
> index d7f06f74087f..28b832352247 100644
> --- a/snmplib/system.c
> +++ b/snmplib/system.c
> @@ -762,6 +762,24 @@ netsnmp_validator_context(void)
>  int
>  netsnmp_gethostbyname_v4(const char* name, in_addr_t *addr_out)
>  {
> +if (getenv("NETSNMP_DNS_WORKAROUND")) {
> +/*
> + * A hack that avoids that T070com2sec_simple fails due to the 
> DNS
> + * client filtering out 127.0.0.x addresses and/or redirecting 
> DNS
> + * resolution failures to a web page.
> + */
> +if (strcmp(name, "onea.net-snmp.org") == 0) {
> +*addr_out = htonl(INADDR_LOOPBACK);
> +return 0;
> +} else if (strcmp(name, "twoa.net-snmp.org") == 0) {
> +*addr_out = htonl(INADDR_LOOPBACK + 1);
> +return 0;
> +} else if (strcmp(name, "no.such.address.") == 0) {
> +return -1;
> +}
> +}
> +
> +{
>  #if HAVE_GETADDRINFO
>  struct addrinfo *addrs = NULL;
>  struct addrinfo hint;
> @@ -826,6 +844,7 @@ netsnmp_gethostbyname_v4(const char* name, in_addr_t 
> *addr_out)
>  #else /* HAVE_GETIPNODEBYNAME */
>  return -1;
>  #endif
> +}
>  }
>  
>  int
> 
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> Net-snmp-coders mailing list
> Net-snmp-coders@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/net-snmp-coders


-- 
Thanks,
Keith (pantherse)

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Net-snmp-coders mailing list
Net-snmp-coders@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/net-snmp-coders

Re: Unstable tests

[Bart Van Assche]

On 04/05/18 16:38, Lee wrote:
> On 4/5/18, Keith Mendoza  wrote:
>> So again, at what point do we stop adding code to net-snmp because
>> ISP's are messing around as if they're doing us a favor by letting us
>> use their services?
> 
> Since people don't read the docs, how about adding a test to see if
> dns is borked; if it is link to a faq entry for possible ways to fix
> it.
> 
> verizon 'opt out of dns assistance' link:
>
> https://www.verizon.com/support/residential/internet/home-network/settings/opt-out-of-dns-assist

How about the patch below? With this patch applied Net-SNMP developers can
make the tests pass easily by setting the NETSNMP_DNS_WORKAROUND environment
variable before running the tests and normal operation of snmpd is not
affected.

Thanks,

Bart.



Subject: [PATCH] snmplib: Avoid that test T070com2sec_simple fails due to DNS 
filtering

---
 snmplib/system.c | 19 +++
 1 file changed, 19 insertions(+)

diff --git a/snmplib/system.c b/snmplib/system.c
index d7f06f74087f..28b832352247 100644
--- a/snmplib/system.c
+++ b/snmplib/system.c
@@ -762,6 +762,24 @@ netsnmp_validator_context(void)
 int
 netsnmp_gethostbyname_v4(const char* name, in_addr_t *addr_out)
 {
+if (getenv("NETSNMP_DNS_WORKAROUND")) {
+/*
+ * A hack that avoids that T070com2sec_simple fails due to the DNS
+ * client filtering out 127.0.0.x addresses and/or redirecting DNS
+ * resolution failures to a web page.
+ */
+if (strcmp(name, "onea.net-snmp.org") == 0) {
+*addr_out = htonl(INADDR_LOOPBACK);
+return 0;
+} else if (strcmp(name, "twoa.net-snmp.org") == 0) {
+*addr_out = htonl(INADDR_LOOPBACK + 1);
+return 0;
+} else if (strcmp(name, "no.such.address.") == 0) {
+return -1;
+}
+}
+
+{
 #if HAVE_GETADDRINFO
 struct addrinfo *addrs = NULL;
 struct addrinfo hint;
@@ -826,6 +844,7 @@ netsnmp_gethostbyname_v4(const char* name, in_addr_t 
*addr_out)
 #else /* HAVE_GETIPNODEBYNAME */
 return -1;
 #endif
+}
 }
 
 int


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Net-snmp-coders mailing list
Net-snmp-coders@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/net-snmp-coders

Re: Unstable tests

[Lee]

On 4/5/18, Keith Mendoza  wrote:
> So again, at what point do we stop adding code to net-snmp because
> ISP's are messing around as if they're doing us a favor by letting us
> use their services?

Since people don't read the docs, how about adding a test to see if
dns is borked; if it is link to a faq entry for possible ways to fix
it.

verizon 'opt out of dns assistance' link:
  
https://www.verizon.com/support/residential/internet/home-network/settings/opt-out-of-dns-assist

Regards,
Lee

> On Wed, Apr 4, 2018 at 7:13 PM, Bart Van Assche  wrote:
>> On 04/04/18 09:30, Keith Mendoza wrote:
>>>
>>> I actually found no.such.address in line 79 of T070com2sec_simple.
>>> That host has been hijacked by barefruit.co.uk who "generates highly
>>> targeted traffic for ISPs by replacing DNS and HTTP errors with
>>> relevant advertising"; which is now causing the test case to fail.
>>> Attempting to talk HTTP to the IP gave the the impression that either
>>> my ISP is cahoots with this company; or they're intercepting the IP
>>> back to their advertisement page. On the upside, Google's and
>>> cloudflare's DNS are not resolving; but, that still means point your
>>> network to use those nameservers :(
>>
>>
>> That means that your ISP is doing something dubious. Anyway, we could add
>> a
>> workaround for broken ISP DNS configurations using the same approach as
>> for
>> the domain names onea.net-snmp.org and twoa.net-snmp.org.
>>
>> Bart.

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Net-snmp-coders mailing list
Net-snmp-coders@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/net-snmp-coders

Re: Unstable tests

[Keith Mendoza]

So again, at what point do we stop adding code to net-snmp because
ISP's are messing around as if they're doing us a favor by letting us
use their services?

Thanks,
Keith
Thanks,
Keith


On Wed, Apr 4, 2018 at 7:13 PM, Bart Van Assche  wrote:
> On 04/04/18 09:30, Keith Mendoza wrote:
>>
>> I actually found no.such.address in line 79 of T070com2sec_simple.
>> That host has been hijacked by barefruit.co.uk who "generates highly
>> targeted traffic for ISPs by replacing DNS and HTTP errors with
>> relevant advertising"; which is now causing the test case to fail.
>> Attempting to talk HTTP to the IP gave the the impression that either
>> my ISP is cahoots with this company; or they're intercepting the IP
>> back to their advertisement page. On the upside, Google's and
>> cloudflare's DNS are not resolving; but, that still means point your
>> network to use those nameservers :(
>
>
> That means that your ISP is doing something dubious. Anyway, we could add a
> workaround for broken ISP DNS configurations using the same approach as for
> the domain names onea.net-snmp.org and twoa.net-snmp.org.
>
> Bart.

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Net-snmp-coders mailing list
Net-snmp-coders@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/net-snmp-coders

Re: Unstable tests

[Bart Van Assche]

On 04/04/18 09:30, Keith Mendoza wrote:

I actually found no.such.address in line 79 of T070com2sec_simple.
That host has been hijacked by barefruit.co.uk who "generates highly
targeted traffic for ISPs by replacing DNS and HTTP errors with
relevant advertising"; which is now causing the test case to fail.
Attempting to talk HTTP to the IP gave the the impression that either
my ISP is cahoots with this company; or they're intercepting the IP
back to their advertisement page. On the upside, Google's and
cloudflare's DNS are not resolving; but, that still means point your
network to use those nameservers :(


That means that your ISP is doing something dubious. Anyway, we could 
add a workaround for broken ISP DNS configurations using the same 
approach as for the domain names onea.net-snmp.org and twoa.net-snmp.org.


Bart.

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Net-snmp-coders mailing list
Net-snmp-coders@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/net-snmp-coders

Re: Unstable tests

[Keith Mendoza]

Bart,


On Sun, Apr 1, 2018 at 8:26 AM, Bart Van Assche  wrote:
> On 03/31/18 22:19, Keith Mendoza wrote:
>>
>> I personally feel that whoever is running the automated tests should
>> make the necessary changes to their environment to resolve any
>> hostname that is needed to run the tests.
>
>
> I don't have root access to the BSD and AIX systems that I use for testing
> Net-SNMP. So modifying /etc/resolv.conf or equivalent on these systems is
> not an option.

I'm wiling to take on testing BSD if you can give me the spec for the
you're using. Unfortunately, I don't  have access to an AIX hardware.

>
>> From a software development and testing point-of-view I feel that we
>> will eventually get caught in expanding the number of hostnames that
>> will have to be handled by all the hostname resolution utility
>> functions in snmplib/system.c. I haven't had a chance to do a complete
>> analysis of how many each of the *gethostbyname* function variants are
>> used through the net-snmp code base; so, I'm not going to speak as to
>> whether these changes is limited to netsnmp_gethostbyname_v4(), or
>> should be applied elsewhere.
>
>
> That's not correct. onea.net-snmp.org and twoa.net-snmp.org are the only two
> that need special handling. If more functions than
> netsnmp_gethostbyname_v4() would have to recognize these hostnames, we can
> refactor the host name resolution functions such that only one function
> needs to know about these special hostnames.

I actually found no.such.address in line 79 of T070com2sec_simple.
That host has been hijacked by barefruit.co.uk who "generates highly
targeted traffic for ISPs by replacing DNS and HTTP errors with
relevant advertising"; which is now causing the test case to fail.
Attempting to talk HTTP to the IP gave the the impression that either
my ISP is cahoots with this company; or they're intercepting the IP
back to their advertisement page. On the upside, Google's and
cloudflare's DNS are not resolving; but, that still means point your
network to use those nameservers :(

>
> Bart.
>
>
>

Thanks,
Keith

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Net-snmp-coders mailing list
Net-snmp-coders@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/net-snmp-coders

Re: Unstable tests

[Bart Van Assche]

On 03/31/18 22:19, Keith Mendoza wrote:

I personally feel that whoever is running the automated tests should
make the necessary changes to their environment to resolve any
hostname that is needed to run the tests.


I don't have root access to the BSD and AIX systems that I use for 
testing Net-SNMP. So modifying /etc/resolv.conf or equivalent on these 
systems is not an option.



From a software development and testing point-of-view I feel that we
will eventually get caught in expanding the number of hostnames that
will have to be handled by all the hostname resolution utility
functions in snmplib/system.c. I haven't had a chance to do a complete
analysis of how many each of the *gethostbyname* function variants are
used through the net-snmp code base; so, I'm not going to speak as to
whether these changes is limited to netsnmp_gethostbyname_v4(), or
should be applied elsewhere.


That's not correct. onea.net-snmp.org and twoa.net-snmp.org are the only 
two that need special handling. If more functions than 
netsnmp_gethostbyname_v4() would have to recognize these hostnames, we 
can refactor the host name resolution functions such that only one 
function needs to know about these special hostnames.


Bart.




--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Net-snmp-coders mailing list
Net-snmp-coders@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/net-snmp-coders

Re: Unstable tests

[Keith Mendoza]

I personally feel that whoever is running the automated tests should
make the necessary changes to their environment to resolve any
hostname that is needed to run the tests. In my case,
onea.net-snmp.org and twoa.net-snmp.org resolves to IP's in the
127.0.0.0/24 network; however no.such.address resolves to 92.242.140.2
which is an IP assigned to barefruit.co.uk. We now have a case of some
unscrupulous advertiser that "highly targeted traffic for ISPs by
replacing DNS and HTTP errors with relevant advertising" (that's from
their home page).

>From a software development and testing point-of-view I feel that we
will eventually get caught in expanding the number of hostnames that
will have to be handled by all the hostname resolution utility
functions in snmplib/system.c. I haven't had a chance to do a complete
analysis of how many each of the *gethostbyname* function variants are
used through the net-snmp code base; so, I'm not going to speak as to
whether these changes is limited to netsnmp_gethostbyname_v4(), or
should be applied elsewhere.

I understand full well the desire to make it as easy as possible for
as many users to be able to build, test, and deploy any open-source
software. However, I feel that the typical user's abilities should
also be considered. I feel that assuming that they can make the
necessary changes to their test environment to resolve a given set of
hostnames is a reasonable one.

Thanks,
Keith (pantherse)
Thanks,
Keith


On Sat, Mar 31, 2018 at 8:48 PM, Bart Van Assche  wrote:
> On 03/22/18 12:22, Eric S. Raymond wrote:
>>
>> Bart Van Assche :
>>>
>>> Hello Eric,
>>>
>>> These are the only two tests that sometimes fail on my test setup.
>>> Whether
>>> or not these tests pass depends on your DNS server. If I e.g. add
>>> "nameserver 8.8.8.8" as the first entry in /etc/resolv.conf then these
>>> tests
>>> pass on my setup. I think the reason is that the domain names used by
>>> that
>>> test resolve into 127.0.0.x and because some DNS servers filter these
>>> results.
>>>
>>> Bart.
>>
>>
>> You guys have been at this too long.  You're failing to document your
>> assumptions.
>>
>>> From 6cd949342a65ff2260253bca234bfa08f8e3b5c2 Mon Sep 17 00:00:00 2001
>>
>> From: "Eric S. Raymond" 
>> Date: Thu, 22 Mar 2018 15:20:21 -0400
>> Subject: [PATCH] INSTALL: explain workaround for comsec test failures.
>>
>> ---
>>   INSTALL | 11 +++
>>   1 file changed, 11 insertions(+)
>>
>> diff --git a/INSTALL b/INSTALL
>> index aad9099..9bcd65a 100644
>> --- a/INSTALL
>> +++ b/INSTALL
>> @@ -6,6 +6,7 @@ TABLE OF CONTENTS
>>   * Net-SNMP Specific Information
>> Long (but you should read these) Instructions
>> Installing the Perl/SNMP Module
>> +  Tests
>>   * Compilers and Options
>> Compiling For Multiple Architectures
>> Installation Names
>> @@ -155,6 +156,16 @@ Net-SNMP libraries and demon applications.
>>   make test
>>   make install (as root)
>>   +Tests
>> +=
>> +
>> +The ordinary self-test sequence can be invoked with "make test". There
>> +are more comprehensive options.
>> +
>> +Spurious failures on the "comsec" tests can be due to misconfigured
>> +DNS upstream of you. A workaround is to point your DNS server at a
>> +non-broken one. Adding "nameserver 8.8.8.8" as the first entry in
>> +/etc/resolv.conf will do.
>> Compilers and Options
>>   =
>
>
> How about the patch below? It makes test T070com2sec_simple pass on my
> setup.
>
> Thanks,
>
> Bart.
>
> ---
>  snmplib/system.c | 14 ++
>  1 file changed, 14 insertions(+)
>
> diff --git a/snmplib/system.c b/snmplib/system.c
> index d7f06f74087f..c9dbea344f71 100644
> --- a/snmplib/system.c
> +++ b/snmplib/system.c
> @@ -762,6 +762,19 @@ netsnmp_validator_context(void)
>  int
>  netsnmp_gethostbyname_v4(const char* name, in_addr_t *addr_out)
>  {
> +/*
> + * A hack that avoids that T070com2sec_simple fails due to the DNS
> + * client filtering out 127.0.0.x addresses.
> + */
> +if (strcmp(name, "onea.net-snmp.org") == 0) {
> +*addr_out = htonl(INADDR_LOOPBACK);
> +return 0;
> +} else if (strcmp(name, "twoa.net-snmp.org") == 0) {
> +*addr_out = htonl(INADDR_LOOPBACK + 1);
> +return 0;
> +}
> +
> +{
>  #if HAVE_GETADDRINFO
>  struct addrinfo *addrs = NULL;
>  struct addrinfo hint;
> @@ -826,6 +839,7 @@ netsnmp_gethostbyname_v4(const char* name, in_addr_t
> *addr_out)
>  #else /* HAVE_GETIPNODEBYNAME */
>  return -1;
>  #endif
> +}
>  }
>
>  int
>
>
>
>
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> Net-snmp-coders mailing list
> Net-snmp-coders@lists.sourceforge.net
> 

Re: Unstable tests

[Bart Van Assche]

On 03/22/18 12:22, Eric S. Raymond wrote:

Bart Van Assche :

Hello Eric,

These are the only two tests that sometimes fail on my test setup. Whether
or not these tests pass depends on your DNS server. If I e.g. add
"nameserver 8.8.8.8" as the first entry in /etc/resolv.conf then these tests
pass on my setup. I think the reason is that the domain names used by that
test resolve into 127.0.0.x and because some DNS servers filter these
results.

Bart.


You guys have been at this too long.  You're failing to document your
assumptions.


From 6cd949342a65ff2260253bca234bfa08f8e3b5c2 Mon Sep 17 00:00:00 2001

From: "Eric S. Raymond" 
Date: Thu, 22 Mar 2018 15:20:21 -0400
Subject: [PATCH] INSTALL: explain workaround for comsec test failures.

---
  INSTALL | 11 +++
  1 file changed, 11 insertions(+)

diff --git a/INSTALL b/INSTALL
index aad9099..9bcd65a 100644
--- a/INSTALL
+++ b/INSTALL
@@ -6,6 +6,7 @@ TABLE OF CONTENTS
  * Net-SNMP Specific Information
Long (but you should read these) Instructions
Installing the Perl/SNMP Module
+  Tests
  * Compilers and Options
Compiling For Multiple Architectures
Installation Names
@@ -155,6 +156,16 @@ Net-SNMP libraries and demon applications.
  make test
  make install (as root)
  
+Tests

+=
+
+The ordinary self-test sequence can be invoked with "make test". There
+are more comprehensive options.
+
+Spurious failures on the "comsec" tests can be due to misconfigured
+DNS upstream of you. A workaround is to point your DNS server at a
+non-broken one. Adding "nameserver 8.8.8.8" as the first entry in
+/etc/resolv.conf will do.
  
  Compilers and Options

  =


How about the patch below? It makes test T070com2sec_simple pass on my
setup.

Thanks,

Bart.

---
 snmplib/system.c | 14 ++
 1 file changed, 14 insertions(+)

diff --git a/snmplib/system.c b/snmplib/system.c
index d7f06f74087f..c9dbea344f71 100644
--- a/snmplib/system.c
+++ b/snmplib/system.c
@@ -762,6 +762,19 @@ netsnmp_validator_context(void)
 int
 netsnmp_gethostbyname_v4(const char* name, in_addr_t *addr_out)
 {
+/*
+ * A hack that avoids that T070com2sec_simple fails due to the DNS
+ * client filtering out 127.0.0.x addresses.
+ */
+if (strcmp(name, "onea.net-snmp.org") == 0) {
+*addr_out = htonl(INADDR_LOOPBACK);
+return 0;
+} else if (strcmp(name, "twoa.net-snmp.org") == 0) {
+*addr_out = htonl(INADDR_LOOPBACK + 1);
+return 0;
+}
+
+{
 #if HAVE_GETADDRINFO
 struct addrinfo *addrs = NULL;
 struct addrinfo hint;
@@ -826,6 +839,7 @@ netsnmp_gethostbyname_v4(const char* name, in_addr_t 
*addr_out)

 #else /* HAVE_GETIPNODEBYNAME */
 return -1;
 #endif
+}
 }

 int



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Net-snmp-coders mailing list
Net-snmp-coders@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/net-snmp-coders

Re: Unstable tests

[Ian Bruene]

On 03/25/2018 07:20 PM, Ian Bruene wrote:


The actual tests that are failing for me now: [snip]



I finally tracked down the issue: snmpd crashes before it ever creates 
the pid file:


snmpd: error while loading shared libraries: libnetsnmpagent.so.35: 
cannot open shared object file: No such file or directory



--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Net-snmp-coders mailing list
Net-snmp-coders@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/net-snmp-coders

Re: Unstable tests

[Ian Bruene]

Apologies for only just now getting into this, excuse of finishing other 
projects.


On 03/21/2018 10:41 PM, Eric S. Raymond wrote:

Ian and I note that the test suite seems a bit compromised.

On current Ubuntu, Intel x86_64, stock build, two tests - 1 and 31
fail out of the box.

We see those same failures on ARM (a RasPi 3).

But Ian's laptop gives a diffeent and larger list of errors.

Are there known pastform dependencies in the tests?  Is it known that
1 and 31 can fail on a stock build?


Closer examination of the outputs reveals that my pi gives perl errors + 
some authentication related stuff. My laptop just gives the perl errors.


The first tier of those perl errors was simply the lack of the SNMP 
module which obviously cleared up with a make install on both laptop and 
Pi, however there is another layer under that which I'm working on now. 
Don't know yet if they are on my end or bugs yet though.


The actual tests that are failing for me now:

Basic perl functionality to a UDP agent 
. 1/10 Can't kill a non-numeric process 
ID at 
/home/ian/Documents/Programming/net-snmp-code/testing/fulltests/support/NetSNMPTest.pm 
line 86.
Basic perl functionality to a UDP agent 
. Dubious, test returned 2 (wstat 512, 
0x200)

Failed 9/10 subtests
Perl UDP IPv4 Test 
.. Can't kill a 
non-numeric process ID at 
/home/ian/Documents/Programming/net-snmp-code/testing/fulltests/support/NetSNMPTest.pm 
line 86.
Perl UDP IPv4 Test 
.. Dubious, test 
returned 2 (wstat 512, 0x200)

Failed 2/2 subtests
Perl TCP IPv4 Test 
.. Can't kill a 
non-numeric process ID at 
/home/ian/Documents/Programming/net-snmp-code/testing/fulltests/support/NetSNMPTest.pm 
line 86.
Perl TCP IPv4 Test 
.. Dubious, test 
returned 2 (wstat 512, 0x200)

Failed 2/2 subtests
Perl UDP IPv6 Test 
.. Can't kill a 
non-numeric process ID at 
/home/ian/Documents/Programming/net-snmp-code/testing/fulltests/support/NetSNMPTest.pm 
line 86.
Perl UDP IPv6 Test 
.. Dubious, test 
returned 2 (wstat 512, 0x200)

Failed 2/2 subtests
Perl TCP IPv6 Test 
.. Can't kill a 
non-numeric process ID at 
/home/ian/Documents/Programming/net-snmp-code/testing/fulltests/support/NetSNMPTest.pm 
line 86.
Perl TCP IPv6 Test 
.. Dubious, test 
returned 2 (wstat 512, 0x200)

Failed 2/2 subtests
Perl Unix Domain Socket Test 
 Can't kill a non-numeric 
process ID at 
/home/ian/Documents/Programming/net-snmp-code/testing/fulltests/support/NetSNMPTest.pm 
line 86.
Perl Unix Domain Socket Test 
 Dubious, test returned 2 
(wstat 512, 0x200)

Failed 2/2 subtests


--
/"In the end; what separates a Man, from a Slave? Money? Power? No. A 
Man Chooses, a Slave Obeys."/ -- Andrew Ryan


/"Utopia cannot precede the Utopian. It will exist the moment we are fit 
to occupy it."/ -- Sophia Lamb


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Net-snmp-coders mailing list
Net-snmp-coders@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/net-snmp-coders

Re: Unstable tests

[Bart Van Assche]

On 03/22/18 12:22, Eric S. Raymond wrote:

Bart Van Assche :

Hello Eric,

These are the only two tests that sometimes fail on my test setup. Whether
or not these tests pass depends on your DNS server. If I e.g. add
"nameserver 8.8.8.8" as the first entry in /etc/resolv.conf then these tests
pass on my setup. I think the reason is that the domain names used by that
test resolve into 127.0.0.x and because some DNS servers filter these
results.

Bart.


You guys have been at this too long.  You're failing to document your
assumptions.


From 6cd949342a65ff2260253bca234bfa08f8e3b5c2 Mon Sep 17 00:00:00 2001

From: "Eric S. Raymond" 
Date: Thu, 22 Mar 2018 15:20:21 -0400
Subject: [PATCH] INSTALL: explain workaround for comsec test failures.

---
  INSTALL | 11 +++
  1 file changed, 11 insertions(+)

diff --git a/INSTALL b/INSTALL
index aad9099..9bcd65a 100644
--- a/INSTALL
+++ b/INSTALL
@@ -6,6 +6,7 @@ TABLE OF CONTENTS
  * Net-SNMP Specific Information
Long (but you should read these) Instructions
Installing the Perl/SNMP Module
+  Tests
  * Compilers and Options
Compiling For Multiple Architectures
Installation Names
@@ -155,6 +156,16 @@ Net-SNMP libraries and demon applications.
  make test
  make install (as root)
  
+Tests

+=
+
+The ordinary self-test sequence can be invoked with "make test". There
+are more comprehensive options.
+
+Spurious failures on the "comsec" tests can be due to misconfigured
+DNS upstream of you. A workaround is to point your DNS server at a
+non-broken one. Adding "nameserver 8.8.8.8" as the first entry in
+/etc/resolv.conf will do.


Thanks for the patch but some people do not read the documentation 
unfortunately. It would make me happier if we could modify the test such 
that its outcome does not depend on whether or not an intermediate DNS 
server filters out invalid answers.


Bart.

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Net-snmp-coders mailing list
Net-snmp-coders@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/net-snmp-coders

Re: Unstable tests

[Eric S. Raymond]

Bart Van Assche :
> Hello Eric,
> 
> These are the only two tests that sometimes fail on my test setup. Whether
> or not these tests pass depends on your DNS server. If I e.g. add
> "nameserver 8.8.8.8" as the first entry in /etc/resolv.conf then these tests
> pass on my setup. I think the reason is that the domain names used by that
> test resolve into 127.0.0.x and because some DNS servers filter these
> results.
> 
> Bart.

You guys have been at this too long.  You're failing to document your
assumptions.

>From 6cd949342a65ff2260253bca234bfa08f8e3b5c2 Mon Sep 17 00:00:00 2001
From: "Eric S. Raymond" 
Date: Thu, 22 Mar 2018 15:20:21 -0400
Subject: [PATCH] INSTALL: explain workaround for comsec test failures.

---
 INSTALL | 11 +++
 1 file changed, 11 insertions(+)

diff --git a/INSTALL b/INSTALL
index aad9099..9bcd65a 100644
--- a/INSTALL
+++ b/INSTALL
@@ -6,6 +6,7 @@ TABLE OF CONTENTS
 * Net-SNMP Specific Information
   Long (but you should read these) Instructions
   Installing the Perl/SNMP Module
+  Tests
 * Compilers and Options
   Compiling For Multiple Architectures
   Installation Names
@@ -155,6 +156,16 @@ Net-SNMP libraries and demon applications.
 make test
 make install (as root)
 
+Tests
+=
+
+The ordinary self-test sequence can be invoked with "make test". There
+are more comprehensive options.
+
+Spurious failures on the "comsec" tests can be due to misconfigured
+DNS upstream of you. A workaround is to point your DNS server at a
+non-broken one. Adding "nameserver 8.8.8.8" as the first entry in
+/etc/resolv.conf will do.
 
 Compilers and Options
 =
-- 
2.7.4



-- 
http://www.catb.org/~esr/;>Eric S. Raymond

My work is funded by the Internet Civil Engineering Institute: https://icei.org
Please visit their site and donate: the civilization you save might be your own.



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Net-snmp-coders mailing list
Net-snmp-coders@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/net-snmp-coders

Re: Unstable tests

[Eric S. Raymond]

Bill Fenner :
> So before, you meant you failed test 31 of the "com2sec directive" suite?

I'm not even sure that's what I saw. My apologies, I'm new to this code and
the test output is somwhat confusing.

> Try "cd testing; SNMP_VERBOSE=2 ./RUNFULLTESTS -v -r T070com2sec" to get
> more detail about why it failed.
> 
> Test 30 and 31 could fail if your DNS resolver performs "helpful" NXDOMAIN
> remapping - e.g., does
> > ping no.such.address.
> return
> ping: unknown host no.such.address.
> or does it return something like
> PING no.such.address (92.242.140.21) 56(84) bytes of data.

The latter. I guess that's the problem.

> There's an assumption in the test suite that the DNS server is not insane,
> but of course most home ISPs provide insane DNS servers by default.  Most
> providers give you the option to use a sane one, or of course you could
> change your /etc/resolv.conf to point to a sane public DNS provider.

I have a Verizon FIOS business account.  The rot has spread to more than
"home" service.

This problem should be documented where people running the tests will see it.
-- 
http://www.catb.org/~esr/;>Eric S. Raymond

My work is funded by the Internet Civil Engineering Institute: https://icei.org
Please visit their site and donate: the civilization you save might be your own.



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Net-snmp-coders mailing list
Net-snmp-coders@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/net-snmp-coders

Re: Unstable tests

[Bart Van Assche]

On 03/21/18 20:41, Eric S. Raymond wrote:

Ian and I note that the test suite seems a bit compromised.


Hello Eric,

When running compile tests and the test suite I think it is a good idea 
to enable as many Net-SNMP features as possible. You may want to have a 
look at the attached shell scripts - these are the scripts I use myself 
to rebuild Net-SNMP and to run the regression tests. The 
net-snmp-run-perl-tests script needs more work - I have not yet had the 
time to figure out all the environment variable settings to make it pick 
up its dependencies from the build directory instead of requiring "make 
install" to be run first.


Bart.
#!/usr/bin/env bash

if [ -e .git ]; then
  cat .git/HEAD | sed 's:.*/::'
elif [ -e .svn ]; then
  svn info . | sed -n 's=^URL: .*/\([^/]*\)$=\1=p'
else
  echo branch-name-not-known
fi
#!/usr/bin/env bash

# Necessary software packages for building Net-SNMP:
# autoconf automake libtool m4
# glibc glibc-devel libssh2 libssh2-devel openssl openssl-devel rpm-devel
# perl-ExtUtils-MakeMaker perl-ExtUtils-Embed perl-Test-Harness
# python python-devel python-setuptools

scriptdir="$(dirname "$0")"

branch_name="$("$scriptdir/branch-name")"

is_5_4_or_above="false"
is_5_5_or_above="false"
is_5_6_or_above="false"
is_5_7_or_above="false"
is_5_8_or_above="false"
if [ "${branch_name#master}" != "${branch_name}" ]; then
  is_5_8_or_above="true"
  is_5_7_or_above="true"
  is_5_6_or_above="true"
  is_5_5_or_above="true"
  is_5_4_or_above="true"
elif [ "${branch_name#V5-7-}" != "${branch_name}" ]; then
  is_5_7_or_above="true"
  is_5_6_or_above="true"
  is_5_5_or_above="true"
  is_5_4_or_above="true"
elif [ "${branch_name#V5-6-}" != "${branch_name}" ]; then
  is_5_6_or_above="true"
  is_5_5_or_above="true"
  is_5_4_or_above="true"
elif [ "${branch_name#V5-5-}" != "${branch_name}" ]; then
  is_5_5_or_above="true"
  is_5_4_or_above="true"
elif [ "${branch_name#V5-4-}" != "${branch_name}" ] ||
 [ "${branch_name%-5.4.2.1}" != "${branch_name}" ]
then
  is_5_4_or_above="true"
fi
have_rtnetlink=true
#srcdir=${1:-.}; shift
if ! echo "$*" | grep -Eq 'with-openssl=(no|internal)'; then
opensslver="$(
case "${OSTYPE}" in
  linux*)
for p in openssl-devel libopenssl-devel libressl-devel; do
  rpm -q --qf '%{version}\n' $p 2>/dev/null;
done;
dpkg-query --show -f '${Version}\n' libssl-dev 2>/dev/null;
pacman -Q openssl 2>&1 | sed -n 's/^openssl[[:blank:]]*//p';;
  *)
echo 1.0;;
esac)"
fi
if grep -q 'CentOS release 4' /etc/issue 2>/dev/null; then
  opensslver=""
  have_rtnetlink=false
fi
opensshver="$({ rpm -q openssh-devel; rpm -q libssh2-devel; rpm -q 
libssh-devel; } 2>/dev/null | sed -n 
's/^.*ssh[0-9]*-devel-\([0-9a-z.]*\)-.*$/\1/p' | head -n1)"
srcdir=.
cflags="-g -O0 -pipe ${CFLAGS}"
#cflags="-DNETSNMP_USE_ASSERT ${CFLAGS}"
if true; then
  # Disable inlining to make debugging easier
  cflags="$cflags -fno-strict-aliasing -fno-inline -fno-inline-functions 
-fno-inline-functions-called-once -fno-early-inlining -Wno-inline"
  if gcc -fno-inline-small-functions -xc -c - /dev/null 2>&1; then
cflags="$cflags -fno-inline-small-functions"
  fi
fi

# See also the top-level Net-SNMP Makefile.in.
export NOAUTODEPS=y

options=()
options+=(--enable-developer)
options+=(--enable-ipv6)
options+=("--with-cflags=$cflags")
options+=(--with-defaults)
transports=()
[ "$opensshver" != "" ] && $is_5_5_or_above && transports+=(SSH)
# Mib names can be found as follows:
# (cd agent/mibgroup &&
# git grep -lE 'void[[:blank:]]*init_.*\(void\)|config_require\(' |
#  sed -n 's/\.h$//p')
mibs=()
mibs+=(examples/data_set)
mibs+=(examples/delayed_instance)
mibs+=(examples/example)
mibs+=(examples/notification)
mibs+=(examples/scalar_int)
mibs+=(examples/ucdDemoPublic)
mibs+=(examples/watched)
mibs+=(mibII)
mibs+=(smux)

case "$(uname)" in
  Linux*)
  $is_5_8_or_above && options+=(--enable-blumenthal-aes)
  options+=(--enable-new-features)
  options+=("--prefix=/usr/local/net-snmp-${branch_name}")
  $is_5_7_or_above && mibs+=(deliver/deliverByNotify)
  mibs+=(disman/event)
  mibs+=(disman/expression)
  mibs+=(disman/mteEventNotificationTable)
  mibs+=(disman/mteEventTable)
  mibs+=(disman/mteObjectsTable)
  mibs+=(disman/mteTriggerBooleanTable)
  mibs+=(disman/mteTriggerDeltaTable)
  mibs+=(disman/mteTriggerExistenceTable)
  mibs+=(disman/mteTriggerTable)
  mibs+=(disman/mteTriggerThresholdTable)
  mibs+=(disman/nslookup-mib)
  mibs+=(disman/ping-mib)
  mibs+=(disman/schedule)
  mibs+=(disman/traceroute-mib)
  $is_5_5_or_above && $have_rtnetlink && mibs+=(etherlike-mib)
  mibs+=(examples/netSnmpHostsTable)
  mibs+=(hardware/cpu)
  $is_5_5_or_above && mibs+=(hardware/fsys)
  mibs+=(hardware/memory)
  $is_5_5_or_above && mibs+=(hardware/sensors)
  mibs+=(host)
  mibs+=(ip-forward-mib)
  

Re: Unstable tests

[Bart Van Assche]

On 03/22/18 05:33, Eric S. Raymond wrote:

Now I'm getting a different result, which is a little disturbing in itself:

com2sec directive (Wstat: 
256 Tests: 31 Failed: 1)
   Failed test:  30
   Non-zero exit status: 1
Files=76, Tests=327, 109 wallclock secs ( 0.28 usr  0.07 sys + 13.58 cusr  4.74 
csys = 18.67 CPU)
Result: FAIL

We failed these 1 tests:
   com2sec directive ( 
/home/esr/software/net-snmp-code/testing/fulltests/default/T070com2sec_simple )


Hello Eric,

These are the only two tests that sometimes fail on my test setup. 
Whether or not these tests pass depends on your DNS server. If I e.g. 
add "nameserver 8.8.8.8" as the first entry in /etc/resolv.conf then 
these tests pass on my setup. I think the reason is that the domain 
names used by that test resolve into 127.0.0.x and because some DNS 
servers filter these results.


Bart.

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Net-snmp-coders mailing list
Net-snmp-coders@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/net-snmp-coders

Re: Unstable tests

[Bill Fenner]

On Thu, Mar 22, 2018 at 8:33 AM, Eric S. Raymond  wrote:

> Bill Fenner :
> > By "test 31" do you mean "snmpv1 traps are sent by snmpd API"?
>
> Now I'm getting a different result, which is a little disturbing in itself:
>
> com2sec directive
>  (Wstat: 256 Tests: 31 Failed: 1)
>   Failed test:  30
>   Non-zero exit status: 1
> Files=76, Tests=327, 109 wallclock secs ( 0.28 usr  0.07 sys + 13.58 cusr
> 4.74 csys = 18.67 CPU)
> Result: FAIL
>

So before, you meant you failed test 31 of the "com2sec directive" suite?

Try "cd testing; SNMP_VERBOSE=2 ./RUNFULLTESTS -v -r T070com2sec" to get
more detail about why it failed.

Test 30 and 31 could fail if your DNS resolver performs "helpful" NXDOMAIN
remapping - e.g., does
> ping no.such.address.
return
ping: unknown host no.such.address.
or does it return something like
PING no.such.address (92.242.140.21) 56(84) bytes of data.

There's an assumption in the test suite that the DNS server is not insane,
but of course most home ISPs provide insane DNS servers by default.  Most
providers give you the option to use a sane one, or of course you could
change your /etc/resolv.conf to point to a sane public DNS provider.

  Bill
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Net-snmp-coders mailing list
Net-snmp-coders@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/net-snmp-coders

Re: Unstable tests

[Eric S. Raymond]

Bill Fenner :
> The configure script test can fail if you've regenerated it using autoconf
> but not using 2.68.

Running 2.69.

I never ran autoconf explicitly, just configure as per instructions.

> By "test 31" do you mean "snmpv1 traps are sent by snmpd API"?

Now I'm getting a different result, which is a little disturbing in itself:

com2sec directive (Wstat: 
256 Tests: 31 Failed: 1)
  Failed test:  30
  Non-zero exit status: 1
Files=76, Tests=327, 109 wallclock secs ( 0.28 usr  0.07 sys + 13.58 cusr  4.74 
csys = 18.67 CPU)
Result: FAIL

We failed these 1 tests:
  com2sec directive ( 
/home/esr/software/net-snmp-code/testing/fulltests/default/T070com2sec_simple )

> What branch did you use to run your tests?

master
-- 
http://www.catb.org/~esr/;>Eric S. Raymond

My work is funded by the Internet Civil Engineering Institute: https://icei.org
Please visit their site and donate: the civilization you save might be your own.



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Net-snmp-coders mailing list
Net-snmp-coders@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/net-snmp-coders

Re: Unstable tests

[Bill Fenner]

I have an unofficial Travis build setup; in it the only tests that fail are
because the Travis environment seems to not have even localhost IPv6
available: https://travis-ci.org/fenner/net-snmp

The configure script test can fail if you've regenerated it using autoconf
but not using 2.68.

By "test 31" do you mean "snmpv1 traps are sent by snmpd API"?

What branch did you use to run your tests?

  Bill


On Wed, Mar 21, 2018 at 11:41 PM, Eric S. Raymond  wrote:

> Ian and I note that the test suite seems a bit compromised.
>
> On current Ubuntu, Intel x86_64, stock build, two tests - 1 and 31
> fail out of the box.
>
> We see those same failures on ARM (a RasPi 3).
>
> But Ian's laptop gives a diffeent and larger list of errors.
>
> Are there known pastform dependencies in the tests?  Is it known that
> 1 and 31 can fail on a stock build?
> --
> http://www.catb.org/~esr/;>Eric S. Raymond
>
> All forms of government are pernicious, including good government.
> -- Edward Abbey
>
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> Net-snmp-coders mailing list
> Net-snmp-coders@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
>
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Net-snmp-coders mailing list
Net-snmp-coders@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/net-snmp-coders