Re: Network connection

[Adam Kennedy]

Additional note.

The internet is not all the same. Different network environments will screw
you on almost anything you try, especially if it’s not HTTP (which at least
has protocols in operating systems for detecting “real internet”)

Don’t do it in install tests, generally.

Adam

On Tue, 14 Nov 2017 at 9:29 pm, Adam Kennedy  wrote:

> I have fragments and variations of this stuff tucked away in old code I’m
> sure.
>
> Look at LWP::Online and Test::NeedsDisplay, and I may have done similar
> things in the test suites for ORDB::*
>
> Adam
>
> On Sun, 12 Nov 2017 at 7:00 am, James E Keenan  wrote:
>
>> Is there any code which is considered "best practice" for "skip_all
>> unless I have an Internet connection"?
>>
>> For the first time I want to write a library whose tests will benefit
>> from connection to an FTP server.  If there is some code that I could
>> just drop in for situations where the test suite is run offline, that
>> would be helpful.
>>
>> Thank you very much.
>> Jim Keenan
>>
>

Re: Network connection

[Adam Kennedy]

I have fragments and variations of this stuff tucked away in old code I’m
sure.

Look at LWP::Online and Test::NeedsDisplay, and I may have done similar
things in the test suites for ORDB::*

Adam

On Sun, 12 Nov 2017 at 7:00 am, James E Keenan  wrote:

> Is there any code which is considered "best practice" for "skip_all
> unless I have an Internet connection"?
>
> For the first time I want to write a library whose tests will benefit
> from connection to an FTP server.  If there is some code that I could
> just drop in for situations where the test suite is run offline, that
> would be helpful.
>
> Thank you very much.
> Jim Keenan
>

Re: Network connection

[James E Keenan]

On 11/12/2017 12:01 PM, David Golden wrote:
There's https://metacpan.org/pod/Test::Internet and 
https://metacpan.org/pod/Test::RequiresInternet .


I do recommend restricting those AUTOMATED_TESTING or AUTHOR_TESTING.

There's also the core-only approach like is used in HTTP::Tiny: 
https://metacpan.org/source/DAGOLDEN/HTTP-Tiny-0.070/t/200_live.t


David


On Sun, Nov 12, 2017 at 9:59 AM, James E Keenan > wrote:


Is there any code which is considered "best practice" for "skip_all
unless I have an Internet connection"?

For the first time I want to write a library whose tests will
benefit from connection to an FTP server.  If there is some code
that I could just drop in for situations where the test suite is run
offline, that would be helpful.

Thank you very much.
Jim Keenan



I ended up going with both Test::RequiresInternet and an envvar named 
PERL_ALLOW_NETWORK_TESTING normally set to off.  So time-consuming tests 
will only be attempted if there's a network connection and if the user 
wants to run tests which attempt that connection.


Thank you very much.
Jim Keenan

Re: Network connection

[Kent Fredric]

On 13 November 2017 at 04:46, Chad Granum  wrote:
> Kent, (or anyone really)
>
> Can you point to any documentation that covers these env vars, and the best
> practices you layed out? I have never even heard of the NO_NETWORK_TESTING
> var, and considering what modules I maibtain that is worrysome.
>
> I am not asking out of doubt, I want to educate myself, and make these vars
> clearly documented along with when to use them in Test-Simple and related
> modules.

Its more an emerging convention that is gaining adoption, than an
official strategy.

I started an email to this list a few years back,
https://www.nntp.perl.org/group/perl.cpan.workers/2014/06/msg1117.html
and this network variable ended up being rolled out in a few places as a result.

The rationale as to why this approach is warranted is spelled out in
that thread, but the gist of it is its not always safe to use sniffing
to determine network availability, doing so can get given
false-positives through network sandboxes and weird routing
configurations, and syscall sandboxing can result in network calls
tripping a SIGKILL on your process.

Hence the application of an ENV var that states "Do not under any
circumstances attempt to access the internet" serves as a decent gate
keeper.

This ENV var is still far from ubiquitous, but it is far more applied
across CPAN than its competition these days.

And for what its worth, NO_NETWORK_TESTING is employed by default when
Gentoo builds and tests packages, unless the user explicitly opts to
enable network tests, and anything that respects that ENV var
automatically benefits. ( Because from a security standpoint, the
machine which fetches the .tar.gz's and the machine that installs and
tests a package need not be the same machine, nor need be the same
security level, and so the default approach is to ensure the build
environment is tightly locked down, but this can cause spurious test
failures when tests try to dial out to nowhere. Sometimes things as
innocuous as trying to resolve "localhost" can get you having problems
)

Re: Network connection

[David Golden]

There's https://metacpan.org/pod/Test::Internet and
https://metacpan.org/pod/Test::RequiresInternet .

I do recommend restricting those AUTOMATED_TESTING or AUTHOR_TESTING.

There's also the core-only approach like is used in HTTP::Tiny:
https://metacpan.org/source/DAGOLDEN/HTTP-Tiny-0.070/t/200_live.t

David


On Sun, Nov 12, 2017 at 9:59 AM, James E Keenan  wrote:

> Is there any code which is considered "best practice" for "skip_all unless
> I have an Internet connection"?
>
> For the first time I want to write a library whose tests will benefit from
> connection to an FTP server.  If there is some code that I could just drop
> in for situations where the test suite is run offline, that would be
> helpful.
>
> Thank you very much.
> Jim Keenan
>



-- 
David Golden  Twitter/IRC/GitHub: @xdg

Re: Network connection

[Chad Granum]

Kent, (or anyone really)

Can you point to any documentation that covers these env vars, and the best
practices you layed out? I have never even heard of the NO_NETWORK_TESTING
var, and considering what modules I maibtain that is worrysome.

I am not asking out of doubt, I want to educate myself, and make these vars
clearly documented along with when to use them in Test-Simple and related
modules.

-Chad

On Nov 12, 2017 7:42 AM, "Kent Fredric"  wrote:

> On 13 November 2017 at 03:59, James E Keenan  wrote:
> > Is there any code which is considered "best practice" for "skip_all
> unless I
> > have an Internet connection"?
> >
> > For the first time I want to write a library whose tests will benefit
> from
> > connection to an FTP server.  If there is some code that I could just
> drop
> > in for situations where the test suite is run offline, that would be
> > helpful.
> >
> > Thank you very much.
> > Jim Keenan
>
> There's a defacto ENV var, which if set, one should do no kind of
> network activity:
>
> NO_NETWORK_TESTING
>
> After taking that in consideration, whether you do any subsequent
> network IO depends on the nature of what you're building.
>
> Ideally, you should aim to be maximally capable of testing your
> library in the absence of a network connection, and have safe-guards
> in place that determine if the network-environment is satisfactory (
> eg: first by checking for the aforementioned ENV var, and then doing
> basic TCP probes ), and then making the tests execution determinate on
> that being satisfactory.
>
> But if Internet access is a quintessential element of whatever you're
> designing, then you should try more eagerly to run network tests.
>
> At least, that's the recommended flow for "end users"
>
> When considering maintainers and AUTOMATED_TESTING, one should be even
> more eager to execute tests, because for these targets, a spurious
> test failure is has fewer negative consequences, and you want to
> actively see any anomalies in the tests.
>
> So for the AUTOMATED_TESTING/AUTHOR_TESTING/RELEASE_TESTING targets,
> one should aim to run network IO tests in all circumstances other than
> an explicit NO_NETWORK_TESTING circumstance.
>
> --
> Kent
>
> KENTNL - https://metacpan.org/author/KENTNL
>

Re: Network connection

[Kent Fredric]

On 13 November 2017 at 03:59, James E Keenan  wrote:
> Is there any code which is considered "best practice" for "skip_all unless I
> have an Internet connection"?
>
> For the first time I want to write a library whose tests will benefit from
> connection to an FTP server.  If there is some code that I could just drop
> in for situations where the test suite is run offline, that would be
> helpful.
>
> Thank you very much.
> Jim Keenan

There's a defacto ENV var, which if set, one should do no kind of
network activity:

NO_NETWORK_TESTING

After taking that in consideration, whether you do any subsequent
network IO depends on the nature of what you're building.

Ideally, you should aim to be maximally capable of testing your
library in the absence of a network connection, and have safe-guards
in place that determine if the network-environment is satisfactory (
eg: first by checking for the aforementioned ENV var, and then doing
basic TCP probes ), and then making the tests execution determinate on
that being satisfactory.

But if Internet access is a quintessential element of whatever you're
designing, then you should try more eagerly to run network tests.

At least, that's the recommended flow for "end users"

When considering maintainers and AUTOMATED_TESTING, one should be even
more eager to execute tests, because for these targets, a spurious
test failure is has fewer negative consequences, and you want to
actively see any anomalies in the tests.

So for the AUTOMATED_TESTING/AUTHOR_TESTING/RELEASE_TESTING targets,
one should aim to run network IO tests in all circumstances other than
an explicit NO_NETWORK_TESTING circumstance.

-- 
Kent

KENTNL - https://metacpan.org/author/KENTNL

Re: Network connection

[Chad Granum]

I personally would make it an author test that requires the AUTHOR_TESTING
env var.

I have apent too much time debugging network tests that fail because a file
moved, a domain was down/expired, or the connection was not sufficient, but
present. There are also corperate/school blacklists/whitelists. Any number
of things outside your control.

Run the test for release, but do not aubject evertone to these
uncontrollable failure cases.

On Nov 12, 2017 7:00 AM, "James E Keenan"  wrote:

> Is there any code which is considered "best practice" for "skip_all unless
> I have an Internet connection"?
>
> For the first time I want to write a library whose tests will benefit from
> connection to an FTP server.  If there is some code that I could just drop
> in for situations where the test suite is run offline, that would be
> helpful.
>
> Thank you very much.
> Jim Keenan
>