Re: [PATCH v1 2/2] travis-ci: disable GIT_TEST_HTTPD for macOS

[Torsten Bögershausen]

On 16.11.16 15:39, Heiko Voigt wrote:
> On Tue, Nov 15, 2016 at 10:31:59AM -0500, Jeff King wrote:
>> On Tue, Nov 15, 2016 at 01:07:18PM +0100, Heiko Voigt wrote:
>>
>>> On Fri, Nov 11, 2016 at 09:22:51AM +0100, Lars Schneider wrote:
 To all macOS users on the list:
 Does anyone execute the tests with GIT_TEST_HTTPD enabled successfully?
>>>
>>> Nope. The following tests fail for me on master: 5539, 5540, 5541, 5542,
>>> 5550, 5551, 5561, 5812.
>>
>> Failing how? Does apache fail to start up? Do tests fails? What does
>> "-v" say? Is there anything interesting in httpd/error.log in the trash
>> directory?
> 
> This is what I see for 5539:
> 
> $ GIT_TEST_HTTPD=1 ./t5539-fetch-http-shallow.sh -v
> Initialized empty Git repository in /Users/hvoigt/Repository/git4/t/trash 
> directory.t5539-fetch-http-shallow/.git/
> checking prerequisite: NOT_ROOT
> 
> mkdir -p "$TRASH_DIRECTORY/prereq-test-dir" &&
> (
>   cd "$TRASH_DIRECTORY/prereq-test-dir" &&
>   uid=$(id -u) &&
>   test "$uid" != 0
> 
> )
> prerequisite NOT_ROOT ok
> httpd: Syntax error on line 65 of 
> /Users/hvoigt/Repository/git4/t/lib-httpd/apache.conf: Cannot load 
> modules/mod_mpm_prefork.so into server: 
> dlopen(/Users/hvoigt/Repository/git4/t/trash 
> directory.t5539-fetch-http-shallow/httpd/modules/mod_mpm_prefork.so, 10): 
> image not found
> error: web server setup failed
> 
Yes, same here.

If we take that out:

diff --git a/t/lib-httpd/apache.conf b/t/lib-httpd/apache.conf
index c3e6313..1925fdb 100644
--- a/t/lib-httpd/apache.conf
+++ b/t/lib-httpd/apache.conf
@@ -61,9 +61,6 @@ LockFile accept.lock
 
LoadModule access_compat_module modules/mod_access_compat.so
 
-
-   LoadModule mpm_prefork_module modules/mod_mpm_prefork.so
-
 
LoadModule unixd_module modules/mod_unixd.so
 

I run into other issues:

 [core:emerg] [pid 2502] (2)No such file or directory: AH00023: Couldn't create 
the rewrite-map mutex (file /private/var/run/rewrite-map.2502)
AH00016: Configuration Failed

(apache2 comes via MacPorts)

Re: [PATCH v1 2/2] travis-ci: disable GIT_TEST_HTTPD for macOS

[Heiko Voigt]

On Tue, Nov 15, 2016 at 10:31:59AM -0500, Jeff King wrote:
> On Tue, Nov 15, 2016 at 01:07:18PM +0100, Heiko Voigt wrote:
> 
> > On Fri, Nov 11, 2016 at 09:22:51AM +0100, Lars Schneider wrote:
> > > To all macOS users on the list:
> > > Does anyone execute the tests with GIT_TEST_HTTPD enabled successfully?
> > 
> > Nope. The following tests fail for me on master: 5539, 5540, 5541, 5542,
> > 5550, 5551, 5561, 5812.
> 
> Failing how? Does apache fail to start up? Do tests fails? What does
> "-v" say? Is there anything interesting in httpd/error.log in the trash
> directory?

This is what I see for 5539:

$ GIT_TEST_HTTPD=1 ./t5539-fetch-http-shallow.sh -v
Initialized empty Git repository in /Users/hvoigt/Repository/git4/t/trash 
directory.t5539-fetch-http-shallow/.git/
checking prerequisite: NOT_ROOT

mkdir -p "$TRASH_DIRECTORY/prereq-test-dir" &&
(
cd "$TRASH_DIRECTORY/prereq-test-dir" &&
uid=$(id -u) &&
test "$uid" != 0

)
prerequisite NOT_ROOT ok
httpd: Syntax error on line 65 of 
/Users/hvoigt/Repository/git4/t/lib-httpd/apache.conf: Cannot load 
modules/mod_mpm_prefork.so into server: 
dlopen(/Users/hvoigt/Repository/git4/t/trash 
directory.t5539-fetch-http-shallow/httpd/modules/mod_mpm_prefork.so, 10): image 
not found
error: web server setup failed


It seems the other failures have the same cause.

Cheers Heiko

Re: [PATCH v1 2/2] travis-ci: disable GIT_TEST_HTTPD for macOS

[Jeff King]

On Tue, Nov 15, 2016 at 01:07:18PM +0100, Heiko Voigt wrote:

> On Fri, Nov 11, 2016 at 09:22:51AM +0100, Lars Schneider wrote:
> > To all macOS users on the list:
> > Does anyone execute the tests with GIT_TEST_HTTPD enabled successfully?
> 
> Nope. The following tests fail for me on master: 5539, 5540, 5541, 5542,
> 5550, 5551, 5561, 5812.

Failing how? Does apache fail to start up? Do tests fails? What does
"-v" say? Is there anything interesting in httpd/error.log in the trash
directory?

-Peff

Re: [PATCH v1 2/2] travis-ci: disable GIT_TEST_HTTPD for macOS

[Lars Schneider]

> On 15 Nov 2016, at 13:07, Heiko Voigt  wrote:
> 
> On Fri, Nov 11, 2016 at 09:22:51AM +0100, Lars Schneider wrote:
>> To all macOS users on the list:
>> Does anyone execute the tests with GIT_TEST_HTTPD enabled successfully?
> 
> Nope. The following tests fail for me on master: 5539, 5540, 5541, 5542,
> 5550, 5551, 5561, 5812.

That's what I see, too. Apache needs to be configured in some special way 
to make them work and I was wondering if anyone has figured that out
already for macOS...

However, I much prefer Peff's idea to test against real world servers:
http://public-inbox.org/git/2016092824.qqgrmhtkuw3wp...@sigill.intra.peff.net/

- Lars

Re: [PATCH v1 2/2] travis-ci: disable GIT_TEST_HTTPD for macOS

[Heiko Voigt]

On Fri, Nov 11, 2016 at 09:22:51AM +0100, Lars Schneider wrote:
> To all macOS users on the list:
> Does anyone execute the tests with GIT_TEST_HTTPD enabled successfully?

Nope. The following tests fail for me on master: 5539, 5540, 5541, 5542,
5550, 5551, 5561, 5812.

Cheers Heiko

Re: [PATCH v1 2/2] travis-ci: disable GIT_TEST_HTTPD for macOS

[Jeff King]

On Fri, Nov 11, 2016 at 10:13:44AM +0100, Lars Schneider wrote:

> > If you did want to have a more real-world network-based test, I think
> > the right solution is not for GitHub to set up a bunch of mock servers,
> > but to design client-side tests that hit the _real_ GitHub (or GitLab,
> > or whatever) and perform some basic operations. OTOH, people running
> > "master" (or "next", etc) are doing that implicitly every day.
> 
> That is actually a neat idea. We could setup a test repo on each of the major 
> Git hosting sites and then the TravisCI run could clone a repo and push
> changes to it. That shouldn't take long and would probably be a good real
> world test.
> 
> The credentials of these repos could be stored encrypted in Travis CI [1].
> 
> Where would such a test repo live on github.com? On github.com/git or would
> you prefer a separate organization? (no worries, I am not going to tackle this
> anytime soon -- too many things in my backlog already).

I think I'd prefer for it to live outside of the "git" organization
entirely, if only because it's credentials will be necessarily less
secure. It would be nice if people could point the suite at their own
user/repo, too, so they can run it outside of travis.

> BTW: I just noticed https://github.com/git/hello-world ... is this legitimate
> or did someone hack github.com/git? :)

Hmm. I wondered myself. There's no audit record of who created it, but
the creation timestamp is from 2008-07-23, which predates a lot of the
logging. So offhand, I'd say the likely explanation is "Scott Chacon
experimenting".

It's probably worth cleaning out now, though.

-Peff

Re: [PATCH v1 2/2] travis-ci: disable GIT_TEST_HTTPD for macOS

[Lars Schneider]

On 11 Nov 2016, at 09:47, Jeff King  wrote:

> On Fri, Nov 11, 2016 at 09:22:51AM +0100, Lars Schneider wrote:
> 
>> There would be an alternative way to approach the problem:
>> Someone (GitHub?, BitBucket?, GitLab?, ...) could setup a bunch of webservers
>> with popular configurations and a way to reset a clean test environment. 
>> Then 
>> the TravisCI client tests could go against these servers.
>> 
>> I realize that this idea is probably unrealistic because too much setup and
>> maintenance work would be required.
> 
> Yeah, it seems like it adds a lot of complexity for little gain. Plus it
> creates a network dependency on running the tests. I know you care
> mostly about Travis, but I am much more interested in all of the people
> (developers and not) who run "make test" on their own platforms.
> 
> If you did want to have a more real-world network-based test, I think
> the right solution is not for GitHub to set up a bunch of mock servers,
> but to design client-side tests that hit the _real_ GitHub (or GitLab,
> or whatever) and perform some basic operations. OTOH, people running
> "master" (or "next", etc) are doing that implicitly every day.

That is actually a neat idea. We could setup a test repo on each of the major 
Git hosting sites and then the TravisCI run could clone a repo and push
changes to it. That shouldn't take long and would probably be a good real
world test.

The credentials of these repos could be stored encrypted in Travis CI [1].

Where would such a test repo live on github.com? On github.com/git or would
you prefer a separate organization? (no worries, I am not going to tackle this
anytime soon -- too many things in my backlog already).

BTW: I just noticed https://github.com/git/hello-world ... is this legitimate
or did someone hack github.com/git? :)

Cheers,
Lars


[1] 
https://docs.travis-ci.com/user/environment-variables/#Defining-encrypted-variables-in-.travis.yml

Re: [PATCH v1 2/2] travis-ci: disable GIT_TEST_HTTPD for macOS

[Jeff King]

On Fri, Nov 11, 2016 at 09:22:51AM +0100, Lars Schneider wrote:

> There would be an alternative way to approach the problem:
> Someone (GitHub?, BitBucket?, GitLab?, ...) could setup a bunch of webservers
> with popular configurations and a way to reset a clean test environment. Then 
> the TravisCI client tests could go against these servers.
>
> I realize that this idea is probably unrealistic because too much setup and
> maintenance work would be required.

Yeah, it seems like it adds a lot of complexity for little gain. Plus it
creates a network dependency on running the tests. I know you care
mostly about Travis, but I am much more interested in all of the people
(developers and not) who run "make test" on their own platforms.

If you did want to have a more real-world network-based test, I think
the right solution is not for GitHub to set up a bunch of mock servers,
but to design client-side tests that hit the _real_ GitHub (or GitLab,
or whatever) and perform some basic operations. OTOH, people running
"master" (or "next", etc) are doing that implicitly every day.

-Peff

Re: [PATCH v1 2/2] travis-ci: disable GIT_TEST_HTTPD for macOS

[Lars Schneider]

On 10 Nov 2016, at 17:10, Jeff King  wrote:

> On Thu, Nov 10, 2016 at 12:07:14PM +0100, Lars Schneider wrote:
> 
>>> Using Apache in the tests has been the source of frequent portability
>>> problems and configuration headaches. I do wonder if we'd be better off
>>> using some small special-purpose web server (even a short perl script
>>> written around HTTP::Server::Simple or something).
>>> 
>>> On the other hand, testing against Apache approximates a more real-world
>>> case, which has value. It might be nice if our tests supported multiple
>>> web servers, but that would mean duplicating the config for each
>>> manually.
>> 
>> I agree that the real-world Apache test is more valuable and I really want
>> to keep the Linux Apache test running. However, I don't think many people
>> use macOS as Git web server and therefore I thought it is not worth the
>> effort to investigate this problem further.
> 
> IMHO, the value in the http tests is not testing the server side, but
> the client side. Without being able to set up a dummy HTTP server, we do
> not have any way to exercise the client side of git-over-http at all.
> And people on macOS _do_ use that. :)

Well, I haven't seen it from that perspective, yet, but I agree :-)


To all macOS users on the list:
Does anyone execute the tests with GIT_TEST_HTTPD enabled successfully?


There would be an alternative way to approach the problem:
Someone (GitHub?, BitBucket?, GitLab?, ...) could setup a bunch of webservers
with popular configurations and a way to reset a clean test environment. Then 
the TravisCI client tests could go against these servers.

I realize that this idea is probably unrealistic because too much setup and
maintenance work would be required.

Cheers,
Lars

Re: [PATCH v1 2/2] travis-ci: disable GIT_TEST_HTTPD for macOS

[Jeff King]

On Thu, Nov 10, 2016 at 01:49:52PM -0800, Junio C Hamano wrote:

> Yes, I recall the IIS one raised and discussed at least twice on the
> list in the past, and it sounded that we want some solution to that.

The patches had some issues. I suspect the population of people who want
to run a git server on IIS is relatively small. I am content to wait for
somebody who has such a setup to produce a working patch.

> >   3. What happens when you ask for "foo.git/info/refs" and "foo.git" is
> >  a bundle file (Apache gives you a 404, lighttpd serves the bundle).
> 
> That's a bad one.  Do we want a client-side "I am connecting to a
> site that knows how to talk smart-http" option or something to work
> it around?

It doesn't matter because we don't actually support fetching HTTP
bundles via "git fetch" yet.

But I ran it across the issue and did make such a fix when I was
implementing that feature long ago. See the discussion of "surprise" in
[1] and [2].

Wow, that series is exactly 5 years old today. Have I really been
procrastinating on re-rolling it that long? Yikes.

-Peff

[1] http://public-inbox.org/git/2010075052.gi27...@sigill.intra.peff.net/

[2] http://public-inbox.org/git/2010075052.gi27...@sigill.intra.peff.net/

Re: [PATCH v1 2/2] travis-ci: disable GIT_TEST_HTTPD for macOS

[Junio C Hamano]

Jeff King  writes:

>> I however do not know what the universally available simplest dummy
>> HTTP server would be.  There probably are better alternative than
>> Apache with distro-customized ways of configuration that we have to
>> adjust.  
>> 
>> A solution around HTTP::Server::Simple sounds attractive but is it
>> a realistic alternative or too much effort required?  I dunno.
>
> I'm less concerned about the amount of effort (though I agree it may be
> a blocker) than about the fact that it may not behave similarly to real
> servers.

I was wondering if it takes too much effort to make it behave
similarly to real servers, so I guess we are on the same page.  

Yes, I recall the IIS one raised and discussed at least twice on the
list in the past, and it sounded that we want some solution to that.

>   3. What happens when you ask for "foo.git/info/refs" and "foo.git" is
>  a bundle file (Apache gives you a 404, lighttpd serves the bundle).

That's a bad one.  Do we want a client-side "I am connecting to a
site that knows how to talk smart-http" option or something to work
it around?

Re: [PATCH v1 2/2] travis-ci: disable GIT_TEST_HTTPD for macOS

[Jeff King]

On Thu, Nov 10, 2016 at 01:33:29PM -0800, Junio C Hamano wrote:

> Jeff King  writes:
> 
> > IMHO, the value in the http tests is not testing the server side, but
> > the client side. Without being able to set up a dummy HTTP server, we do
> > not have any way to exercise the client side of git-over-http at all.
> > And people on macOS _do_ use that. :)
> 
> Amen to that.
> 
> I however do not know what the universally available simplest dummy
> HTTP server would be.  There probably are better alternative than
> Apache with distro-customized ways of configuration that we have to
> adjust.  
> 
> A solution around HTTP::Server::Simple sounds attractive but is it
> a realistic alternative or too much effort required?  I dunno.

I'm less concerned about the amount of effort (though I agree it may be
a blocker) than about the fact that it may not behave similarly to real
servers. We have had real bugs and surprises with the way that various
web servers implement things. A few that come to mind are:

  1. Buffering/deadlock issues between the webserver and the CGI (this
 was an issue with Apache, but I don't know about other servers).

  2. The handling of CONTENT_LENGTH with chunked-encoding (this is still
 an issue with IIS).

  3. What happens when you ask for "foo.git/info/refs" and "foo.git" is
 a bundle file (Apache gives you a 404, lighttpd serves the bundle).

Ideally we'd test against a lot of different webservers, but that's
expensive (in CPU, but also in developer time). But I'd guess that
Apache is at least more representative than HTTP::Server::Simple of real
servers in the wild.

So if we had a simple fallback in addition to Apache, I'd be OK with
that. But we still have a problem that (say) people on MacOS would never
actually test against Apache, because it's not supported there.

-Peff

Re: [PATCH v1 2/2] travis-ci: disable GIT_TEST_HTTPD for macOS

[Junio C Hamano]

Jeff King  writes:

> IMHO, the value in the http tests is not testing the server side, but
> the client side. Without being able to set up a dummy HTTP server, we do
> not have any way to exercise the client side of git-over-http at all.
> And people on macOS _do_ use that. :)

Amen to that.

I however do not know what the universally available simplest dummy
HTTP server would be.  There probably are better alternative than
Apache with distro-customized ways of configuration that we have to
adjust.  

A solution around HTTP::Server::Simple sounds attractive but is it
a realistic alternative or too much effort required?  I dunno.

Re: [PATCH v1 2/2] travis-ci: disable GIT_TEST_HTTPD for macOS

[Jeff King]

On Thu, Nov 10, 2016 at 12:07:14PM +0100, Lars Schneider wrote:

> > Using Apache in the tests has been the source of frequent portability
> > problems and configuration headaches. I do wonder if we'd be better off
> > using some small special-purpose web server (even a short perl script
> > written around HTTP::Server::Simple or something).
> > 
> > On the other hand, testing against Apache approximates a more real-world
> > case, which has value. It might be nice if our tests supported multiple
> > web servers, but that would mean duplicating the config for each
> > manually.
> 
> I agree that the real-world Apache test is more valuable and I really want
> to keep the Linux Apache test running. However, I don't think many people
> use macOS as Git web server and therefore I thought it is not worth the
> effort to investigate this problem further.

IMHO, the value in the http tests is not testing the server side, but
the client side. Without being able to set up a dummy HTTP server, we do
not have any way to exercise the client side of git-over-http at all.
And people on macOS _do_ use that. :)

-Peff

Re: [PATCH v1 2/2] travis-ci: disable GIT_TEST_HTTPD for macOS

[Lars Schneider]

> On 07 Nov 2016, at 22:20, Jeff King  wrote:
> 
> On Sun, Nov 06, 2016 at 10:42:36PM +0100, Lars Schneider wrote:
> 
>>> From: Lars Schneider 
>>> 
>>> TravisCI changed their default macOS image from 10.10 to 10.11 [1].
>>> Unfortunately the HTTPD tests do not run out of the box using the
>>> pre-installed Apache web server anymore. Therefore we enable these
>>> tests only for Linux and disable them for macOS.
>> [...]
>> Hi Junio,
>> 
>> the patch above is one of two patches to make TravisCI pass, again.
>> Could you queue it?
> 
> I don't really mind disabling tests if they don't run on a platform. But
> the more interesting question to me is: why don't they run any more? Is
> there some config tweak needed, or is it an insurmountable problem?

I can't really remember what the problem was. I think some apache config
required some module that was not present and I wasn't able to get this
working quickly.


> Using Apache in the tests has been the source of frequent portability
> problems and configuration headaches. I do wonder if we'd be better off
> using some small special-purpose web server (even a short perl script
> written around HTTP::Server::Simple or something).
> 
> On the other hand, testing against Apache approximates a more real-world
> case, which has value. It might be nice if our tests supported multiple
> web servers, but that would mean duplicating the config for each
> manually.

I agree that the real-world Apache test is more valuable and I really want
to keep the Linux Apache test running. However, I don't think many people
use macOS as Git web server and therefore I thought it is not worth the
effort to investigate this problem further.

- Lars

Re: [PATCH v1 2/2] travis-ci: disable GIT_TEST_HTTPD for macOS

[Jeff King]

On Sun, Nov 06, 2016 at 10:42:36PM +0100, Lars Schneider wrote:

> > From: Lars Schneider 
> > 
> > TravisCI changed their default macOS image from 10.10 to 10.11 [1].
> > Unfortunately the HTTPD tests do not run out of the box using the
> > pre-installed Apache web server anymore. Therefore we enable these
> > tests only for Linux and disable them for macOS.
> [...]
> Hi Junio,
> 
> the patch above is one of two patches to make TravisCI pass, again.
> Could you queue it?

I don't really mind disabling tests if they don't run on a platform. But
the more interesting question to me is: why don't they run any more? Is
there some config tweak needed, or is it an insurmountable problem?

Using Apache in the tests has been the source of frequent portability
problems and configuration headaches. I do wonder if we'd be better off
using some small special-purpose web server (even a short perl script
written around HTTP::Server::Simple or something).

On the other hand, testing against Apache approximates a more real-world
case, which has value. It might be nice if our tests supported multiple
web servers, but that would mean duplicating the config for each
manually.

-Peff

Re: [PATCH v1 2/2] travis-ci: disable GIT_TEST_HTTPD for macOS

[Lars Schneider]

> On 17 Oct 2016, at 02:25, larsxschnei...@gmail.com wrote:
> 
> From: Lars Schneider 
> 
> TravisCI changed their default macOS image from 10.10 to 10.11 [1].
> Unfortunately the HTTPD tests do not run out of the box using the
> pre-installed Apache web server anymore. Therefore we enable these
> tests only for Linux and disable them for macOS.
> 
> [1] https://blog.travis-ci.com/2016-10-04-osx-73-default-image-live/
> 
> Signed-off-by: Lars Schneider 
> ---
> .travis.yml | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/.travis.yml b/.travis.yml
> index 37a1e1f..d752447 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -32,7 +32,6 @@ env:
> - DEFAULT_TEST_TARGET=prove
> - GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
> - GIT_TEST_OPTS="--verbose --tee"
> -- GIT_TEST_HTTPD=true
> - GIT_TEST_CLONE_2GB=YesPlease
> # t9810 occasionally fails on Travis CI OS X
> # t9816 occasionally fails with "TAP out of sequence errors" on Travis CI 
> OS X
> @@ -57,6 +56,8 @@ before_install:
>   - >
> case "${TRAVIS_OS_NAME:-linux}" in
> linux)
> +  export GIT_TEST_HTTPD=YesPlease
> +
>   mkdir --parents custom/p4
>   pushd custom/p4
> wget --quiet 
> http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION/bin.linux26x86_64/p4d
> -- 
> 2.10.0
> 

Hi Junio,

the patch above is one of two patches to make TravisCI pass, again.
Could you queue it?

The other patch is [1] which is still under discussion.

Thank you,
Lars

[1] http://public-inbox.org/git/8c67ff53-c26f-4993-908f-a5183c5e4...@gmail.com/