[issue10483] http.server - what is executable on Windows

[mike mcleod]

Change by mike mcleod :


--
pull_requests: +28437
pull_request: https://github.com/python/cpython/pull/30216

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

[STINNER Victor]

Change by STINNER Victor :


--
nosy:  -vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

[mike mcleod]

Change by mike mcleod :


--
keywords: +patch
pull_requests: +28141
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/29624

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

[mike mcleod]

mike mcleod  added the comment:

I will work on this next week.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

[STINNER Victor]

STINNER Victor  added the comment:

> I got the impression that most core developers used Unix-variants for their 
> development, and likely most CGI users as well, and that Windows use of CGI 
> just wasn't a priority for anyone in the core team.

Core devs are volunteers who are free to pick what they like the most. There 
are few core devs who use Python on Windows, whereas they are many Python users 
on Windows.

If you are able to split your enhancements as small PRs, I may be able to have 
a look. We do run our test suite on Windows. If each enhancement has a related 
test valided on Windows, it's ok.


> Deprecation would certainly be a disservice to the community, as CGI is 
the only (as far as I know) universal service available on pretty much 
all web server implementations.

The other question is not if CGI is useful or not, but if it should be 
maintained inside the stdlib, or on PyPI. Well, it's not exclusive :-) It's 
possible to have a more complete implementation on PyPI under a different name.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

[Glenn Linderman]

Glenn Linderman  added the comment:

On 11/18/2021 7:36 AM, STINNER Victor wrote:
> STINNER Victor  added the comment:
>
> I searched for open issues which contain "cgi" in their title. I found 43 
> open issues. The oldest is 101 months ago.
>
> In 10 years, Lib/cgi.py got 43 commits. Only 13 commits in the last 5 years 
> (since 2016-01-01).
>
> It seems like the cgi module is not really maintained anymore. One option is 
> to do nothing: I guess that the most basic features continue to work.
I got the definite feeling 10 years ago that CGI wasn't being 
maintained, at least for Windows use, and that's why I forked it for 
private use and enhancement.  Nothing in the interim has made me change 
my mind, and your statistics support that.  I got the impression that 
most core developers used Unix-variants for their development, and 
likely most CGI users as well, and that Windows use of CGI just wasn't a 
priority for anyone in the core team.  But it has been useful for me, 
and if a little work were put into supporting it, I suspect it could be 
useful to others as well. But as a lowly non-core user unfamiliar with 
the ever-changing processes for submitting patches, I felt rather 
ignored (which apparently isn't an uncommon issue for users like me), 
but neither did I have the time to invest to learn the submission 
protocols, much less to advance to core development status, so I 
realized that it was partly my fault as well.

> Another option is to start deprecating all code related to CGI in the stdlib.
>
> While CGI is not "commonly" used, it seems like it remains popular for 
> specific usages. So I don't think that it's time to deprecate it :-)

Deprecation would certainly be a disservice to the community, as CGI is 
the only (as far as I know) universal service available on pretty much 
all web server implementations.  There are probably better interfaces to 
user-supplied code on many web servers, but the ones I've heard about 
are not as universal, and not as simple for users to implement.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

[Ethan Furman]

Change by Ethan Furman :


--
versions: +Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

[mike mcleod]

mike mcleod  added the comment:

Hi Victor,

Ok I'll do that.

Regards,
Mike

On Thu, 18 Nov 2021 at 15:36, STINNER Victor  wrote:

>
> STINNER Victor  added the comment:
>
> I searched for open issues which contain "cgi" in their title. I found 43
> open issues. The oldest is 101 months ago.
>
> In 10 years, Lib/cgi.py got 43 commits. Only 13 commits in the last 5
> years (since 2016-01-01).
>
> It seems like the cgi module is not really maintained anymore. One option
> is to do nothing: I guess that the most basic features continue to work.
> Another option is to start deprecating all code related to CGI in the
> stdlib.
>
> While CGI is not "commonly" used, it seems like it remains popular for
> specific usages. So I don't think that it's time to deprecate it :-)
>
>
> > You can for sure make a pull request and see if a core dev will review
> and merge it
>
> Yep, just do that ;-) So far, nobody proposed a PR to fix this issue.
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

[STINNER Victor]

STINNER Victor  added the comment:

I searched for open issues which contain "cgi" in their title. I found 43 open 
issues. The oldest is 101 months ago.

In 10 years, Lib/cgi.py got 43 commits. Only 13 commits in the last 5 years 
(since 2016-01-01).

It seems like the cgi module is not really maintained anymore. One option is to 
do nothing: I guess that the most basic features continue to work. Another 
option is to start deprecating all code related to CGI in the stdlib.

While CGI is not "commonly" used, it seems like it remains popular for specific 
usages. So I don't think that it's time to deprecate it :-)


> You can for sure make a pull request and see if a core dev will review and 
> merge it

Yep, just do that ;-) So far, nobody proposed a PR to fix this issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

[STINNER Victor]

STINNER Victor  added the comment:

> I have put the question on Discuss and wait for (any) responses,

Link to the discussion:
https://discuss.python.org/t/issue-10483-http-server-what-is-executable-on-windows/11856

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

[mike mcleod]

mike mcleod  added the comment:

I have put the question on Discuss and wait for (any) responses,

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

[Éric Araujo]

Éric Araujo  added the comment:

It’s not clear to me if CGI support in http.server is a relic or still 
something useful and in wide use.  You can for sure make a pull request and see 
if a core dev will review and merge it, or you could raise the question on 
Discuss to see what people think.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

[mike mcleod]

mike mcleod  added the comment:

Should I go ahead and make the changes as per msg122208 ? on my local copy and 
test?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

[Glenn Linderman]

Glenn Linderman  added the comment:

Ethan,  3.7.2 at the moment.  I doubt there'd be any issues moving it to 3.11, 
unless there have been significant changes to the main branch in that area, I 
just haven't bothered because it is working for me as is.  I forget what 
version I started with, but I guess 3.6 was when I first reported it?  I know I 
did have to tweak it a little when I upgraded Python once, and there had been 
changes to http.server in the meantime.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

[Ethan Furman]

Ethan Furman  added the comment:

Glen, which version of Python is your fork using?

--
nosy: +ethan.furman

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

[Glenn Linderman]

Glenn Linderman  added the comment:

As the original author of this report, getting significant pushback from the 
then maintainer of the code, I went ahead and implemented what I considered to 
be a useful set of features to make CGI work on Windows.

I never proposed them as a patch, because they were bundled with a bunch of 
other things that certainly would have been considered feature requests, and I 
never figured out the process for submitting patches, which has changed a 
couple times since the bug was submitted, and I didn't have time to do all of
(1) learn the churning patch mechanisms
(2) separate my code changes into independent chunks
(3) also get my projects completed

The only reason that CGI support on Windows may not be relevant today is that 
it is so outdated, with no support or encouragement from the maintainers to 
encourage patches to make it useful, discouraging its use on Windows.  With my 
private, forked implementation, I have a quite useful Windows testbed for 
implementing web server code that can run on Linux Apache servers.  If only 
there were such support built-in!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

[mike mcleod]

mike mcleod  added the comment:

Hi  Éric,

Then would the easiest way of dealing with this issue, to close it without
any further work?
Possibly adding to the documents it doesn't work on windows..
I can move to the next issue.

Regards,
Mike

On Sun, 31 Oct 2021 at 18:03, Éric Araujo  wrote:

>
> Éric Araujo  added the comment:
>
> I don’t know if CGI on windows servers is very relevant nowadays.
>
> --
> nosy: +eric.araujo
> versions:  -Python 3.2
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

[Éric Araujo]

Éric Araujo  added the comment:

I don’t know if CGI on windows servers is very relevant nowadays.

--
nosy: +eric.araujo
versions:  -Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

[mike mcleod]

mike mcleod  added the comment:

Hi,
I would like to help on this issue.
Let me know what can be done?

--
nosy: +mikecmcleod

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

[Glenn Linderman]

Glenn Linderman  added the comment:

Martin, you are splitting hairs about the "reported problem".  The original 
message does have a paragraph about the executable bits being wrong.  But the 
bulk of the message is commenting about the difficulty of figuring out what to 
replace it with.

So it looks like in spite of the hair splitting, we have iterated to a design 
of making run_cgi a bit friendlier in this regard.

I find it sufficient to define a method fully extracted from run_cgi as follows:

def make_cmdline( self, scriptfile, query ):
cmdline = [scriptfile]
if self.is_python(scriptfile):
interp = sys.executable
if interp.lower().endswith("w.exe"):
# On Windows, use python.exe, not pythonw.exe
interp = interp[:-5] + interp[-4:]
cmdline = [interp, '-u'] + cmdline
if '=' not in query:
cmdline.append(query)

This leaves run_cgi with:

import subprocess
cmdline = self.make_cmdline( scriptfile, query )
self.log_message("command: %s", subprocess.list2cmdline(cmdline))


Apologies: I don't know what format of patch is acceptable, but this is a 
simple cut-n-paste change.  I was sort of holding off until the hg conversion 
to figure out how to make code submissions, since otherwise I'd have to learn 
it twice in short order.

I have reimplemented my work-arounds in terms of the above fix, and they 
function correctly, so this fix would suffice for me, for this issue.  (N.B. 
I'm sure you've noticed that I have entered a number of issues for http.server; 
I hope that was the right way to do it, to attempt to separate the issues.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

[Senthil Kumaran]

Changes by Senthil Kumaran :


--
nosy: +orsenthil

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

[Martin v . Löwis]

Martin v. Löwis  added the comment:

> Your suggestion of making CGIHTTPRequestHandler easier to subclass is
> certainly a good one, and is almost imperative to implement to fix
> this bug in a useful manner without implementing an insufficient set
> of Windows extensions (for someone's definition of wrong).

It's indeed the approach I would prefer over the alternatives you
suggested - I particularly dislike Python implementing a strategy
where #! files become considered on Windows (you then immediately
run into subsequent problems, such as /usr/bin/perl being no valid
filename on most Windows installations).

So I maintain that technically, in order to resolve the *reported*
issue (msg121875), it is sufficient to define that executables
on Windows are the files ending with .exe. To recall, the reported
issue is "is simply wrong ... is not clear what to use instead"
(to you as the reporter). My job as a maintainer is to resolve this,
and I will decide to resolve this as suggested. Even the refactoring
to allow substitution of process creation is an independent feature,
but I'm willing to accept patches.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

[Glenn Linderman]

Glenn Linderman  added the comment:

The rest of the code has clearly never had its deficiencies exposed on Windows, 
simply because executable() has prevented that.  So what the rest of the code 
"already supports" is basically nothing.  Reasonable Windows support is 
appropriate to implement as part of the bugfix.

You state that it isn't the function of http.server to extend Windows, however, 
even MS IIS has extended Windows to provide reasonable web scripting 
functionality, albeit it its own way, thus convicting the Windows facilities of 
being insufficient.  Attempting to use http.server to get a web testing 
environment running so that Python scripts can be tested locally requires some 
way of using an existing environment (except, of course, for "all new" web 
sites).  I suppose you would claim that using http.server for a web testing 
environment is an inappropriate use of http.server, also.  

Yet http.server on Unix appears to provide an adequate web testing environment: 
yes, some of that is because of Unix's #! feature.  This would certainly not be 
the first case where more code is required on Windows than Unix to implement 
reasonable functionality.

My desire for support for Perl is not an attempt to convince Python developers 
to use Perl instead of Python, but simply a reflection of the practicality of 
life: There are a lot of Perl CGI scripts used for pieces of Web servers.  
Reinventing them in Python may be fun, but can be more time consuming than 
projects may have the luxury to do.

Your claim that it already supports Python CGI scripts must be tempered by the 
documentation claim that it provides "altered semantics".  "altered semantics", 
as best as I can read in the code, is that the query string is passed to the 
Python script as a command line if it doesn't happen to contain an "=" sign.  
This is weird, unlikely to be found in a real web server, and hence somewhat 
useless for use as a test server also.

http.server has chosen to use subprocess which has chosen to use CreateProcess 
as its way of executing CGI.  There are other Windows facilities for executing 
programs, such as ShellExecute, but of course it takes the opposite tack: it 
can "execute" nearly any file, via registry-based associations.  Neither of 
these seem to be directly appropriate for use by http.server, the former being 
too restrictive without enhancements, the latter being too liberal in executing 
too many file types, although the requirement that CGI scripts live in specific 
directories may sufficiently rein in that liberality.

However, you have made me think through the process: it seems that an 
appropriate technique for Windows is to allow for a specific set of file 
extensions, and permit them to be executed using the registry-based association 
to do so.  However, for .cgi files, which depend heavily on the Unix #!, 
emulation of #! seems appropriate (and Windows doesn't seem to have an 
association for .cgi files either).

Your suggestion of making CGIHTTPRequestHandler easier to subclass is certainly 
a good one, and is almost imperative to implement to fix this bug in a useful 
manner without implementing an insufficient set of Windows extensions (for 
someone's definition of wrong).  There should be a way to sidestep the "altered 
semantics" for Python scripts (and Python scripts shouldn't have to be a 
special case, they should work with the general case), without replacing the 
whole run_cgi() function.  There should be a hook to define the list of 
executable extensions, and how to run them, and/or a hook to alter the command 
line passed to subprocess.Popen to achieve same.

So is_executable and is_python both seem to currently be replacable.  What is 
missing is a hook to implement cmdline creation before calling 
subprocess.Popen()  (besides the other reported bugs, of course)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

[Martin v . Löwis]

Martin v. Löwis  added the comment:

> Martin, that is an interesting viewpoint, and one I considered, but
> didn't state, because it seems much too restrictive.  Most CGI
> programs are written in scripting languages, not compiled to .exe.
> So it seems the solution should allow for launching at least Perl and
> Python scripts, as well as .exe.

Notice that it does support launching Python scripts. I disagree that
Perl scripts need to be supported. The idea of CGI is really that
"programs" get run by the web server, with the notion of "programs"
clearly deviating from system to system. Window really doesn't support
"scripts" (in the hash-bang sense), and it isn't the function of
http.server to extend Windows here. At best, support for .bat files
might be negotiable, using cmd.exe to launch them (but I personally
would not).

Anybody who wants support for other kinds of scripts on Windows will
have to subclass CGIHTTPRequestHandler (and it might be useful to
simplify subclassing that class).

In any case, the bug as stated ("def executable is simply wrong")
has a clear resolution - make it match what the rest of the code
supports. Anything beyond that is a feature request.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

[Glenn Linderman]

Glenn Linderman  added the comment:

Martin, that is an interesting viewpoint, and one I considered, but didn't 
state, because it seems much too restrictive.  Most CGI programs are written in 
scripting languages, not compiled to .exe.  So it seems the solution should 
allow for launching at least Perl and Python scripts, as well as .exe.  Whether 
subprocess.Popen can directly execute it, or whether it needs help from the 
registry or a #! line to get the execution going is just a matter of tweaking 
the coding for what gets passed to subprocess.Popen.  Declaring the definition 
based on what the existing code can already do is self-limiting.

Another possible interpretation of executable might be PATHEXT environment 
variable, but that is similar to declaring a list in the server source, which I 
did mention.  One might augment the other.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

[Martin v . Löwis]

Martin v. Löwis  added the comment:

I think it's fairly clear what should be considered executable in this context: 
anything that subprocess.Popen is able to launch (because if is_executable 
succeeds, this is what will happen to the file). This excludes word files, and 
also excludes files ending in .cgi; only files ending in .exe are executable.

--
nosy: +loewis

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10483] http.server - what is executable on Windows

[Glenn Linderman]

New submission from Glenn Linderman :

The def executable for CGIHTTPRequestHandler is simply wrong on Windows.  The 
Unix executable bits do not apply.

Yet it is not clear what to use instead.  One could check the extension against 
PATHEXT, perhaps, but Windows doesn't limit itself to that except when not 
finding the exactly specified executable name.  Or one could require and borrow 
the unix #! convention.  As an experiment, since I'm mostly dealing the script 
files, I tried out a hack that implements two #! lines, the first for Unix and 
the second for Windows, and only consider something executable if the second 
line exists.  This fails miserably for .exe files, of course.

Another possibility would be to see if there is an association for the 
extension, but that rule would permit a Word document to be "executable" 
because there is a way to open it using MS Word.

Another possibility would be to declare a list of extensions in the server 
source, like the list of directories from which CGIs are found.

Another possibility would be to simply assume that anything found in the CGI 
directory is executable.

Another possibility is to require the .cgi extension only to be executable, but 
then it is hard to know how to run it.

Another possibility is to require two "extensions"... the "real" one for 
Windows, and then .cgi just before it.  So to make a program executable, it 
would be renamed from file.ext to file.cgi.ext

But the current technique is clearly insufficient.

--
components: Library (Lib)
messages: 121875
nosy: v+python
priority: normal
severity: normal
status: open
title: http.server - what is executable on Windows
type: behavior
versions: Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com