[issue34980] KillPython target doesn't detect 64-bit processes

[Jeremy Kloth]

Jeremy Kloth  added the comment:

The changed succeeded in killing the actively stuck process, so I say its all 
good!  Thanks for the merge.

--
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34980] KillPython target doesn't detect 64-bit processes

[Steve Dower]

Steve Dower  added the comment:

Jeremy - feel free to close this when you're happy with the buildbot results.

--
stage: patch review -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34980] KillPython target doesn't detect 64-bit processes

[miss-islington]

miss-islington  added the comment:


New changeset 69d0372fc9c5a600ecdfb7dd80f852b26c6ed087 by Miss Islington (bot) 
in branch '3.6':
bpo-34980: P/Invoke QueryFullProcessImageName to get process names (GH-9901)
https://github.com/python/cpython/commit/69d0372fc9c5a600ecdfb7dd80f852b26c6ed087


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34980] KillPython target doesn't detect 64-bit processes

[miss-islington]

miss-islington  added the comment:


New changeset 7a253dcd97fa669b8615476b287ef4dd0a935014 by Miss Islington (bot) 
in branch '3.7':
bpo-34980: P/Invoke QueryFullProcessImageName to get process names (GH-9901)
https://github.com/python/cpython/commit/7a253dcd97fa669b8615476b287ef4dd0a935014


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34980] KillPython target doesn't detect 64-bit processes

[miss-islington]

Change by miss-islington :


--
pull_requests: +9392

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34980] KillPython target doesn't detect 64-bit processes

[miss-islington]

Change by miss-islington :


--
pull_requests: +9393

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34980] KillPython target doesn't detect 64-bit processes

[Steve Dower]

Steve Dower  added the comment:


New changeset fa5329424f4206630c34f75629fa78738db647f0 by Steve Dower (Jeremy 
Kloth) in branch 'master':
bpo-34980: P/Invoke QueryFullProcessImageName to get process names (GH-9901)
https://github.com/python/cpython/commit/fa5329424f4206630c34f75629fa78738db647f0


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34980] KillPython target doesn't detect 64-bit processes

[Steve Dower]

Steve Dower  added the comment:

Sorry for missing the PR. One little tweak for consistency, but otherwise it 
looks fine.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34980] KillPython target doesn't detect 64-bit processes

[Jeremy Kloth]

Jeremy Kloth  added the comment:

It seems my buildbot has a stuck process again.

The "sticking" occurred in this case due to test_concurrent_futures being hung 
(for over 38hrs! which is a different issue) and a DSL link reset at the same 
time.

So now, all builds on the master branch fail because the compile step fails due 
to process in use error.

I've submitted a PR implementing Eryk Sun's idea.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34980] KillPython target doesn't detect 64-bit processes

[Jeremy Kloth]

Change by Jeremy Kloth :


--
keywords: +patch
pull_requests: +9263
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34980] KillPython target doesn't detect 64-bit processes

[Eryk Sun]

Eryk Sun  added the comment:

> there shouldn't be any problem with 33-bit/64-bit here. Windows 
> doesn't separate processes like that.

Accessing the MainModule property of a .NET Process object raises an exception 
if the current process is 32-bit and the process is 64-bit [1]. This is because 
a 32-bit process can't sensibly read the PEB data of a 64-bit process, such as 
the loader's module data. 

It seems we have to P/Invoke QueryFullProcessImageName (which makes an 
NtQueryInformationProcess system call instead of reading data directly). We 
already have p.Handle, so the process doesn't have to be opened. Here's an 
example in 32-bit posh (i.e. 
%SystemRoot%\SysWOW64\WindowsPowerShell\v1.0\powershell.exe):

$MethodDefinition = @"
[DllImport("kernel32.dll", SetLastError=true, CharSet=CharSet.Unicode)]
public static extern bool QueryFullProcessImageName(
[In]IntPtr hProcess, [In]int dwFlags, [Out]StringBuilder lpExeName, 
ref int lpdwSize);
"@

$Kernel32 = Add-Type -Using "System.Text" -MemberDefinition `
$MethodDefinition -Name "Kernel32" -Namespace "Win32" -PassThru

$p = [System.Diagnostics.Process]::GetProcessesByName("explorer")[0]
$size = 32768
$name = [System.Text.StringBuilder]($size)

32-bit posh can't directly read the executable name:

PS C:\> $p.MainModule.FileName -eq $null
True

But QueryFullProcessImageName works fine:

PS C:\> $Kernel32::QueryFullProcessImageName($p.Handle, 0, $name, 
[ref]$size)
True
PS C:\> $name.ToString()
C:\Windows\explorer.exe

[1]: https://docs.microsoft.com/en-us/dotnet/api/system.diagnostics.process

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34980] KillPython target doesn't detect 64-bit processes

[Jeremy Kloth]

Jeremy Kloth  added the comment:

Alternatively, to test for yourself:

1) build a 64-bit python:

   > build -e -d -k -v -p x64

2) start the newly built interpreter:

   > amd64\python_d.exe

3) in a different command prompt (using dummy target to just do the KillPython)

   > build -e -d -k -v -p x64 -t foo

The expected result should be that the Python interpreter in the first window 
would by closed, but does not with the 32-bit MSBuild.  It does close however 
with the 64-bit MSBuild.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34980] KillPython target doesn't detect 64-bit processes

[Jeremy Kloth]

Jeremy Kloth  added the comment:

My testing shows differently:

D:\Public\Devel\cpython\master\PCbuild>set MSBUILD="C:\Program Files 
(x86)\MSBuild\14.0\Bin\amd64\MSBuild.exe"

D:\Public\Devel\cpython\master\PCbuild>build -k -v -t foo
Using py -3.6 (found 3.6 with py.exe)
Fetching external libraries...
bzip2-1.0.6 already exists, skipping.
sqlite-3.21.0.0 already exists, skipping.
xz-5.2.2 already exists, skipping.
zlib-1.2.11 already exists, skipping.
Fetching external binaries...
openssl-bin-1.1.0i already exists, skipping.
tcltk-8.6.8.0 already exists, skipping.
Finished.
Using "C:\Program Files (x86)\MSBuild\14.0\Bin\amd64\MSBuild.exe" (found in the 
environment)

D:\Public\Devel\cpython\master\PCbuild>"C:\Program Files 
(x86)\MSBuild\14.0\Bin\amd64\MSBuild.exe" 
"D:\Public\Devel\cpython\master\PCbuild\\pythoncore.vcxproj" /t:KillPython /v:n 
/p:Configuration=Release /p:Platform=Win32 /p:KillPython=true
Microsoft (R) Build Engine version 14.0.25420.1
Copyright (C) Microsoft Corporation. All rights reserved.

Build started 10/15/2018 10:05:36 AM.
Project "D:\Public\Devel\cpython\master\PCbuild\pythoncore.vcxproj" on node 1 
(KillPython target(s)).
KillPython:
  Killing any running python.exe instances...
  Looking for D:\Public\Devel\cpython\master\PCbuild\win32\python.exe
  Found running process: C:\Program Files 
(x86)\Google\Chrome\Application\chrome.exe
  Found running process: C:\Program Files\Common 
Files\LogiShrd\sp6\LU1\LogitechUpdate.exe
  Found running process: C:\Program Files (x86)\Microsoft Visual 
Studio\Preview\Community\Common7\ServiceHub\Hosts\Serv
  iceHub.Host.Node.x86\ServiceHub.Host.Node.x86.exe
  Found running process: C:\Windows\System32\MicrosoftEdgeCP.exe
  Found running process: C:\Windows\System32\MicrosoftEdgeCP.exe
  Found running process: C:\Program Files 
(x86)\Google\Chrome\Application\chrome.exe
  Found running process: 
C:\Windows\SystemApps\Microsoft.Windows.Cortana_cw5n1h2txyewy\SearchUI.exe
  Found running process: C:\Program Files 
(x86)\Google\Chrome\Application\chrome.exe
  Found running process: C:\Program Files\TortoiseHg\TortoiseHgOverlayServer.exe
  Found running process: C:\Program Files 
(x86)\Google\Chrome\Application\chrome.exe
  Found running process: C:\Program Files (x86)\Common 
Files\Acronis\Schedule2\schedhlp.exe
  Found running process: C:\Program Files 
(x86)\Google\Chrome\Application\chrome.exe
  Found running process: C:\Program Files 
(x86)\Google\Chrome\Application\chrome.exe
  Found running process: C:\Program Files (x86)\Origin\QtWebEngineProcess.exe
  Found running process: C:\Program Files 
(x86)\Google\Chrome\Application\chrome.exe
  Found running process: C:\Program Files (x86)\Microsoft Visual 
Studio\Preview\Community\Common7\ServiceHub\Hosts\Serv
  iceHub.Host.CLR.x86\ServiceHub.SettingsHost.exe
  Found running process: C:\WINDOWS\system32\browser_broker.exe
  Found running process: C:\Program Files 
(x86)\Google\Chrome\Application\chrome.exe
  Found running process: C:\Program Files 
(x86)\Google\Chrome\Application\chrome.exe
  Found running process: C:\Program Files 
(x86)\Google\Chrome\Application\chrome.exe
  Found running process: C:\Program Files 
(x86)\Google\Chrome\Application\chrome.exe
  Found running process: C:\Windows\System32\GameBarPresenceWriter.exe
  Found running process: C:\Program Files 
(x86)\Google\Chrome\Application\chrome.exe
  Found running process: C:\Program 
Files\WindowsApps\Microsoft.Windows.Photos_2018.18091.13420.0_x64__8wekyb3d8bbwe\Mi
  crosoft.Photos.exe
  Found running process: C:\Windows\System32\MicrosoftEdgeCP.exe
  Found running process: C:\Program Files 
(x86)\Steam\bin\cef\cef.win7x64\steamwebhelper.exe
  Found running process: C:\Program Files 
(x86)\Google\Chrome\Application\chrome.exe
  Found running process: C:\Program Files 
(x86)\Google\Chrome\Application\chrome.exe
  Found running process: C:\Program Files 
(x86)\Google\Chrome\Application\chrome.exe
  Found running process: C:\WINDOWS\system32\conhost.exe
  Found running process: C:\Program Files 
(x86)\Google\Chrome\Application\chrome.exe
  Found running process: C:\Program Files 
(x86)\Google\Chrome\Application\chrome.exe
  Found running process: C:\Program Files 
(x86)\Steam\bin\cef\cef.win7x64\steamwebhelper.exe
  Found running process: C:\Program Files 
(x86)\Google\Chrome\Application\chrome.exe
  Found running process: C:\Program Files 
(x86)\Google\Chrome\Application\chrome.exe
  Found running process: C:\Program Files 
(x86)\Steam\bin\cef\cef.win7x64\steamwebhelper.exe
  Found running process: C:\Program Files 
(x86)\Google\Chrome\Application\chrome.exe
  Found running process: C:\Program Files 
(x86)\Google\Chrome\Application\chrome.exe
  Found running process: C:\Windows\System32\RuntimeBroker.exe
  Found running process: C:\Program Files 
(x86)\Google\Chrome\Application\chrome.exe
  Found running process: C:\Windows\System32\RuntimeBroker.exe
  Found running process: C:\Program Files (x86)\Microsoft Visual 
Studio\Preview\

[issue34980] KillPython target doesn't detect 64-bit processes

[Steve Dower]

Steve Dower  added the comment:

I haven't looked at the logs, but there shouldn't be any problem with 
33-bit/64-bit here. Windows doesn't separate processes like that.

Perhaps we have builds with slightly different names (casing?) that are failing 
the comparison?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34980] KillPython target doesn't detect 64-bit processes

[Jeremy Kloth]

New submission from Jeremy Kloth :

Since the KillPython target has been rewritten as an InlineTask, it can no 
longer detect 64-bit processes due to MSBuild being 32-bit.

This leads to stuck buildbot runs:
  https://buildbot.python.org/all/#/builders/17/builds/348

A few solutions that I can think of:

1) Switch the InlineTask to an Exec using a PowerShell script to kill the 
processes.
   In my limited testing, PowerShell is installed by default on the supported 
OSes (Win7+).

   

2) When building for platform x64, modify find_msbuild.bat to locate the 64-bit 
MSBuild executable

3) On 64-bit OS, always use the 64-bit MSBuild executable

4) Only use the 64-bit MSBuild (when available) for the KillPython target (the 
64-bit KillPython target can detect 32-bit processes)

I am unsure of any issues that may arise from building with the 64-bit MSBuild 
toolchain.

--
components: Build, Windows
messages: 327694
nosy: jkloth, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: KillPython target doesn't detect 64-bit processes
versions: Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com