Re: unable to run the basic Embedded Python example
Hi Dave, I can tell you where the error comes from, but I don't know how to fix it correctly: Am 24.06.23 um 19:29 schrieb Dave Ohlsson: 9. And now, when I ran embedded_python.exe: 20:14:06: Starting C:\temp\build-embedded_python-Desktop_Qt_6_1_3_MSVC2019_64bit-Debug\debug\embedded_python.exe... Could not find platform independent libraries Python path configuration: PYTHONHOME = (not set) PYTHONPATH = (not set) program name = Python consists of the DLL that you have lined it with plus a large number of files (standard library) which contain, e.g., the encoding data and builtin functions etc. It needs to find these in order to run. You could set PYTHONHOME as an environment variable to point to that folder. If you don't use the regular Pyton installer, then you need to install these files yourself. Maybe there is also a way to set it from the C code. Christian -- https://mail.python.org/mailman/listinfo/python-list
Re: unable to run the basic Embedded Python example
Forgot to try it or not in an Apple computer, but there’s one or two apple
computers in my home when I just began to learn computer science. I should try
it. Because I remembered clearly, I tried it on others.
Get Outlook for iOS<https://aka.ms/o0ukef>
From: Fulian Wang
Sent: Saturday, June 24, 2023 7:43:56 PM
To: Dave Ohlsson ; python-list@python.org
Subject: Re: unable to run the basic Embedded Python example
I study computer science in byu.edu, we choose the 3.4.3 vision. No matter
which kind of computers or windows, I needed to download it from Microsoft
store. Now I can download it directly from python.org
Get Outlook for iOS<https://aka.ms/o0ukef>
From: Python-list on
behalf of Dave Ohlsson via Python-list
Sent: Saturday, June 24, 2023 15:35
To: python-list@python.org
Subject: unable to run the basic Embedded Python example
I cannot get the simple program
https://docs.python.org/3/extending/embedding.html#very-high-level-embedding
to work.
My environment:
* Windows 10
* Microsoft Visual Studio 2019
* Qt Creator 10.0.1 (only as an IDE)
* Python not installed
This is what I did:
1. I downloaded https://www.python.org/ftp/python/3.11.4/Python-3.11.4.tgz
2. I extracted Python-3.11.4.tgz into C:\temp\Python-3.11.4
3. I opened the "x64 Native Tools Command Prompt for VS 2019" which comes
with Microsoft Visual Studio 2019
4. In "x64 Native Tools Command Prompt for VS 2019":
**
** Visual Studio 2019 Developer Command Prompt v16.11.26
** Copyright (c) 2021 Microsoft Corporation
**
[vcvarsall.bat] Environment initialized for: 'x64'
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community>cd
C:\temp\Python-3.11.4\PCbuild
C:\temp\Python-3.11.4\PCbuild>build.bat -d // <= option '-d' means "debug"
Downloading nuget...
[many lines omitted]
Build succeeded.
C:\temp\Python-3.11.4\PCbuild\pythoncore.vcxproj(578,5): warning MSB3073:
The command ""C:\Program Files\Git\cmd\git.exe" name-rev --name-only HEAD >
"C:\temp\Python-3.11.4\PCbuild\obj\311amd64_Debug\pythoncore\gitbranch.txt""
exited with code 128.
C:\temp\Python-3.11.4\PCbuild\pythoncore.vcxproj(578,5): warning MSB4181:
The "Exec" task returned false but did not log an error.
C:\temp\Python-3.11.4\PCbuild\pythoncore.vcxproj(579,5): warning MSB3073:
The command ""C:\Program Files\Git\cmd\git.exe" rev-parse --short HEAD >
"C:\temp\Python-3.11.4\PCbuild\obj\311amd64_Debug\pythoncore\gitversion.txt""
exited with code 128.
C:\temp\Python-3.11.4\PCbuild\pythoncore.vcxproj(579,5): warning MSB4181:
The "Exec" task returned false but did not log an error.
C:\temp\Python-3.11.4\PCbuild\pythoncore.vcxproj(580,5): warning MSB3073:
The command ""C:\Program Files\Git\cmd\git.exe" describe --all --always
--dirty >
"C:\temp\Python-3.11.4\PCbuild\obj\311amd64_Debug\pythoncore\gittag.txt""
exited with code 128.
C:\temp\Python-3.11.4\PCbuild\pythoncore.vcxproj(580,5): warning MSB4181:
The "Exec" task returned false but did not log an error.
6 Warning(s)
0 Error(s)
Time Elapsed 00:00:34.87
C:\temp\Python-3.11.4\PCbuild>
5. I copied the example
https://docs.python.org/3/extending/embedding.html#very-high-level-embedding,
unchanged, to 'C:\temp\embedded_python\main.c'.
6. I created Qt Creator project file 'C:\temp\embedded_python\
embedded_python.pro':
TEMPLATE = app
TARGET = embedded_python
SOURCES += main.c
PYTHON_DIR = C:/temp/Python-3.11.4
INCLUDEPATH += $${PYTHON_DIR}/Include
INCLUDEPATH += $${PYTHON_DIR}/PC
LIBS += $${PYTHON_DIR}/PCbuild/amd64/python311_d.lib
7. I built the project:
20:02:38: Running steps for project embedded_python...
20:02:38: Starting: "C:\Qt\6.1.3\msvc2019_64\bin\qmake.exe"
C:\temp\embedded_python\embedded_python.pro -spec win32-msvc
"CONFIG+=debug" "CONFIG-=qml_debug" "CONFIG-=qtquickcompiler"
Info: creating stash file
C:\temp\build-embedded_python-Desktop_Qt_6_1_3_MSVC2019_64bit-Debug\.qmake.stash
20:02:38: The process "C:\Qt\6.1.3\msvc2019_64\bin\qmake.exe" exited
normally.
20:02:38: Starting: "C:\Qt\Tools\QtCreator\bin\jom\jom.exe" -f
C:/temp/build-embedded_python-Desktop_Qt_6_1_3_MSVC2019_64bit-Debug/Makefile
qmake_all
jom 1.1.3 - empower your cores
20:02:38: The process "C:\Qt\Tools\QtCreator\bin\jom\jom.exe" exited
normally.
20:02:38: Starting: "C:\Qt\Tools\QtCreator\bin\jom\jom.exe"
C:\Qt\Tools\QtCreator\bin\jom\jom.exe -f Makefile.Debug
cl -c -nologo -Zc:wchar_t -FS -Zc:strictStrings -Zi -MDd -utf-8 -W3 -w44456
-w44457 -w44458 /Fddebug\embedded_python.vc.pdb -DUNICODE -D_UNICODE
-DWIN32 -D_ENABLE_EXTEND
Re: unable to run the basic Embedded Python example
I study computer science in byu.edu, we choose the 3.4.3 vision. No matter
which kind of computers or windows, I needed to download it from Microsoft
store. Now I can download it directly from python.org
Get Outlook for iOS<https://aka.ms/o0ukef>
From: Python-list on
behalf of Dave Ohlsson via Python-list
Sent: Saturday, June 24, 2023 15:35
To: python-list@python.org
Subject: unable to run the basic Embedded Python example
I cannot get the simple program
https://docs.python.org/3/extending/embedding.html#very-high-level-embedding
to work.
My environment:
* Windows 10
* Microsoft Visual Studio 2019
* Qt Creator 10.0.1 (only as an IDE)
* Python not installed
This is what I did:
1. I downloaded https://www.python.org/ftp/python/3.11.4/Python-3.11.4.tgz
2. I extracted Python-3.11.4.tgz into C:\temp\Python-3.11.4
3. I opened the "x64 Native Tools Command Prompt for VS 2019" which comes
with Microsoft Visual Studio 2019
4. In "x64 Native Tools Command Prompt for VS 2019":
**
** Visual Studio 2019 Developer Command Prompt v16.11.26
** Copyright (c) 2021 Microsoft Corporation
**
[vcvarsall.bat] Environment initialized for: 'x64'
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community>cd
C:\temp\Python-3.11.4\PCbuild
C:\temp\Python-3.11.4\PCbuild>build.bat -d // <= option '-d' means "debug"
Downloading nuget...
[many lines omitted]
Build succeeded.
C:\temp\Python-3.11.4\PCbuild\pythoncore.vcxproj(578,5): warning MSB3073:
The command ""C:\Program Files\Git\cmd\git.exe" name-rev --name-only HEAD >
"C:\temp\Python-3.11.4\PCbuild\obj\311amd64_Debug\pythoncore\gitbranch.txt""
exited with code 128.
C:\temp\Python-3.11.4\PCbuild\pythoncore.vcxproj(578,5): warning MSB4181:
The "Exec" task returned false but did not log an error.
C:\temp\Python-3.11.4\PCbuild\pythoncore.vcxproj(579,5): warning MSB3073:
The command ""C:\Program Files\Git\cmd\git.exe" rev-parse --short HEAD >
"C:\temp\Python-3.11.4\PCbuild\obj\311amd64_Debug\pythoncore\gitversion.txt""
exited with code 128.
C:\temp\Python-3.11.4\PCbuild\pythoncore.vcxproj(579,5): warning MSB4181:
The "Exec" task returned false but did not log an error.
C:\temp\Python-3.11.4\PCbuild\pythoncore.vcxproj(580,5): warning MSB3073:
The command ""C:\Program Files\Git\cmd\git.exe" describe --all --always
--dirty >
"C:\temp\Python-3.11.4\PCbuild\obj\311amd64_Debug\pythoncore\gittag.txt""
exited with code 128.
C:\temp\Python-3.11.4\PCbuild\pythoncore.vcxproj(580,5): warning MSB4181:
The "Exec" task returned false but did not log an error.
6 Warning(s)
0 Error(s)
Time Elapsed 00:00:34.87
C:\temp\Python-3.11.4\PCbuild>
5. I copied the example
https://docs.python.org/3/extending/embedding.html#very-high-level-embedding,
unchanged, to 'C:\temp\embedded_python\main.c'.
6. I created Qt Creator project file 'C:\temp\embedded_python\
embedded_python.pro':
TEMPLATE = app
TARGET = embedded_python
SOURCES += main.c
PYTHON_DIR = C:/temp/Python-3.11.4
INCLUDEPATH += $${PYTHON_DIR}/Include
INCLUDEPATH += $${PYTHON_DIR}/PC
LIBS += $${PYTHON_DIR}/PCbuild/amd64/python311_d.lib
7. I built the project:
20:02:38: Running steps for project embedded_python...
20:02:38: Starting: "C:\Qt\6.1.3\msvc2019_64\bin\qmake.exe"
C:\temp\embedded_python\embedded_python.pro -spec win32-msvc
"CONFIG+=debug" "CONFIG-=qml_debug" "CONFIG-=qtquickcompiler"
Info: creating stash file
C:\temp\build-embedded_python-Desktop_Qt_6_1_3_MSVC2019_64bit-Debug\.qmake.stash
20:02:38: The process "C:\Qt\6.1.3\msvc2019_64\bin\qmake.exe" exited
normally.
20:02:38: Starting: "C:\Qt\Tools\QtCreator\bin\jom\jom.exe" -f
C:/temp/build-embedded_python-Desktop_Qt_6_1_3_MSVC2019_64bit-Debug/Makefile
qmake_all
jom 1.1.3 - empower your cores
20:02:38: The process "C:\Qt\Tools\QtCreator\bin\jom\jom.exe" exited
normally.
20:02:38: Starting: "C:\Qt\Tools\QtCreator\bin\jom\jom.exe"
C:\Qt\Tools\QtCreator\bin\jom\jom.exe -f Makefile.Debug
cl -c -nologo -Zc:wchar_t -FS -Zc:strictStrings -Zi -MDd -utf-8 -W3 -w44456
-w44457 -w44458 /Fddebug\embedded_python.vc.pdb -DUNICODE -D_UNICODE
-DWIN32 -D_ENABLE_EXTENDED_ALIGNED_STORAGE -DWIN64 -DQT_GUI_LIB
-DQT_CORE_LIB -I..\embedded_python -I. -I..\Python-3.11.4\Include
-I..\Python-3.11.4\PC -I..\..\Qt\6.1.3\msvc2019_64\include
-I..\..\Qt\6.1.3\msvc2019_64\include\QtGui
-I..\..\Qt\6.1.3\msvc2019_64\include\QtCore -Idebug -IC:/VulkanSDK/
1.0.51.0/Include -I..\..\Qt\6.1.3\msvc2019_64\mkspecs\win32-msvc -Fodebug\
@C:\Users\dave\AppData\Local\Temp\main.obj.21168.0.jom
main.c
..\embedded_python\main.c(12): warning C4996
unable to run the basic Embedded Python example
I cannot get the simple program
https://docs.python.org/3/extending/embedding.html#very-high-level-embedding
to work.
My environment:
* Windows 10
* Microsoft Visual Studio 2019
* Qt Creator 10.0.1 (only as an IDE)
* Python not installed
This is what I did:
1. I downloaded https://www.python.org/ftp/python/3.11.4/Python-3.11.4.tgz
2. I extracted Python-3.11.4.tgz into C:\temp\Python-3.11.4
3. I opened the "x64 Native Tools Command Prompt for VS 2019" which comes
with Microsoft Visual Studio 2019
4. In "x64 Native Tools Command Prompt for VS 2019":
**
** Visual Studio 2019 Developer Command Prompt v16.11.26
** Copyright (c) 2021 Microsoft Corporation
**
[vcvarsall.bat] Environment initialized for: 'x64'
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community>cd
C:\temp\Python-3.11.4\PCbuild
C:\temp\Python-3.11.4\PCbuild>build.bat -d // <= option '-d' means "debug"
Downloading nuget...
[many lines omitted]
Build succeeded.
C:\temp\Python-3.11.4\PCbuild\pythoncore.vcxproj(578,5): warning MSB3073:
The command ""C:\Program Files\Git\cmd\git.exe" name-rev --name-only HEAD >
"C:\temp\Python-3.11.4\PCbuild\obj\311amd64_Debug\pythoncore\gitbranch.txt""
exited with code 128.
C:\temp\Python-3.11.4\PCbuild\pythoncore.vcxproj(578,5): warning MSB4181:
The "Exec" task returned false but did not log an error.
C:\temp\Python-3.11.4\PCbuild\pythoncore.vcxproj(579,5): warning MSB3073:
The command ""C:\Program Files\Git\cmd\git.exe" rev-parse --short HEAD >
"C:\temp\Python-3.11.4\PCbuild\obj\311amd64_Debug\pythoncore\gitversion.txt""
exited with code 128.
C:\temp\Python-3.11.4\PCbuild\pythoncore.vcxproj(579,5): warning MSB4181:
The "Exec" task returned false but did not log an error.
C:\temp\Python-3.11.4\PCbuild\pythoncore.vcxproj(580,5): warning MSB3073:
The command ""C:\Program Files\Git\cmd\git.exe" describe --all --always
--dirty >
"C:\temp\Python-3.11.4\PCbuild\obj\311amd64_Debug\pythoncore\gittag.txt""
exited with code 128.
C:\temp\Python-3.11.4\PCbuild\pythoncore.vcxproj(580,5): warning MSB4181:
The "Exec" task returned false but did not log an error.
6 Warning(s)
0 Error(s)
Time Elapsed 00:00:34.87
C:\temp\Python-3.11.4\PCbuild>
5. I copied the example
https://docs.python.org/3/extending/embedding.html#very-high-level-embedding,
unchanged, to 'C:\temp\embedded_python\main.c'.
6. I created Qt Creator project file 'C:\temp\embedded_python\
embedded_python.pro':
TEMPLATE = app
TARGET = embedded_python
SOURCES += main.c
PYTHON_DIR = C:/temp/Python-3.11.4
INCLUDEPATH += $${PYTHON_DIR}/Include
INCLUDEPATH += $${PYTHON_DIR}/PC
LIBS += $${PYTHON_DIR}/PCbuild/amd64/python311_d.lib
7. I built the project:
20:02:38: Running steps for project embedded_python...
20:02:38: Starting: "C:\Qt\6.1.3\msvc2019_64\bin\qmake.exe"
C:\temp\embedded_python\embedded_python.pro -spec win32-msvc
"CONFIG+=debug" "CONFIG-=qml_debug" "CONFIG-=qtquickcompiler"
Info: creating stash file
C:\temp\build-embedded_python-Desktop_Qt_6_1_3_MSVC2019_64bit-Debug\.qmake.stash
20:02:38: The process "C:\Qt\6.1.3\msvc2019_64\bin\qmake.exe" exited
normally.
20:02:38: Starting: "C:\Qt\Tools\QtCreator\bin\jom\jom.exe" -f
C:/temp/build-embedded_python-Desktop_Qt_6_1_3_MSVC2019_64bit-Debug/Makefile
qmake_all
jom 1.1.3 - empower your cores
20:02:38: The process "C:\Qt\Tools\QtCreator\bin\jom\jom.exe" exited
normally.
20:02:38: Starting: "C:\Qt\Tools\QtCreator\bin\jom\jom.exe"
C:\Qt\Tools\QtCreator\bin\jom\jom.exe -f Makefile.Debug
cl -c -nologo -Zc:wchar_t -FS -Zc:strictStrings -Zi -MDd -utf-8 -W3 -w44456
-w44457 -w44458 /Fddebug\embedded_python.vc.pdb -DUNICODE -D_UNICODE
-DWIN32 -D_ENABLE_EXTENDED_ALIGNED_STORAGE -DWIN64 -DQT_GUI_LIB
-DQT_CORE_LIB -I..\embedded_python -I. -I..\Python-3.11.4\Include
-I..\Python-3.11.4\PC -I..\..\Qt\6.1.3\msvc2019_64\include
-I..\..\Qt\6.1.3\msvc2019_64\include\QtGui
-I..\..\Qt\6.1.3\msvc2019_64\include\QtCore -Idebug -IC:/VulkanSDK/
1.0.51.0/Include -I..\..\Qt\6.1.3\msvc2019_64\mkspecs\win32-msvc -Fodebug\
@C:\Users\dave\AppData\Local\Temp\main.obj.21168.0.jom
main.c
..\embedded_python\main.c(12): warning C4996: 'Py_SetProgramName':
deprecated in 3.11
link /NOLOGO /DYNAMICBASE /NXCOMPAT /DEBUG /SUBSYSTEM:WINDOWS
"/MANIFESTDEPENDENCY:type='win32' name='Microsoft.Windows.Common-Controls'
version='6.0.0.0' publicKeyToken='6595b64144ccf1df' language='*'
processorArchitecture='*'" /MANIFEST:embed /OUT:debug\embedded_python.exe
@C:\Users\dave\AppData\Local\Temp\embedded_python.exe.21168.484.jom
20:02:39: The process "C:\Qt\Tools\QtCreator\bin\jom\jom.exe" exited
normally.
20:02:39: Elapsed time: 00:01.
8. So far, so good. Next, I tried to run embedded_python.exe. In order for
the Python DLL to be available:
8a. I closed Qt Creator.
8b. In the Control Panel, "System", "Advanced system settings", tab
"Advanced", "Environment Variables...", I added
'C:\temp\Python-3.11.4\PCbu
Re: Embedded python is not 100% stable
On 4/13/23 03:40, Guenther Sohler wrote: Attachments are stripped, so they weren't included. Glancing at the branch and the 2 lines you mentioned. You have a comment with a link for python 2.3 documentation. Yet you have python 3.10 code included elsewhere (and openscad itself requires the modern spec C++17, so I assume your aim is at current versions) https://docs.python.org/3.10/extending/newtypes.html is the equivalent doc for that. I am not experienced in defining PyType objects in C, but notice syntactical discrepancies, missing "static" on dealloc, use of typecasted malloc rather than PyType_GenericAlloc() https://docs.python.org/3.10/c-api/typeobj.html#c.PyTypeObject.tp_alloc Without digging in deeper and learning more I can't say for certain whether or not those are issues, but intuitively I assume they would be and that the implementation needs to more closely mirror the syntactical flow implementations of modern working examples.. -- https://mail.python.org/mailman/listinfo/python-list
Re: Embedded python is not 100% stable
Guenther Sohler wrote at 2023-4-13 09:40 +0200: > ... >I have been working on adding embedded python into OpenSCAD ( >www.openscad.org) >for some time already. For that i coded/added an additional Python Type >Object >which means to hold openscad geometric data. > >It works quite well but unfortunately its not 100% stable and i have been >heavily checking >all the functions which are referenced in the PyType Object and tried to >match >them with the documentation which i found in the python web site The Python C/C++ interface is complex: it is easy to make mistakes which may lead to crashes. Often, `cython` (--> PyPI) can help you to define extension types in a much safer way. Maybe, you check its documentation? -- https://mail.python.org/mailman/listinfo/python-list
Embedded python is not 100% stable
Hi Python LIst, I have been working on adding embedded python into OpenSCAD ( www.openscad.org) for some time already. For that i coded/added an additional Python Type Object which means to hold openscad geometric data. It works quite well but unfortunately its not 100% stable and i have been heavily checking all the functions which are referenced in the PyType Object and tried to match them with the documentation which i found in the python web site but now i am stuck and i cant progress any further I am specially interested why the program constantly crashes when i activate the lines 14 and 15 in pyopenscad.cc. I have attached the two most relevant files and I would be happy if anybody interested could tell me, if there is an obvious bug popping into your eyes. If anybody is interested to see the complete picture, all source code is available checked in in github @ https://github.com/gsohler/openscad.git branch "libfive" Thank you for reading. Any spotted bug found will make me happy -- https://mail.python.org/mailman/listinfo/python-list
Re: Embedded python: How to debug code in an isolated way
> On 24 Aug 2020, at 12:52, Eko palypse wrote: > > Thank you very much for your interest in my little problem. > >> When the app calls into python does the event loop of the gui block? > > Yes, the cpp app calls a callback function from the embedded python > interpreter synchronously. > >> When you print does that trigger the event loop to run in a nested call > back into the cpp? > > I can't say for sure but I don't think so. At least my script or the > embedded interpreter doesn't do that. > > >> Maybe you need to run python in its own thread and Marshall in and out > of the gui main thread with the event loop? > > I've checked by using GetCurrentThreadId > <https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-getcurrentthreadid> > that the registered callback function and debugger run in the same thread > but it is different to the UI thread. > Do you think I have to run the debugger in its own thread? I'll give it a > try anyways. :-) No I doubt that can will work. What I was thinking is that your life was made more complex by multi threads and GUI event loops. I think that you should not attempt to use a debugger at all. Add code to put the information you need to find the bug into a log file. You might want to build a feature into the GUI that will show the log file to you if it not easy to get at a log file on disk. As an aside I have never used a python debugger to fix python code. I have always relied on logging key information to allow me to fix bugs. I have used gdb to debug C/C++ and python. All the non-trivial projects I work on have multi stream logging builtin. I tend to design the logging with the aim of debugging and maintaining code from the outset these days. When a new bug turns up I just switch on the appropriate stream of logs to help me fix it, add new logs as necessary. Barry -- https://mail.python.org/mailman/listinfo/python-list
Re: Embedded python: How to debug code in an isolated way
On Tue, Aug 25, 2020 at 6:30 AM Schachner, Joseph wrote: > > Another suggestion: If your Python code only references few things outside > of itself, make a simulated environment in Python on your PC, so that you can > run your embedded code after importing your simulated environment, which > should supply the functions it expects to call and variables it expects to > access. > > Then you can use any PC based debugger (PyScripter, Jetbrains' PyCharm, > Visual Studio with Python support, etc) to debug in the simulated environment. > That seldom works very well. since most of the point of running code on embedded devices is to use the features of those devices. My brother built a network-attached doorbell on an RPi, and all the debugging work revolved around the exact electrical signals on the GPIO pins. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
RE: Embedded python: How to debug code in an isolated way
Another suggestion: If your Python code only references few things outside of itself, make a simulated environment in Python on your PC, so that you can run your embedded code after importing your simulated environment, which should supply the functions it expects to call and variables it expects to access. Then you can use any PC based debugger (PyScripter, Jetbrains' PyCharm, Visual Studio with Python support, etc) to debug in the simulated environment. --- Joseph S. -Original Message- From: Grant Edwards Sent: Sunday, August 23, 2020 12:59 PM To: python-list@python.org Subject: Re: Embedded python: How to debug code in an isolated way On 2020-08-22, Chris Angelico wrote: > On Sun, Aug 23, 2020 at 5:51 AM Eko palypse wrote: >> So the question is, what do I need to read/learn/understand in order to >> solve this issue? >> Or in other words, how can I debug my script in an isolated environment. > > I'd go for the old standby - IIDPIO: If In Doubt, Print It Out! > Instead of trying to use a debug harness, just run your code normally, > and print out whatever you think might be of interest. If you don't > have a console, well, that would be the first thing to do - you > *always* need a console. Yep. Even if you have to bit-bang a tx-only UART on a GPIO pin. I've had to do that many times, and the last time was only a couple years ago. Though I must admit I never had to do that _in_ Python or on a platform capable of running Python... -- Grant -- https://mail.python.org/mailman/listinfo/python-list
Re: Embedded python: How to debug code in an isolated way
Thank you very much for your interest in my little problem. > When the app calls into python does the event loop of the gui block? Yes, the cpp app calls a callback function from the embedded python interpreter synchronously. > When you print does that trigger the event loop to run in a nested call back into the cpp? I can't say for sure but I don't think so. At least my script or the embedded interpreter doesn't do that. > Maybe you need to run python in its own thread and Marshall in and out of the gui main thread with the event loop? I've checked by using GetCurrentThreadId <https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-getcurrentthreadid> that the registered callback function and debugger run in the same thread but it is different to the UI thread. Do you think I have to run the debugger in its own thread? I'll give it a try anyways. :-) Thx Eren Am So., 23. Aug. 2020 um 22:55 Uhr schrieb Barry : > > > > On 22 Aug 2020, at 20:53, Eko palypse wrote: > > > > Hello, > > > > background info first. On windows, python3.8.5 > > > > A cpp app has an embedded python interpreter which allows to > modify/enhance the cpp app > > by providing objects to manipulate the cpp app and callbacks to act on > certain events, > > like fileopen, fileclose, updateui ... which are send by the cpp app. > > The embedded python interpreter acts more or less like the standard > python REPL. > > If you run a script with code like > > > > def test(i): > >print(i*2) > > > > test(4) > > > > then the test function is now globally available as long as the cpp app > is alive. > > > > I've written a debugger based on bdb and encountered a situation which I > haven't found out how to handle correctly yet. > > > > Let's assume I've registered a callback when something changes in the UI. > > In that case an object function gets called which highlights certain > aspects in the UI. > > > > Now if I run a script via this debugger it works as long as I don't use > the same object function. > > If I do use it, then the stack gets corrupted(?). > > When the app calls into python does the event loop of the gui block? > When you print does that trigger the event loop to run in a nested call > back into the cpp? > > Maybe you need to run python in its own thread and Marshall in and out of > the gui main thread with the event loop? > > Barry > > > For example it look like this > > > > stack: > > , > 580 > > ', line 1, code >, 1 > > code >, 8 > > > > now if I enter the function which call this object function this gets add > > > > add_match>, 5 > > > > I step through the code and suddenly the stack looks like this > > > > , > 580 > > ', line 1, code >, 1 > > code >, 8 > > code >, 154 > > code paint_it>, 102)] > > > > EnhanceAnyLexer is neither part of the test_pydebugger script nor part > of the debugger. > > It is used to register a callback on the updateui event. > > > > So the question is, what do I need to read/learn/understand in order to > solve this issue? > > Or in other words, how can I debug my script in an isolated environment. > > I can't manipulate the __main__.dict as this seems to have general > impact on registered callbacks as well. > > Any ideas? > > > > Thank you > > Eren > > -- > > https://mail.python.org/mailman/listinfo/python-list > > > > -- https://mail.python.org/mailman/listinfo/python-list
Re: Embedded python: How to debug code in an isolated way
On 2020-08-23, Chris Angelico wrote: > On Mon, Aug 24, 2020 at 6:00 AM Grant Edwards > wrote: >> >> On 2020-08-22, Chris Angelico wrote: >> > On Sun, Aug 23, 2020 at 5:51 AM Eko palypse wrote: >> >> So the question is, what do I need to read/learn/understand in order to >> >> solve this issue? >> >> Or in other words, how can I debug my script in an isolated environment. >> > >> > I'd go for the old standby - IIDPIO: If In Doubt, Print It Out! >> > Instead of trying to use a debug harness, just run your code normally, >> > and print out whatever you think might be of interest. If you don't >> > have a console, well, that would be the first thing to do - you >> > *always* need a console. >> >> Yep. Even if you have to bit-bang a tx-only UART on a GPIO pin. >> >> I've had to do that many times, and the last time was only a couple >> years ago. Though I must admit I never had to do that _in_ Python or >> on a platform capable of running Python... > > Haha, yep. That's also why the first "hello world" on any embedded > system is also an important tool in itself - if you can make an LED > flicker on and off in response to your code, then you have the most > rudimentary of print functionality that you can use to debug > everything else... Yep, you can send morse code, or a simple "one flash when I get here", "two flashes when I get here", etc. Turn it on here off there, and now you can profile performance. -- Grant -- https://mail.python.org/mailman/listinfo/python-list
Re: Embedded python: How to debug code in an isolated way
> On 22 Aug 2020, at 20:53, Eko palypse wrote: > > Hello, > > background info first. On windows, python3.8.5 > > A cpp app has an embedded python interpreter which allows to modify/enhance > the cpp app > by providing objects to manipulate the cpp app and callbacks to act on > certain events, > like fileopen, fileclose, updateui ... which are send by the cpp app. > The embedded python interpreter acts more or less like the standard python > REPL. > If you run a script with code like > > def test(i): >print(i*2) > > test(4) > > then the test function is now globally available as long as the cpp app is > alive. > > I've written a debugger based on bdb and encountered a situation which I > haven't found out how to handle correctly yet. > > Let's assume I've registered a callback when something changes in the UI. > In that case an object function gets called which highlights certain aspects > in the UI. > > Now if I run a script via this debugger it works as long as I don't use the > same object function. > If I do use it, then the stack gets corrupted(?). When the app calls into python does the event loop of the gui block? When you print does that trigger the event loop to run in a nested call back into the cpp? Maybe you need to run python in its own thread and Marshall in and out of the gui main thread with the event loop? Barry > For example it look like this > > stack: > , 580 > ', line 1, code >, 1 > >, 8 > > now if I enter the function which call this object function this gets add > > add_match>, 5 > > I step through the code and suddenly the stack looks like this > > , 580 > ', line 1, code >, 1 > >, 8 > >, 154 > paint_it>, 102)] > > EnhanceAnyLexer is neither part of the test_pydebugger script nor part of the > debugger. > It is used to register a callback on the updateui event. > > So the question is, what do I need to read/learn/understand in order to solve > this issue? > Or in other words, how can I debug my script in an isolated environment. > I can't manipulate the __main__.dict as this seems to have general impact on > registered callbacks as well. > Any ideas? > > Thank you > Eren > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Embedded python: How to debug code in an isolated way
On Mon, Aug 24, 2020 at 6:00 AM Grant Edwards wrote: > > On 2020-08-22, Chris Angelico wrote: > > On Sun, Aug 23, 2020 at 5:51 AM Eko palypse wrote: > >> So the question is, what do I need to read/learn/understand in order to > >> solve this issue? > >> Or in other words, how can I debug my script in an isolated environment. > > > > I'd go for the old standby - IIDPIO: If In Doubt, Print It Out! > > Instead of trying to use a debug harness, just run your code normally, > > and print out whatever you think might be of interest. If you don't > > have a console, well, that would be the first thing to do - you > > *always* need a console. > > Yep. Even if you have to bit-bang a tx-only UART on a GPIO pin. > > I've had to do that many times, and the last time was only a couple > years ago. Though I must admit I never had to do that _in_ Python or > on a platform capable of running Python... > Haha, yep. That's also why the first "hello world" on any embedded system is also an important tool in itself - if you can make an LED flicker on and off in response to your code, then you have the most rudimentary of print functionality that you can use to debug everything else... ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Embedded python: How to debug code in an isolated way
On 2020-08-22, Chris Angelico wrote: > On Sun, Aug 23, 2020 at 5:51 AM Eko palypse wrote: >> So the question is, what do I need to read/learn/understand in order to >> solve this issue? >> Or in other words, how can I debug my script in an isolated environment. > > I'd go for the old standby - IIDPIO: If In Doubt, Print It Out! > Instead of trying to use a debug harness, just run your code normally, > and print out whatever you think might be of interest. If you don't > have a console, well, that would be the first thing to do - you > *always* need a console. Yep. Even if you have to bit-bang a tx-only UART on a GPIO pin. I've had to do that many times, and the last time was only a couple years ago. Though I must admit I never had to do that _in_ Python or on a platform capable of running Python... -- Grant -- https://mail.python.org/mailman/listinfo/python-list
Re: Embedded python: How to debug code in an isolated way
Thx for your tip/suggestion. > If In Doubt, Print It Out! That's the current situation and that's usually enough, but then there's this situation where it gets annoying because you realize that the print wouldn't make more sense at this point but at that point and that's where a debugger is just nicer, I think. Eren Am Sa., 22. Aug. 2020 um 22:59 Uhr schrieb Chris Angelico : > On Sun, Aug 23, 2020 at 5:51 AM Eko palypse wrote: > > So the question is, what do I need to read/learn/understand in order to > solve this issue? > > Or in other words, how can I debug my script in an isolated environment. > > I'd go for the old standby - IIDPIO: If In Doubt, Print It Out! > Instead of trying to use a debug harness, just run your code normally, > and print out whatever you think might be of interest. If you don't > have a console, well, that would be the first thing to do - you > *always* need a console. > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Embedded python: How to debug code in an isolated way
On Sun, Aug 23, 2020 at 5:51 AM Eko palypse wrote: > So the question is, what do I need to read/learn/understand in order to solve > this issue? > Or in other words, how can I debug my script in an isolated environment. I'd go for the old standby - IIDPIO: If In Doubt, Print It Out! Instead of trying to use a debug harness, just run your code normally, and print out whatever you think might be of interest. If you don't have a console, well, that would be the first thing to do - you *always* need a console. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Embedded python: How to debug code in an isolated way
Hello, background info first. On windows, python3.8.5 A cpp app has an embedded python interpreter which allows to modify/enhance the cpp app by providing objects to manipulate the cpp app and callbacks to act on certain events, like fileopen, fileclose, updateui ... which are send by the cpp app. The embedded python interpreter acts more or less like the standard python REPL. If you run a script with code like def test(i): print(i*2) test(4) then the test function is now globally available as long as the cpp app is alive. I've written a debugger based on bdb and encountered a situation which I haven't found out how to handle correctly yet. Let's assume I've registered a callback when something changes in the UI. In that case an object function gets called which highlights certain aspects in the UI. Now if I run a script via this debugger it works as long as I don't use the same object function. If I do use it, then the stack gets corrupted(?). For example it look like this stack: , 580 ', line 1, code >, 1 >, 8 now if I enter the function which call this object function this gets add , 5 I step through the code and suddenly the stack looks like this , 580 ', line 1, code >, 1 >, 8 >, 154 , 102)] EnhanceAnyLexer is neither part of the test_pydebugger script nor part of the debugger. It is used to register a callback on the updateui event. So the question is, what do I need to read/learn/understand in order to solve this issue? Or in other words, how can I debug my script in an isolated environment. I can't manipulate the __main__.dict as this seems to have general impact on registered callbacks as well. Any ideas? Thank you Eren -- https://mail.python.org/mailman/listinfo/python-list
Re: Threading module and embedded python
t; but it has been developing since I started using Cython. > > I haven't done any stack analysis yet but hey, there's always a first > time. > > I think my stubbornness could be of help here :-) > > Its a very useful when the simple debug stuff fails. > > > > > Visual Studio reports that the last location of the thread is in > _ctypes.pyd > > and the call stack window shows that the last execution is > > user32.dll!InternalDailogBox(). > > > > Call Stack > > > > > > > *user32.dll!InternalDialogBox()user32.dll!DialogBoxIndirectParamAorW()user32.dll!DialogBoxIndirectParamW()_ctypes.pyd!07fee7fc17e3()_ctypes.pyd!07fee7fbfee3()_ctypes.pyd!07fee7fbb4c5()_ctypes.pyd!07fee7fbc019()_ctypes.pyd!07fee7fb6dfa()python37.dll!_PyObject_FastCallKeywords(_object > > * callable=0x02fa8c78, _object * const * > stack=0x05261c78, > > __int64 nargs=5, _object * kwnames=0x)* > > My guess is that you are missing an important parameter to the dialog that > allows it be seen. > Test the dialog code outside of the embedded python, with a command line > python. > I recall that you have to pass in the parent for a dialog. Maybe in the > embedded version > you do not have an HWND that is usable, can NULL be used? > Is there a SHOW arg that you need to pass? > > I'd check the MSDN docs for the call you are making and check every param > is as required. > > (Its been a along time since I did low level win32 in anger so forgive the > lack of solutions) > > > > > > > > The thread is neither suspended nor does it have any different status > than > > the main thread > > which loops through its main event queue at this point. > > It is suspended inside the user32.dll. > > Barry > > > > > Thank you > > Eren > > > > > > Am Mi., 15. Apr. 2020 um 22:57 Uhr schrieb Barry >: > > > >> > >> > >>> On 15 Apr 2020, at 21:18, Eko palypse wrote: > >>> > >>> Thank you for your suggestion. I will give it a try. > >>> > >>>> What is the "stuck" thread doing? waiting for a lock? > >>> > >>> No, it should open a dialog created with DialogBoxIndirectParamW > >>> < > >> > https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-dialogboxindirectparamw > >>> > >> > >> I assume you are a C++ developer and can look at the stack of the > thread. > >> What is the thread doing? Is it in python code? Is it in windows code? > >> > >> Barry > >> > >>> . > >>> > >>> Eren > >>> > >>>> Am Mi., 15. Apr. 2020 um 20:12 Uhr schrieb Barry Scott < > >>>> ba...@barrys-emacs.org>: > >>>> > >>>> > >>>> > >>>>>> On 15 Apr 2020, at 13:30, Eko palypse wrote: > >>>>> > >>>>> Hi everyone, > >>>>> > >>>>> the following happens on Windows7 x64 and Python37 x64 > >>>>> > >>>>> I have a plugin DLL for a C++ application in which Python37 is > >> embedded. > >>>>> The plugin itself works, except when I want to use the threading > >> module. > >>>>> > >>>>> If I start a Python script in my plugin which uses the threading > module > >>>>> I can verify via ProcessExplorer that the thread is started, > >>>>> but it doesn't do anything (??) and the c++ application doesn't > really > >>>> do anything anymore either. > >>>>> > >>>>> Only when I stop the C++ Applikation, the thread becomes active for a > >>>> short time. > >>>>> Verified with logging module over time print-outs. > >>>>> > >>>>> Apparently I did not understand everything about threads and embedded > >>>> python. > >>>>> > >>>>> Any idea what I'm doing wrong? > >>>> > >>>> This is what I typically do. > >>>> > >>>> Make sure that you have installed the Python debug files. > >>>> Now you can use the visual C++ debugger to attach to the process and > >>>> look at what the threads are doing. > >>>> > >>>> I always have the python source code on hand to read as well. > >>>> > >>>> This should give you a clue.
Re: Threading module and embedded python
> On 16 Apr 2020, at 14:55, Eko palypse wrote: > > Barry, sorry for sending you a private message yesterday, was not intended. > > No, I only have rudimentary knowledge of C++, > but it has been developing since I started using Cython. > I haven't done any stack analysis yet but hey, there's always a first time. > I think my stubbornness could be of help here :-) Its a very useful when the simple debug stuff fails. > > Visual Studio reports that the last location of the thread is in _ctypes.pyd > and the call stack window shows that the last execution is > user32.dll!InternalDailogBox(). > > Call Stack > > > *user32.dll!InternalDialogBox()user32.dll!DialogBoxIndirectParamAorW()user32.dll!DialogBoxIndirectParamW()_ctypes.pyd!07fee7fc17e3()_ctypes.pyd!07fee7fbfee3()_ctypes.pyd!07fee7fbb4c5()_ctypes.pyd!07fee7fbc019()_ctypes.pyd!07fee7fb6dfa()python37.dll!_PyObject_FastCallKeywords(_object > * callable=0x02fa8c78, _object * const * stack=0x05261c78, > __int64 nargs=5, _object * kwnames=0x)* My guess is that you are missing an important parameter to the dialog that allows it be seen. Test the dialog code outside of the embedded python, with a command line python. I recall that you have to pass in the parent for a dialog. Maybe in the embedded version you do not have an HWND that is usable, can NULL be used? Is there a SHOW arg that you need to pass? I'd check the MSDN docs for the call you are making and check every param is as required. (Its been a along time since I did low level win32 in anger so forgive the lack of solutions) > > > The thread is neither suspended nor does it have any different status than > the main thread > which loops through its main event queue at this point. It is suspended inside the user32.dll. Barry > > Thank you > Eren > > > Am Mi., 15. Apr. 2020 um 22:57 Uhr schrieb Barry : > >> >> >>> On 15 Apr 2020, at 21:18, Eko palypse wrote: >>> >>> Thank you for your suggestion. I will give it a try. >>> >>>> What is the "stuck" thread doing? waiting for a lock? >>> >>> No, it should open a dialog created with DialogBoxIndirectParamW >>> < >> https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-dialogboxindirectparamw >>> >> >> I assume you are a C++ developer and can look at the stack of the thread. >> What is the thread doing? Is it in python code? Is it in windows code? >> >> Barry >> >>> . >>> >>> Eren >>> >>>> Am Mi., 15. Apr. 2020 um 20:12 Uhr schrieb Barry Scott < >>>> ba...@barrys-emacs.org>: >>>> >>>> >>>> >>>>>> On 15 Apr 2020, at 13:30, Eko palypse wrote: >>>>> >>>>> Hi everyone, >>>>> >>>>> the following happens on Windows7 x64 and Python37 x64 >>>>> >>>>> I have a plugin DLL for a C++ application in which Python37 is >> embedded. >>>>> The plugin itself works, except when I want to use the threading >> module. >>>>> >>>>> If I start a Python script in my plugin which uses the threading module >>>>> I can verify via ProcessExplorer that the thread is started, >>>>> but it doesn't do anything (??) and the c++ application doesn't really >>>> do anything anymore either. >>>>> >>>>> Only when I stop the C++ Applikation, the thread becomes active for a >>>> short time. >>>>> Verified with logging module over time print-outs. >>>>> >>>>> Apparently I did not understand everything about threads and embedded >>>> python. >>>>> >>>>> Any idea what I'm doing wrong? >>>> >>>> This is what I typically do. >>>> >>>> Make sure that you have installed the Python debug files. >>>> Now you can use the visual C++ debugger to attach to the process and >>>> look at what the threads are doing. >>>> >>>> I always have the python source code on hand to read as well. >>>> >>>> This should give you a clue. >>>> >>>> What is the "stuck" thread doing? waiting for a lock? >>>> >>>> Barry >>>> >>>> >>>> >>>> >>>>> >>>>> >>>>> The whole thing is initialized by the DllMain routine. >>>>> >&g
Re: Threading module and embedded python
Barry, sorry for sending you a private message yesterday, was not intended.
No, I only have rudimentary knowledge of C++,
but it has been developing since I started using Cython.
I haven't done any stack analysis yet but hey, there's always a first time.
I think my stubbornness could be of help here :-)
Visual Studio reports that the last location of the thread is in _ctypes.pyd
and the call stack window shows that the last execution is
user32.dll!InternalDailogBox().
Call Stack
*user32.dll!InternalDialogBox()user32.dll!DialogBoxIndirectParamAorW()user32.dll!DialogBoxIndirectParamW()_ctypes.pyd!07fee7fc17e3()_ctypes.pyd!07fee7fbfee3()_ctypes.pyd!07fee7fbb4c5()_ctypes.pyd!07fee7fbc019()_ctypes.pyd!07fee7fb6dfa()python37.dll!_PyObject_FastCallKeywords(_object
* callable=0x02fa8c78, _object * const * stack=0x05261c78,
__int64 nargs=5, _object * kwnames=0x)*
The thread is neither suspended nor does it have any different status than
the main thread
which loops through its main event queue at this point.
Thank you
Eren
Am Mi., 15. Apr. 2020 um 22:57 Uhr schrieb Barry :
>
>
> > On 15 Apr 2020, at 21:18, Eko palypse wrote:
> >
> > Thank you for your suggestion. I will give it a try.
> >
> >> What is the "stuck" thread doing? waiting for a lock?
> >
> > No, it should open a dialog created with DialogBoxIndirectParamW
> > <
> https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-dialogboxindirectparamw
> >
>
> I assume you are a C++ developer and can look at the stack of the thread.
> What is the thread doing? Is it in python code? Is it in windows code?
>
> Barry
>
> > .
> >
> > Eren
> >
> >> Am Mi., 15. Apr. 2020 um 20:12 Uhr schrieb Barry Scott <
> >> ba...@barrys-emacs.org>:
> >>
> >>
> >>
> >>>> On 15 Apr 2020, at 13:30, Eko palypse wrote:
> >>>
> >>> Hi everyone,
> >>>
> >>> the following happens on Windows7 x64 and Python37 x64
> >>>
> >>> I have a plugin DLL for a C++ application in which Python37 is
> embedded.
> >>> The plugin itself works, except when I want to use the threading
> module.
> >>>
> >>> If I start a Python script in my plugin which uses the threading module
> >>> I can verify via ProcessExplorer that the thread is started,
> >>> but it doesn't do anything (??) and the c++ application doesn't really
> >> do anything anymore either.
> >>>
> >>> Only when I stop the C++ Applikation, the thread becomes active for a
> >> short time.
> >>> Verified with logging module over time print-outs.
> >>>
> >>> Apparently I did not understand everything about threads and embedded
> >> python.
> >>>
> >>> Any idea what I'm doing wrong?
> >>
> >> This is what I typically do.
> >>
> >> Make sure that you have installed the Python debug files.
> >> Now you can use the visual C++ debugger to attach to the process and
> >> look at what the threads are doing.
> >>
> >> I always have the python source code on hand to read as well.
> >>
> >> This should give you a clue.
> >>
> >> What is the "stuck" thread doing? waiting for a lock?
> >>
> >> Barry
> >>
> >>
> >>
> >>
> >>>
> >>>
> >>> The whole thing is initialized by the DllMain routine.
> >>>
> >>>
> >>> BOOL APIENTRY DllMain( HANDLE hModule,
> >>> DWORD reasonForCall,
> >>> LPVOID /* lpReserved */ )
> >>> {
> >>> switch ( reasonForCall )
> >>> {
> >>> case DLL_PROCESS_ATTACH:
> >>> if (!Py_IsInitialized())
> >>> {
> >>> PyImport_AppendInittab("Npp", &PyInit_Npp);
> >>> Py_InitializeEx(0);
> >>> PyEval_InitThreads(); //<- this shouldn't be needed as I
> >> understand that it is called by Py_InitializeEx anyway
> >>> }
> >>> PyImport_ImportModule("Npp");
> >>> break;
> >>> case DLL_PROCESS_DETACH:
> >>> Py_Finalize();
> >>> break;
> >>>
> >>> case DLL_THREAD_ATTACH:
> >>> break;
> >>>
> >>>
Re: Threading module and embedded python
> On 15 Apr 2020, at 21:18, Eko palypse wrote:
>
> Thank you for your suggestion. I will give it a try.
>
>> What is the "stuck" thread doing? waiting for a lock?
>
> No, it should open a dialog created with DialogBoxIndirectParamW
> <https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-dialogboxindirectparamw>
I assume you are a C++ developer and can look at the stack of the thread.
What is the thread doing? Is it in python code? Is it in windows code?
Barry
> .
>
> Eren
>
>> Am Mi., 15. Apr. 2020 um 20:12 Uhr schrieb Barry Scott <
>> ba...@barrys-emacs.org>:
>>
>>
>>
>>>> On 15 Apr 2020, at 13:30, Eko palypse wrote:
>>>
>>> Hi everyone,
>>>
>>> the following happens on Windows7 x64 and Python37 x64
>>>
>>> I have a plugin DLL for a C++ application in which Python37 is embedded.
>>> The plugin itself works, except when I want to use the threading module.
>>>
>>> If I start a Python script in my plugin which uses the threading module
>>> I can verify via ProcessExplorer that the thread is started,
>>> but it doesn't do anything (??) and the c++ application doesn't really
>> do anything anymore either.
>>>
>>> Only when I stop the C++ Applikation, the thread becomes active for a
>> short time.
>>> Verified with logging module over time print-outs.
>>>
>>> Apparently I did not understand everything about threads and embedded
>> python.
>>>
>>> Any idea what I'm doing wrong?
>>
>> This is what I typically do.
>>
>> Make sure that you have installed the Python debug files.
>> Now you can use the visual C++ debugger to attach to the process and
>> look at what the threads are doing.
>>
>> I always have the python source code on hand to read as well.
>>
>> This should give you a clue.
>>
>> What is the "stuck" thread doing? waiting for a lock?
>>
>> Barry
>>
>>
>>
>>
>>>
>>>
>>> The whole thing is initialized by the DllMain routine.
>>>
>>>
>>> BOOL APIENTRY DllMain( HANDLE hModule,
>>> DWORD reasonForCall,
>>> LPVOID /* lpReserved */ )
>>> {
>>> switch ( reasonForCall )
>>> {
>>> case DLL_PROCESS_ATTACH:
>>> if (!Py_IsInitialized())
>>> {
>>> PyImport_AppendInittab("Npp", &PyInit_Npp);
>>> Py_InitializeEx(0);
>>> PyEval_InitThreads(); //<- this shouldn't be needed as I
>> understand that it is called by Py_InitializeEx anyway
>>> }
>>> PyImport_ImportModule("Npp");
>>> break;
>>> case DLL_PROCESS_DETACH:
>>> Py_Finalize();
>>> break;
>>>
>>> case DLL_THREAD_ATTACH:
>>> break;
>>>
>>> case DLL_THREAD_DETACH:
>>> break;
>>> }
>>>
>>> return TRUE;
>>> }
>>>
>>> and the code in the plugin which executes the python scripts is this
>>>
>>> cdef void run_code():
>>> try:
>>> global_dict = globals()
>>> if '__name__' not in global_dict or global_dict['__name__'] !=
>> '__main__':
>>> global_dict.update({"__name__": "__main__",})
>>> exec(compile(editor.getText(), '', 'exec'), global_dict)
>>>
>>> except Exception:
>>> MessageBoxW(nppData._nppHandle,
>>> traceback.format_exc(),
>>> 'RUN CODE EXCEPTION',
>>> 0)
>>>
>>> I don't know if this is important, but the DLL is generated by Cython.
>>>
>>> Thank you for reading and stay healthy
>>>
>>> Eren
>>> --
>>> https://mail.python.org/mailman/listinfo/python-list
>>>
>>
>>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: Threading module and embedded python
Thank you for your suggestion. I will give it a try.
> What is the "stuck" thread doing? waiting for a lock?
No, it should open a dialog created with DialogBoxIndirectParamW
<https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-dialogboxindirectparamw>
.
Eren
Am Mi., 15. Apr. 2020 um 20:12 Uhr schrieb Barry Scott <
ba...@barrys-emacs.org>:
>
>
> > On 15 Apr 2020, at 13:30, Eko palypse wrote:
> >
> > Hi everyone,
> >
> > the following happens on Windows7 x64 and Python37 x64
> >
> > I have a plugin DLL for a C++ application in which Python37 is embedded.
> > The plugin itself works, except when I want to use the threading module.
> >
> > If I start a Python script in my plugin which uses the threading module
> > I can verify via ProcessExplorer that the thread is started,
> > but it doesn't do anything (??) and the c++ application doesn't really
> do anything anymore either.
> >
> > Only when I stop the C++ Applikation, the thread becomes active for a
> short time.
> > Verified with logging module over time print-outs.
> >
> > Apparently I did not understand everything about threads and embedded
> python.
> >
> > Any idea what I'm doing wrong?
>
> This is what I typically do.
>
> Make sure that you have installed the Python debug files.
> Now you can use the visual C++ debugger to attach to the process and
> look at what the threads are doing.
>
> I always have the python source code on hand to read as well.
>
> This should give you a clue.
>
> What is the "stuck" thread doing? waiting for a lock?
>
> Barry
>
>
>
>
> >
> >
> > The whole thing is initialized by the DllMain routine.
> >
> >
> > BOOL APIENTRY DllMain( HANDLE hModule,
> > DWORD reasonForCall,
> > LPVOID /* lpReserved */ )
> > {
> >switch ( reasonForCall )
> >{
> >case DLL_PROCESS_ATTACH:
> >if (!Py_IsInitialized())
> >{
> >PyImport_AppendInittab("Npp", &PyInit_Npp);
> >Py_InitializeEx(0);
> >PyEval_InitThreads(); //<- this shouldn't be needed as I
> understand that it is called by Py_InitializeEx anyway
> >}
> >PyImport_ImportModule("Npp");
> >break;
> >case DLL_PROCESS_DETACH:
> >Py_Finalize();
> >break;
> >
> >case DLL_THREAD_ATTACH:
> >break;
> >
> >case DLL_THREAD_DETACH:
> >break;
> >}
> >
> >return TRUE;
> > }
> >
> > and the code in the plugin which executes the python scripts is this
> >
> > cdef void run_code():
> >try:
> >global_dict = globals()
> >if '__name__' not in global_dict or global_dict['__name__'] !=
> '__main__':
> >global_dict.update({"__name__": "__main__",})
> >exec(compile(editor.getText(), '', 'exec'), global_dict)
> >
> >except Exception:
> >MessageBoxW(nppData._nppHandle,
> >traceback.format_exc(),
> >'RUN CODE EXCEPTION',
> >0)
> >
> > I don't know if this is important, but the DLL is generated by Cython.
> >
> > Thank you for reading and stay healthy
> >
> > Eren
> > --
> > https://mail.python.org/mailman/listinfo/python-list
> >
>
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: Threading module and embedded python
> On 15 Apr 2020, at 13:30, Eko palypse wrote:
>
> Hi everyone,
>
> the following happens on Windows7 x64 and Python37 x64
>
> I have a plugin DLL for a C++ application in which Python37 is embedded.
> The plugin itself works, except when I want to use the threading module.
>
> If I start a Python script in my plugin which uses the threading module
> I can verify via ProcessExplorer that the thread is started,
> but it doesn't do anything (??) and the c++ application doesn't really do
> anything anymore either.
>
> Only when I stop the C++ Applikation, the thread becomes active for a short
> time.
> Verified with logging module over time print-outs.
>
> Apparently I did not understand everything about threads and embedded python.
>
> Any idea what I'm doing wrong?
This is what I typically do.
Make sure that you have installed the Python debug files.
Now you can use the visual C++ debugger to attach to the process and
look at what the threads are doing.
I always have the python source code on hand to read as well.
This should give you a clue.
What is the "stuck" thread doing? waiting for a lock?
Barry
>
>
> The whole thing is initialized by the DllMain routine.
>
>
> BOOL APIENTRY DllMain( HANDLE hModule,
> DWORD reasonForCall,
> LPVOID /* lpReserved */ )
> {
>switch ( reasonForCall )
>{
>case DLL_PROCESS_ATTACH:
>if (!Py_IsInitialized())
>{
>PyImport_AppendInittab("Npp", &PyInit_Npp);
>Py_InitializeEx(0);
>PyEval_InitThreads(); //<- this shouldn't be needed as I
> understand that it is called by Py_InitializeEx anyway
>}
>PyImport_ImportModule("Npp");
>break;
>case DLL_PROCESS_DETACH:
>Py_Finalize();
>break;
>
>case DLL_THREAD_ATTACH:
>break;
>
>case DLL_THREAD_DETACH:
>break;
>}
>
>return TRUE;
> }
>
> and the code in the plugin which executes the python scripts is this
>
> cdef void run_code():
>try:
>global_dict = globals()
>if '__name__' not in global_dict or global_dict['__name__'] !=
> '__main__':
>global_dict.update({"__name__": "__main__",})
>exec(compile(editor.getText(), '', 'exec'), global_dict)
>
>except Exception:
>MessageBoxW(nppData._nppHandle,
>traceback.format_exc(),
>'RUN CODE EXCEPTION',
>0)
>
> I don't know if this is important, but the DLL is generated by Cython.
>
> Thank you for reading and stay healthy
>
> Eren
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list
Threading module and embedded python
Hi everyone,
the following happens on Windows7 x64 and Python37 x64
I have a plugin DLL for a C++ application in which Python37 is embedded.
The plugin itself works, except when I want to use the threading module.
If I start a Python script in my plugin which uses the threading module
I can verify via ProcessExplorer that the thread is started,
but it doesn't do anything (??) and the c++ application doesn't really do
anything anymore either.
Only when I stop the C++ Applikation, the thread becomes active for a short
time.
Verified with logging module over time print-outs.
Apparently I did not understand everything about threads and embedded python.
Any idea what I'm doing wrong?
The whole thing is initialized by the DllMain routine.
BOOL APIENTRY DllMain( HANDLE hModule,
DWORD reasonForCall,
LPVOID /* lpReserved */ )
{
switch ( reasonForCall )
{
case DLL_PROCESS_ATTACH:
if (!Py_IsInitialized())
{
PyImport_AppendInittab("Npp", &PyInit_Npp);
Py_InitializeEx(0);
PyEval_InitThreads(); //<- this shouldn't be needed as I
understand that it is called by Py_InitializeEx anyway
}
PyImport_ImportModule("Npp");
break;
case DLL_PROCESS_DETACH:
Py_Finalize();
break;
case DLL_THREAD_ATTACH:
break;
case DLL_THREAD_DETACH:
break;
}
return TRUE;
}
and the code in the plugin which executes the python scripts is this
cdef void run_code():
try:
global_dict = globals()
if '__name__' not in global_dict or global_dict['__name__'] !=
'__main__':
global_dict.update({"__name__": "__main__",})
exec(compile(editor.getText(), '', 'exec'), global_dict)
except Exception:
MessageBoxW(nppData._nppHandle,
traceback.format_exc(),
'RUN CODE EXCEPTION',
0)
I don't know if this is important, but the DLL is generated by Cython.
Thank you for reading and stay healthy
Eren
--
https://mail.python.org/mailman/listinfo/python-list
Re: Using exec with embedded python interpreter 3.7
@MRAB, I'm building a notepad++ plugin which can execute the written code and if one writes help(os) it gets executed via exec(editor.getText()) and output redirected to the plugin console window. Sorry to you as well as I have also replied to you directly. Thank you -- https://mail.python.org/mailman/listinfo/python-list
Re: Using exec with embedded python interpreter 3.7
Just saw, that I replied to you directly instead to python list, sorry. That did it, changed encoding from function to property and now I'm able to call help(object) Thank you. -- https://mail.python.org/mailman/listinfo/python-list
Re: Using exec with embedded python interpreter 3.7
On 2019-08-31 13:16, Eko palypse wrote: I've already sent this through mail yesterday but it doesn't appear here, maybe because of the help word in the content. Please execute in case it appears a second time. Hello, I'm creating a notepad++ plugin which hosts an embedded python interpreter by using cffi to build the dll. So far so good. One obstacle I've found is that I'm not able to use exec(help(object)) in order to get the wanted info from the object. The error I get is: Traceback (most recent call last): File "", line 131, in run_code File "", line 1, in File "D:\...\Python\Python37_64\Lib\_sitebuiltins.py", line 103, in __call__ return pydoc.help(*args, **kwds) File "D:\...\Python\Python37_64\Lib\pydoc.py", line 1895, in __call__ self.help(request) File "D:\...\Python\Python37_64\Lib\pydoc.py", line 1954, in help else: doc(request, 'Help on %s:', output=self._output) File "D:\...\Python\Python37_64\Lib\pydoc.py", line 1674, in doc pager(render_doc(thing, title, forceload)) File "D:\...\Python\Python37_64\Lib\pydoc.py", line 1451, in pager pager(text) File "D:\...\Python\Python37_64\Lib\pydoc.py", line 1576, in plainpager sys.stdout.write(plain(_escape_stdout(text))) File "D:\...\Python\Python37_64\Lib\pydoc.py", line 1528, in _escape_stdout return text.encode(encoding, 'backslashreplace').decode(encoding) TypeError: encode() argument 1 must be str, not method If I encase the object with quotes then I do get the info that no documentation is found for that object. 1. It doesn't feel right to encase the object with quotes. I don't have to do this using the standard python interpreter shell. What did I miss? 2. If it is needed to use the encased way, why don't it show the help info? The object class contains the module, class and functions doc strings, what did I miss here? I don't understand the "exec(help(object))" bit. help(object) print the help and returns None, so why exec(None)? -- https://mail.python.org/mailman/listinfo/python-list
Re: Using exec with embedded python interpreter 3.7
Eko palypse wrote: > I've already sent this through mail yesterday but it doesn't appear here, > maybe because of the help word in the content. Please execute in case it > appears a second time. > > > Hello, > I'm creating a notepad++ plugin which hosts an embedded python interpreter > by using cffi to build the dll. So far so good. One obstacle I've found is > that I'm not able to use exec(help(object)) in order to get the wanted > info from the object. > > The error I get is: > > Traceback (most recent call last): > File "", line 131, in run_code > File "", line 1, in > File "D:\...\Python\Python37_64\Lib\_sitebuiltins.py", line 103, in > __call__ > return pydoc.help(*args, **kwds) > File "D:\...\Python\Python37_64\Lib\pydoc.py", line 1895, in __call__ > self.help(request) > File "D:\...\Python\Python37_64\Lib\pydoc.py", line 1954, in help > else: doc(request, 'Help on %s:', output=self._output) > File "D:\...\Python\Python37_64\Lib\pydoc.py", line 1674, in doc > pager(render_doc(thing, title, forceload)) > File "D:\...\Python\Python37_64\Lib\pydoc.py", line 1451, in pager > pager(text) > File "D:\...\Python\Python37_64\Lib\pydoc.py", line 1576, in plainpager > sys.stdout.write(plain(_escape_stdout(text))) > File "D:\...\Python\Python37_64\Lib\pydoc.py", line 1528, in > _escape_stdout > return text.encode(encoding, 'backslashreplace').decode(encoding) > TypeError: encode() argument 1 must be str, not method The traceback suggests that sys.stdout was replaced with a custom object that has a method stdout.encoding() rather than the expected stdout.encoding attribute of type str. > If I encase the object with quotes then I do get the info that no > documentation is found for that object. > > 1. It doesn't feel right to encase the object with quotes. I don't have to > do this using the standard python interpreter shell. What did I miss? > > 2. If it is needed to use the encased way, why don't it show the help > info? The object class contains the module, class and functions doc > strings, what did I miss here? > > Thank you > Eren -- https://mail.python.org/mailman/listinfo/python-list
Using exec with embedded python interpreter 3.7
I've already sent this through mail yesterday but it doesn't appear here, maybe because of the help word in the content. Please execute in case it appears a second time. Hello, I'm creating a notepad++ plugin which hosts an embedded python interpreter by using cffi to build the dll. So far so good. One obstacle I've found is that I'm not able to use exec(help(object)) in order to get the wanted info from the object. The error I get is: Traceback (most recent call last): File "", line 131, in run_code File "", line 1, in File "D:\...\Python\Python37_64\Lib\_sitebuiltins.py", line 103, in __call__ return pydoc.help(*args, **kwds) File "D:\...\Python\Python37_64\Lib\pydoc.py", line 1895, in __call__ self.help(request) File "D:\...\Python\Python37_64\Lib\pydoc.py", line 1954, in help else: doc(request, 'Help on %s:', output=self._output) File "D:\...\Python\Python37_64\Lib\pydoc.py", line 1674, in doc pager(render_doc(thing, title, forceload)) File "D:\...\Python\Python37_64\Lib\pydoc.py", line 1451, in pager pager(text) File "D:\...\Python\Python37_64\Lib\pydoc.py", line 1576, in plainpager sys.stdout.write(plain(_escape_stdout(text))) File "D:\...\Python\Python37_64\Lib\pydoc.py", line 1528, in _escape_stdout return text.encode(encoding, 'backslashreplace').decode(encoding) TypeError: encode() argument 1 must be str, not method If I encase the object with quotes then I do get the info that no documentation is found for that object. 1. It doesn't feel right to encase the object with quotes. I don't have to do this using the standard python interpreter shell. What did I miss? 2. If it is needed to use the encased way, why don't it show the help info? The object class contains the module, class and functions doc strings, what did I miss here? Thank you Eren -- https://mail.python.org/mailman/listinfo/python-list
Using exec with embedded python interpreter 3.7
Hello, I'm creating a notepad++ plugin which hosts an embedded python interpreter by using cffi to build the dll. So far so good. One obstacle I've found is that I'm not able to use exec(help(object)) in order to get the wanted info from the object. The error I get is: Traceback (most recent call last): File "", line 131, in run_code File "", line 1, in File "D:\...\Python\Python37_64\Lib\_sitebuiltins.py", line 103, in __call__ return pydoc.help(*args, **kwds) File "D:\...\Python\Python37_64\Lib\pydoc.py", line 1895, in __call__ self.help(request) File "D:\...\Python\Python37_64\Lib\pydoc.py", line 1954, in help else: doc(request, 'Help on %s:', output=self._output) File "D:\...\Python\Python37_64\Lib\pydoc.py", line 1674, in doc pager(render_doc(thing, title, forceload)) File "D:\...\Python\Python37_64\Lib\pydoc.py", line 1451, in pager pager(text) File "D:\...\Python\Python37_64\Lib\pydoc.py", line 1576, in plainpager sys.stdout.write(plain(_escape_stdout(text))) File "D:\...\Python\Python37_64\Lib\pydoc.py", line 1528, in _escape_stdout return text.encode(encoding, 'backslashreplace').decode(encoding) TypeError: encode() argument 1 must be str, not method If I encase the object with quotes then I do get the info that no documentation is found for that object. 1. It doesn't feel right to encase the object with quotes. I don't have to do this using the standard python interpreter shell. What did I miss? 2. If it is needed to use the encased way, why don't it show the help info? The object class contains the module, class and functions doc strings, what did I miss here? Thank you Eren -- https://mail.python.org/mailman/listinfo/python-list
Re: Embedded Python and multiprocessing on Windows?
On Friday, August 10, 2018 at 2:28:45 AM UTC-4, Léo El Amri wrote:
> That may be something simple: Did you actually protected the entry-point
> of your Python script with if __name__ == '__main__': ?
That was my first thought too; the script technically doesn't have top-level
code, so I figured I didn't need a main catch. Just to be sure, I added one and
it still didn't resolve. The function in the second script that correctly
multiprocesses doesn't have a main (or top level code) either.
I'm wondering if it's related to how I call the script in code. I'm using
boost.python, but I'm not sure I'm doing it entirely correctly (the
boost.python documentation is weirdly hazy on the semantics of calling script
files, but is drowning in examples of how to execute hard-coded lines, which
seems insane to me; who on earth would go to the trouble to embed an
interpreter to run hardcoded Python strings?)
Anyway, those lines look like this:
py::object main_module = py::import("__main__");
py::dict global = py::extract(main_module.attr("__dict__"));
py::object result = py::exec_file(scriptName, global, global);
py::object runner = global["EntryFunction"];
retval = py::extract(runner(py::ptr(taskPtr)));
// we pass a pointer to the entry function
--
https://mail.python.org/mailman/listinfo/python-list
Re: Embedded Python and multiprocessing on Windows?
On 09/08/2018 19:33, Apple wrote:> So my program runs one script file, and multiprocessing commands from that script file seem to fail to spawn new processes. > > However, if that script file calls a function in a separate script file that > it has imported, and that function calls multiprocessing functions, it all > works exactly the way it should. > On Thursday, August 9, 2018 at 12:09:36 PM UTC-4, Apple wrote: >> I've been working on a project involving embedding Python into a Windows >> application. I've got all of that working fine on the C++ side, but the >> script side seems to be hitting a dead end with multiprocessing. When my >> script tries to run the same multiprocessing code that works in a >> non-embedded environment, the code doesn't appear to be executed at all. >> >> Still no joy. However, a Python.exe window does pop up for a tenth of a >> second, so *something* is happening. That may be something simple: Did you actually protected the entry-point of your Python script with if __name__ == '__main__': ? -- https://mail.python.org/mailman/listinfo/python-list
Re: Problems with imports on multiple threads, with embedded Python
geoff.ba...@gmail.com writes: > I have a multithreaded application using an embedded Python 3.6.4 (upgraded > from 3.6.2 today in the hope that the problem was now solved: it doesn't seem > to be). The standard library is in a zip file. So long as only one thread is > running Python at a time it seems to work fine. But there seems to be a > problem with the module importing when several Python threads are active. > > I get a variety of errors indeterministically, usually indicating that some > symbol hasn't been imported. This occurs both in my own code and in the > standard library. The most frequent is probably this line: I do not know your specific problem (up to now, I used only Python 2), but I know about import problems in multi threaded applications. The problem is easily understood: import affects the global object "sys.modules". To protect this object, Python must do something. In former Python versions (somewhere before Python 3), it was using a a thread lock: this could cause a deadlock in the case of some import dependencies. The recoomendation to avoid the problem has been to avoid module level imports for modules imported by threads. Python 3 may have changed the way to protect "sys.modules". In case of recursive import dependencies, you may see that some symbols are not defined even in single threaded applications -- however then usually deterministically. At your place, I would start to check whether your application has recursive import dependancies -- and in this case, remove them. -- https://mail.python.org/mailman/listinfo/python-list
Re: Problems with imports on multiple threads, with embedded Python
On Thursday, 21 December 2017 00:33:54 UTC+1, Lawrence D’Oliveiro wrote: > On Thursday, December 21, 2017 at 5:13:33 AM UTC+13, geoff...@gmail.com wrote: > > > > I have a multithreaded application using an embedded Python 3.6.4 ... > > Avoid multithreading if you can. Is your application CPU-bound? If not, you > are getting no benefit from it. Unfortunately that's not an option. As I say, Python is embedded and is only a part of what the application does. The multithreading is not there for the Python part but is a well-established fact in the application. -- https://mail.python.org/mailman/listinfo/python-list
Problems with imports on multiple threads, with embedded Python
Hi all, I have a multithreaded application using an embedded Python 3.6.4 (upgraded from 3.6.2 today in the hope that the problem was now solved: it doesn't seem to be). The standard library is in a zip file. So long as only one thread is running Python at a time it seems to work fine. But there seems to be a problem with the module importing when several Python threads are active. I get a variety of errors indeterministically, usually indicating that some symbol hasn't been imported. This occurs both in my own code and in the standard library. The most frequent is probably this line: dt = datetime.strptime(dtStr, fromFmt) which produces AttributeError: module '_strptime' has no attribute '_strptime_datetime' at random. Does anyone have any insight into this problem? Regards, Geoff Bache -- https://mail.python.org/mailman/listinfo/python-list
Re: Embedded Python import fails with zip/egg files (v3.6.1)
Thanks eryk! It looks like I have to dig deeper and step through the Python code again to see what's going on. On 11.05.2017 15:37, eryk sun wrote: On Thu, May 11, 2017 at 9:02 PM, Griebel, Herbert wrote: 07:59:04,3205458python.exe4224CreateFile C:\Users\hansi\Downloads\python-emb\python36.zipSUCCESS Desired Access: Read Attributes, Synchronize, Disposition: Open, Options: Synchronous IO Non-Alert, Open Reparse Point, Attributes: N, ShareMode: None, AllocationSize: n/a, OpenResult: Opened 07:59:04,3205860python.exe4224 QueryInformationVolume C:\Users\hansi\Downloads\python-emb\python36.zipSUCCESS VolumeCreationTime: 05.05.2015 12:28:45, VolumeSerialNumber: 36B5-A026, SupportsObjects: True, VolumeLabel: OS 07:59:04,3206127python.exe4224 QueryAllInformationFile C:\Users\hansi\Downloads\python-emb\python36.zipBUFFER OVERFLOW CreationTime: 18.04.2017 06:07:23, LastAccessTime: 18.04.2017 06:07:23, LastWriteTime: 21.03.2017 09:06:10, ChangeTime: 18.04.2017 06:07:23, FileAttributes: N, AllocationSize: 2.228.224, EndOfFile: 2.224.303, NumberOfLinks: 1, DeletePending: False, Directory: False, IndexNumber: 0x2000a9467, EaSize: 0, Access: Read Attributes, Synchronize, Position: 0, Mode: Synchronous IO Non-Alert, AlignmentRequirement: Word This looks like a regular Python stat call on Windows. It opens a handle without following links (i.e. reparse points) and calls GetFileInformationByHandle. That in turn gets the volume serial number from the volume information. Then it gets the file information, which includes the filename. But the FILE_ALL_INFORMATION buffer only has space for a single character of the name. That's the reason for the buffer overflow (0x8005). It's an NTSTATUS warning, not an error, and it doesn't fail the GetFileInformationByHandle call. -- https://mail.python.org/mailman/listinfo/python-list
Re: Embedded Python import fails with zip/egg files (v3.6.1)
On Thu, May 11, 2017 at 9:02 PM, Griebel, Herbert wrote: > > 07:59:04,3205458python.exe4224CreateFile > C:\Users\hansi\Downloads\python-emb\python36.zipSUCCESS Desired Access: > Read Attributes, Synchronize, Disposition: Open, Options: Synchronous IO > Non-Alert, Open Reparse Point, Attributes: N, ShareMode: None, > AllocationSize: n/a, OpenResult: Opened > > 07:59:04,3205860python.exe4224 QueryInformationVolume > C:\Users\hansi\Downloads\python-emb\python36.zipSUCCESS > VolumeCreationTime: 05.05.2015 12:28:45, VolumeSerialNumber: 36B5-A026, > SupportsObjects: True, VolumeLabel: OS > > 07:59:04,3206127python.exe4224 QueryAllInformationFile > C:\Users\hansi\Downloads\python-emb\python36.zipBUFFER OVERFLOW > CreationTime: 18.04.2017 06:07:23, LastAccessTime: 18.04.2017 06:07:23, > LastWriteTime: 21.03.2017 09:06:10, ChangeTime: 18.04.2017 06:07:23, > FileAttributes: N, AllocationSize: 2.228.224, EndOfFile: 2.224.303, > NumberOfLinks: 1, DeletePending: False, Directory: False, IndexNumber: > 0x2000a9467, EaSize: 0, Access: Read Attributes, Synchronize, Position: > 0, Mode: Synchronous IO Non-Alert, AlignmentRequirement: Word This looks like a regular Python stat call on Windows. It opens a handle without following links (i.e. reparse points) and calls GetFileInformationByHandle. That in turn gets the volume serial number from the volume information. Then it gets the file information, which includes the filename. But the FILE_ALL_INFORMATION buffer only has space for a single character of the name. That's the reason for the buffer overflow (0x8005). It's an NTSTATUS warning, not an error, and it doesn't fail the GetFileInformationByHandle call. -- https://mail.python.org/mailman/listinfo/python-list
Embedded Python import fails with zip/egg files (v3.6.1)
Hello, I am having trouble importing python modules on certain machines. On some machines import works, on some not (all machines are Win7 64bit). Python is not installed on any of these machines but used embedded. I tried to analyze the problem but did not succeed so here is what I found. First I will use the module xlsxwriter to explain the problem but it also happens with python36.zip (when importing for example codecs). I have a xlsxwriter.egg file which is found by the import mechanism but it cannot be opened. /Traceback (most recent call last):/ // /File "Z:\Documents\///myscript/.py", line 1, in / /import glob, inspect, os, json, base64, xlsxwriter, datetime, string/ /ModuleNotFoundError: No module named 'xlsxwriter'/ When I unzip the egg and create two folders, for code and egg-info, it works, the module is imported. Again, the very same egg file works fine on other machines. Tested on win7 with or without python installed, and freshly setup win7 systems with nothing else installed. I have the same problem with python36.zip that comes with the embedded package. When starting python.exe (from https://www.python.org/ftp/python/3.6.1/python-3.6.1-embed-win32.zip) the codecs module cannot be imported and python.exe crashes. All paths are correctly set. When I unzip the python36.zip into the python.exe folder everything works fine. What I found interesting is that the disk monitor tool (Procmon.exe) shows following detail: 07:59:04,3187854python.exe4224CreateFile C:\Users\hansi\Downloads\python-emb\python36.zipSUCCESS Desired Access: Read Attributes, Synchronize, Disposition: Open, Options: Synchronous IO Non-Alert, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened 07:59:04,3198189python.exe4224CloseFile C:\Users\hansi\Downloads\python-emb\python36.zipSUCCESS 07:59:04,3205458python.exe4224CreateFile C:\Users\hansi\Downloads\python-emb\python36.zipSUCCESS Desired Access: Read Attributes, Synchronize, Disposition: Open, Options: Synchronous IO Non-Alert, Open Reparse Point, Attributes: N, ShareMode: None, AllocationSize: n/a, OpenResult: Opened 07:59:04,3205860python.exe4224 QueryInformationVolume C:\Users\hansi\Downloads\python-emb\python36.zipSUCCESS VolumeCreationTime: 05.05.2015 12:28:45, VolumeSerialNumber: 36B5-A026, SupportsObjects: True, VolumeLabel: OS 07:59:04,3206127python.exe4224 QueryAllInformationFile C:\Users\hansi\Downloads\python-emb\python36.zipBUFFER OVERFLOW CreationTime: 18.04.2017 06:07:23, LastAccessTime: 18.04.2017 06:07:23, LastWriteTime: 21.03.2017 09:06:10, ChangeTime: 18.04.2017 06:07:23, FileAttributes: N, AllocationSize: 2.228.224, EndOfFile: 2.224.303, NumberOfLinks: 1, DeletePending: False, Directory: False, IndexNumber: 0x2000a9467, EaSize: 0, Access: Read Attributes, Synchronize, Position: 0, Mode: Synchronous IO Non-Alert, AlignmentRequirement: Word The interesting line is the one with QueryAllInformationFile and BUFFER OVERFLOW. On machines where it works the buffer overflow does not happen and the query is done with QueryBasicInformationFile and not QueryInformationVolume. Since QueryInformationVolume is most likely only for folders, maybe there is a problem with that. Here is the log when it's working: 06:30:39,6650716python.exe30176CreateFile C:\Projects\Python\rt_win32\python36.zipSUCCESSDesired Access: Read Attributes, Synchronize, Disposition: Open, Options: Synchronous IO Non-Alert, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened 06:30:39,6652657python.exe30176 QueryBasicInformationFile C:\Projects\Python\rt_win32\python36.zipSUCCESS CreationTime: 15.02.2017 13:34:03, LastAccessTime: 15.02.2017 13:34:03, LastWriteTime: 22.12.2016 23:30:40, ChangeTime: 18.04.2017 06:19:36, FileAttributes: A 06:30:39,6673617python.exe30176 QueryStandardInformationFile C:\Projects\Python\rt_win32\python36.zipSUCCESS AllocationSize: 2.240.512, EndOfFile: 2.237.601, NumberOfLinks: 1, DeletePending: False, Directory: False Any help is appreciated! Thanks, Herb -- https://mail.python.org/mailman/listinfo/python-list
How to pass C++ function pointer argument in embedded python environment?
Hi,All, background:I have a python api like this, def a(arg1,arg2,progress_callback = None) obviously argument progress_callback is a callback function.In python,I can define a function like this: def pro_call(prog_arg1,prog_arg2): #do something with arg1 & arg2 and I just need call a function like follow and it worked a(arg1,arg2,progress_callback = pro_call) question: I don't know how to do like the former exam in embedded python environment. thank you for your reading ,thanks for any response! -- https://mail.python.org/mailman/listinfo/python-list
Re: Can't load Tkinter in embedded Python interpreter on Windows
Adding PySys_SetArgv(argc, argv); did the trick. -- Kevin Walzer Code by Kevin/Mobile Code by Kevin http://www.codebykevin.com http://www.wtmobilesoftware.com -- https://mail.python.org/mailman/listinfo/python-list
Can't load Tkinter in embedded Python interpreter on Windows
I am trying to build a stub exe on Windows that embeds Python and
launches my Tkinter app. I want a full-blown exe program starter because
the various Python freezing tools (py2exe, pyinstaller) do not work to
my satisfaction with Python 3.5.
I am able to get the executable built but I cannot get it to load
Tkinter and run. The current error is:
"AttributeError: module 'sys' has no attribute 'argv'"
when called from the Tkinter init method.
Here is my command-line invocation for the compiler:
---
gcc quickwho.c -I
\C:\Users\kevin\AppData\Local\Programs\Python\Python35\include
-LC:\Users\kevin\AppData\Local\Programs\Python\Python35\libs
-LC:\Users\kevin\AppData\Local\Programs\Python\Python35\DLLs -lShlwapi
-lpython35 -o quickwho.exe
And here is my C stub launcher:
#include
#include
#include
#include
#include
int main(int argc, char *argv[])
{
Py_SetProgramName(argv[0]);
Py_Initialize();
TCHAR exedir [MAX_PATH];
#if 0
GetModuleFileName(NULL, exedir, MAX_PATH);
_tprintf("%s/n", exedir);
PathRemoveFileSpec(exedir);
_tprintf("%s/n", exedir);
#endif
/*Get the module's full path, and set to the current working
directory.*/
if (!GetModuleFileName(NULL, exedir, MAX_PATH)) {
TCHAR errmsg[512];
FormatMessage(FORMAT_MESSAGE_FROM_SYSTEM,0,
GetLastError(),0,errmsg,1024,NULL);
_tprintf( TEXT("The path is %s, and the error is %s/n"), exedir,
errmsg );
}
if (!PathRemoveFileSpec(exedir)) {
TCHAR errmsg[512];
FormatMessage(FORMAT_MESSAGE_FROM_SYSTEM,0,
GetLastError(),0,errmsg,1024,NULL);
_tprintf( TEXT("The target dir is %s, and the error is %s/n"),
exedir, errmsg );
}
if (!SetCurrentDirectory(exedir)) {
TCHAR errmsg[512];
FormatMessage(FORMAT_MESSAGE_FROM_SYSTEM,0,
GetLastError(),0,errmsg,1024,NULL);
_tprintf( TEXT("The working dir is %s, and the error is %s/n"),
exedir,errmsg );
}
PyRun_SimpleString("exec(open(\"QuickWho.py\").read())");
Py_Finalize();
return 0;
}
Can anyone suggest what I might do to get Tkinter to load and run my exe?
Thanks,
Kevin
--
Kevin Walzer
Code by Kevin/Mobile Code by Kevin
http://www.codebykevin.com
http://www.wtmobilesoftware.com
--
https://mail.python.org/mailman/listinfo/python-list
Windows - Embedded Python in C++: ImportError: DLL load failed. Python expert advice needed.
I've been busy porting a Linux code to Windows, and originally decided to go
for the mingw-w64 toolchain set, with gcc-4.8 and win32 threading model.
I'm embedding Python in C++ code, and I need to be able to import specific
modules, like PySide, to eventually embed an IPython qtconsole in a C++ Qt
window. This all works under Linux, so the approach is sound. And everything
compiled properly under Windows too. PySide was installed via pip.
Now, from a normal Python/IPython console on the Windows system, I can import
what I want:
import PySide.QtCore
No problem here, as expected. But when on the embed side I do:
PyRun_SimpleString("import PySide.QtCore");
I get the following error:
Traceback (most recent call last):
File "", line 2, in
ImportError: DLL load failed: The specified procedure could not be found.
When I import:
PyRun_SimpleString("import PySide");
It works, but I get an empty PySide object. If I import "sys", "os", or
"math", all the modules work as expected though, which is probably due to the
fact that they are compiled in python27.dll.
So I'm a bit puzzled as to what is happening. Why can I start a Python embedded
interpreter, load the builtin modules but not external modules? (I tried other
modules like matplotlib / numpy, which didn't work either).
I very strongly suspect the problem to be a linking error, as python27.dll is
linked to msvcr90.dll, and mingw-w64 links to msvcrt.dll, meaning my library
will be linked to the wrong dll. But in that case, why am I able to load and
run Python nonetheless?
Thanks for the help!
--
https://mail.python.org/mailman/listinfo/python-list
Re: Strange crash while running a script with a embedded python interpreter
On Mon, Jan 11, 2016 at 6:55 PM, Rickard Englund wrote: > On Friday, January 8, 2016 at 11:28:53 PM UTC+1, Michael Torrie wrote: >> On 01/08/2016 09:18 AM, Rickard Englund wrote: >> > First, some system info >> > * Windows 7 (also tested on 8 and 10) >> > * Python 3.5.1 64bit (previously also tested using several 3.x versions) >> > (also tested with 32 bit, but with 3.4.2) >> > * Microsoft Visual Studio 2015 (earlier version of python also tested with >> > Visual Studio 2013) >> >> Are you using the same version of Visual Studio that Python itself was >> compiled with? If not there can be subtle problems with incompatible >> msvcrt dlls. No idea if this would be contributing to the problem or >> not, though. > > I've just downloaded the python source code and build it myself, the compiler > settings in our project is the same as in the python projects. > > > Though, your comment led me in the correct direction. When using the > libs/binary I built my self it let me see the values of the PyObjects around > the crash and it seems like it is related to our modules and/or how we expose > them to the interpreter. When disabling our own modules everything seems to > be working as it should. > Interesting. So somewhere along the way, you have native code (C code?) that's creating a module, and that's where the trouble starts? I would first look at all your refcount management; if that goes wrong, all sorts of craziness could happen (if you still have a pointer to a block of memory that gets released and reused, hilarity will ensue - or, something will). Have you considered using Cython? That might let you do what you need without worrying about all those annoying internal API details. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Strange crash while running a script with a embedded python interpreter
On Friday, January 8, 2016 at 11:28:53 PM UTC+1, Michael Torrie wrote: > On 01/08/2016 09:18 AM, Rickard Englund wrote: > > First, some system info > > * Windows 7 (also tested on 8 and 10) > > * Python 3.5.1 64bit (previously also tested using several 3.x versions) > > (also tested with 32 bit, but with 3.4.2) > > * Microsoft Visual Studio 2015 (earlier version of python also tested with > > Visual Studio 2013) > > Are you using the same version of Visual Studio that Python itself was > compiled with? If not there can be subtle problems with incompatible > msvcrt dlls. No idea if this would be contributing to the problem or > not, though. I've just downloaded the python source code and build it myself, the compiler settings in our project is the same as in the python projects. Though, your comment led me in the correct direction. When using the libs/binary I built my self it let me see the values of the PyObjects around the crash and it seems like it is related to our modules and/or how we expose them to the interpreter. When disabling our own modules everything seems to be working as it should. I think I got it wokring now, used some ugly hacks I need to clean up before I certain it works. Thanks for the help. -- https://mail.python.org/mailman/listinfo/python-list
Re: Strange crash while running a script with a embedded python interpreter
On 01/08/2016 09:18 AM, Rickard Englund wrote: > First, some system info > * Windows 7 (also tested on 8 and 10) > * Python 3.5.1 64bit (previously also tested using several 3.x versions) > (also tested with 32 bit, but with 3.4.2) > * Microsoft Visual Studio 2015 (earlier version of python also tested with > Visual Studio 2013) Are you using the same version of Visual Studio that Python itself was compiled with? If not there can be subtle problems with incompatible msvcrt dlls. No idea if this would be contributing to the problem or not, though. -- https://mail.python.org/mailman/listinfo/python-list
Re: Strange crash while running a script with a embedded python interpreter
On Friday, January 8, 2016 at 5:18:56 PM UTC+1, Rickard Englund wrote:
> First, some system info
> * Windows 7 (also tested on 8 and 10)
> * Python 3.5.1 64bit (previously also tested using several 3.x versions)
> (also tested with 32 bit, but with 3.4.2)
> * Microsoft Visual Studio 2015 (earlier version of python also tested with
> Visual Studio 2013)
>
> We have tested with both the debug and release libs for 3.5 and 3.5.1 (those
> available from python installer) with the same problem.
>
> The following problem has also been tested on Unix and on MacOS where it
> works as expected, the problem is only in Windows.
>
>
> Problem description:
> We have an application for scientific visualization in which we have enabled
> scripting using embedded python. We have exposed quite some features of our
> application to python using modules, this works perfectly fine.
> The problem we have is that the application crashes as soon as a script is
> trying to import some of python's default libraries, such as os or random.
> For example, running the single line code "import os" produces a crash with
> the following stacktrace:
> http://pastebin.com/QpfGrMY0
>
> We initialize the python interpreter using the following:
> http://pastebin.com/uyx6gdMb
>
> We have tried with and without the Py_NoSiteFlag = 1. When removing that flag
> it works for some modules, for example "import os" works, but other modules,
> for example "import random" crashes.
>
>
> In the pastebin link above for the python initialization there are three
> lines commented out, "importModule("os")","importModule("glog")" and
> "importModule("random")", if I uncomment those lines the application will not
> crash when importing those modules. But that is not a viable solution, we
> can't import all possible modules that way.
>
> The following piece of code is how we execute python code:
> http://pastebin.com/HDUR2LKT
>
>
> It only crashes for modules that exists , if I try to import a module that
> does not at all exist (eg import unexisting_module) , it fails as expected
> with a nice error message saying it does not exist.
>
>
> I don't know if we are doing anything wrong in the initialization of the
> interpreter or when running the script. We can run scripts that are not using
> any modules or are only using the python modules we create in c++ to expose
> functionality of our application.
> Everything is working fine on MacOSX and Linux (tested with Ubuntu) which are
> the platforms we support. We had a similar setup before using python 2.x
> instead of 3.x which also worked without any problems.
>
>
> The software i am talking about is using QT for gui and and OpenGL for
> hardware accelerated graphics. But I have tested it in a minimal application
> where all QT stuff were omitted and the same issue is still there.
>
> The software is open source and can be downloaded/clones from
> github.com/inviwo/inviwo.
>
> The python initialization is done in
> https://github.com/inviwo/inviwo-dev/blob/master/modules/python3/pyinviwo.cpp
> and the script compiling and running is handled in
> https://github.com/inviwo/inviwo-dev/blob/master/modules/python3/pythonscript.cpp
>
>
> I have searched a lot on google and in the issue tracker for python but found
> nothing that can answer my questions. So I hope that someone here can us
> solve this.
I notice now that I have pasted 2 links to our private repository, which will
not work.
The correct links are:
https://github.com/inviwo/inviwo/blob/master/modules/python3/pyinviwo.cpp
https://github.com/inviwo/inviwo/blob/master/modules/python3/pythonscript.cpp
--
https://mail.python.org/mailman/listinfo/python-list
Strange crash while running a script with a embedded python interpreter
First, some system info
* Windows 7 (also tested on 8 and 10)
* Python 3.5.1 64bit (previously also tested using several 3.x versions) (also
tested with 32 bit, but with 3.4.2)
* Microsoft Visual Studio 2015 (earlier version of python also tested with
Visual Studio 2013)
We have tested with both the debug and release libs for 3.5 and 3.5.1 (those
available from python installer) with the same problem.
The following problem has also been tested on Unix and on MacOS where it works
as expected, the problem is only in Windows.
Problem description:
We have an application for scientific visualization in which we have enabled
scripting using embedded python. We have exposed quite some features of our
application to python using modules, this works perfectly fine.
The problem we have is that the application crashes as soon as a script is
trying to import some of python's default libraries, such as os or random. For
example, running the single line code "import os" produces a crash with the
following stacktrace:
http://pastebin.com/QpfGrMY0
We initialize the python interpreter using the following:
http://pastebin.com/uyx6gdMb
We have tried with and without the Py_NoSiteFlag = 1. When removing that flag
it works for some modules, for example "import os" works, but other modules,
for example "import random" crashes.
In the pastebin link above for the python initialization there are three lines
commented out, "importModule("os")","importModule("glog")" and
"importModule("random")", if I uncomment those lines the application will not
crash when importing those modules. But that is not a viable solution, we can't
import all possible modules that way.
The following piece of code is how we execute python code:
http://pastebin.com/HDUR2LKT
It only crashes for modules that exists , if I try to import a module that does
not at all exist (eg import unexisting_module) , it fails as expected with a
nice error message saying it does not exist.
I don't know if we are doing anything wrong in the initialization of the
interpreter or when running the script. We can run scripts that are not using
any modules or are only using the python modules we create in c++ to expose
functionality of our application.
Everything is working fine on MacOSX and Linux (tested with Ubuntu) which are
the platforms we support. We had a similar setup before using python 2.x
instead of 3.x which also worked without any problems.
The software i am talking about is using QT for gui and and OpenGL for hardware
accelerated graphics. But I have tested it in a minimal application where all
QT stuff were omitted and the same issue is still there.
The software is open source and can be downloaded/clones from
github.com/inviwo/inviwo.
The python initialization is done in
https://github.com/inviwo/inviwo-dev/blob/master/modules/python3/pyinviwo.cpp
and the script compiling and running is handled in
https://github.com/inviwo/inviwo-dev/blob/master/modules/python3/pythonscript.cpp
I have searched a lot on google and in the issue tracker for python but found
nothing that can answer my questions. So I hope that someone here can us solve
this.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Embedded Python and C Callback functions
doc.mefi...@gmail.com writes:
> I'm trying to debug and find my error. It goes wrong when:
> PyObject *v;
> v = va_arg(*p_va, PyObject *);
> if (v != NULL) {
> if (*(*p_format - 1) != 'N')
> Py_INCREF(v);
> }
>
> it tries to PyINCREF to my passed callback.
This indicates that it expects the "callback" to be a Python object.
Check the C-API specification. It should tell you which parameters
are expected to be Python objects and which C objects.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Embedded Python and C Callback functions
doc.mefi...@gmail.com schrieb am 07.06.2015 um 10:56: > And I can't use Cython, because I have C++ module, and I have to use it. That's not a valid reason. Cython supports C++ code just fine. http://docs.cython.org/src/userguide/wrapping_CPlusPlus.html Stefan -- https://mail.python.org/mailman/listinfo/python-list
Re: Embedded Python and C Callback functions
> >And I can't use Cython, because I have C++ module, and I have to use it. > >-- > >https://mail.python.org/mailman/listinfo/python-list > > Are you using Boost? > http://www.boost.org/doc/libs/1_58_0/libs/python/doc/ > > It handles lots of the setup for you. > > Laura No, I'm not using Boost. Pure Python.h -- https://mail.python.org/mailman/listinfo/python-list
Re: Embedded Python and C Callback functions
In a message of Sun, 07 Jun 2015 01:56:47 -0700, doc.mefi...@gmail.com writes: >And I can't use Cython, because I have C++ module, and I have to use it. >-- >https://mail.python.org/mailman/listinfo/python-list Are you using Boost? http://www.boost.org/doc/libs/1_58_0/libs/python/doc/ It handles lots of the setup for you. Laura -- https://mail.python.org/mailman/listinfo/python-list
Re: Embedded Python and C Callback functions
I'm trying to debug and find my error. It goes wrong when:
PyObject *v;
v = va_arg(*p_va, PyObject *);
if (v != NULL) {
if (*(*p_format - 1) != 'N')
Py_INCREF(v);
}
it tries to PyINCREF to my passed callback.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Embedded Python and C Callback functions
And I can't use Cython, because I have C++ module, and I have to use it. -- https://mail.python.org/mailman/listinfo/python-list
Re: Embedded Python and C Callback functions
No, myclass is not null. I think my style of passing arguments is wrong. -- https://mail.python.org/mailman/listinfo/python-list
Re: Embedded Python and C Callback functions
doc.mefi...@gmail.com writes: > Hi. I'm a newbie in python. But I want embed it in my C program. > > There is such method of my class: > @staticmethod > def install_instr_callback(callback): > # set hook for every change of PC > m68k.set_instr_hook_callback(callback) > > And in my C code there is such callback function: > static PyObject *my_instr_callback(PyObject *self); > > I want call install_instr_callback from C with specifying of my callback. I > do: > PyObject_CallMethod(myClass, "install_instr_callback", "(O)", > my_instr_callback); > But python writes me: [b]null argument to internal routine[/b]. If this output really comes from "PyObject_CallMethod", then some passed in argument is likely "null" - maybe "myclass"? The Python-C interface is complicated. It might be advicible to use e.g. "cython" to get the boilder plate code and some intricacies taken care of. I have done this with "dm.xmlsec.binding". You can find in its source also an example for the callback defined in Python and called from "C". -- https://mail.python.org/mailman/listinfo/python-list
Embedded Python and C Callback functions
Hi. I'm a newbie in python. But I want embed it in my C program. There is such method of my class: @staticmethod def install_instr_callback(callback): # set hook for every change of PC m68k.set_instr_hook_callback(callback) And in my C code there is such callback function: static PyObject *my_instr_callback(PyObject *self); I want call install_instr_callback from C with specifying of my callback. I do: PyObject_CallMethod(myClass, "install_instr_callback", "(O)", my_instr_callback); But python writes me: [b]null argument to internal routine[/b]. How can I fix this problem? -- https://mail.python.org/mailman/listinfo/python-list
Embedded Python. Debug Version and _ctypes
Hi! I'm trying to use embedding of Python in my program. Simple C-program, compiled in Debug, that uses py-script that just imports "ctypes" gives me an error about "no module named "_ctypes". How to compile python lib in Visual Studio statically with ctypes support? Or how to use shared ctypes lib in debug mode? -- https://mail.python.org/mailman/listinfo/python-list
Re: exporting c_variable to embedded python module
I want to access c-side global variables from the python side. -- https://mail.python.org/mailman/listinfo/python-list
Re: exporting c_variable to embedded python module
On Fri, Jan 9, 2015 at 8:03 AM, wrote: > I'm running a python script loaded via PyImport_Import in my C++ program on > Linux. Is there any way I can pass a value from the c-code to the loaded > python module? To answer this question, first think about how you would like to see that value on the Python side. Does your C code call a function in that module? If so, it can pass it a parameter. Or do you want the top-level import to be aware of something? It might be best to put something into another module, maybe even builtins. What makes sense from the Python end? Pretty much anything can be done. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
exporting c_variable to embedded python module
I'm running a python script loaded via PyImport_Import in my C++ program on Linux. Is there any way I can pass a value from the c-code to the loaded python module? -- https://mail.python.org/mailman/listinfo/python-list
Re: Embedded python 'scripting engine' inside Python app
On Tue, Nov 25, 2014 at 6:38 PM, Patrick Stinson wrote: > Thanks for the stories in this and the other thread. I love these interesting > problems that push the limits :) I agree. How boring is life when we never push the limits! ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Embedded python 'scripting engine' inside Python app
> On Nov 23, 2014, at 4:57 AM, Chris Angelico wrote: > > On Mon, Nov 24, 2014 at 12:20 AM, Patrick Stinson > wrote: >> I think this is the way I’ll take it, and for all the same reasons. The only >> way they can break it is if they really want to. I guess anything other >> Franken-apps would be interesting to hear about too. And I’ll still stick it >> on the app store. >> > > (Please note, the convention on this list/newsgroup - as with most > tech lists - is to put your text underneath what you're quoting - > so-called "bottom-posting" or "interleaved" style.) > > Yep. I got in some trouble for doing this at my last salaried > employment, because the boss couldn't get his head around the idea > that an XKCD 936-compliant password is secure enough for a code > executor, and that the absence of a code executor makes debugging much > harder. (In the end, I just renamed it to "calculator" so it wasn't so > obvious, and left it there. Debugging is important.) In my open-source > MUD client, Gypsum, there's a code executor built-in, which I use > regularly as a simple calculator, and also to manipulate the program's > internals. Some examples: > >> /x 123+456*789 > 359907 >> /x asin(.5)*180/3.14159 > 30.253399374 >> /x "Test "*5 + "Haha" > "Test Test Test Test Test Haha" >> /x window.mainwindow.iconify() > GTK2.Window > (and the main window has just been minimized, aka iconified) Thanks for the stories in this and the other thread. I love these interesting problems that push the limits :) > > Basically, any expression that I type after "/x" will be parsed and > executed as code. There's no sandboxing at all; it's executed in its > own globals() and locals(), but if it wants to tinker with external > state, it's free to do that. Since all the code is there for the user > to peruse anyway, I don't distinguish between "this you may do" and > "this you may not do", but instead between "this is pledged to be > supported throughout this major version" and "this is undocumented, > use at your own risk" - on the understanding that most of the latter > category is actually pretty stable, and can happily be used anyway. > > Interpreted languages tend to make this kind of thing fairly easy, and > on the day you have a weird error that you can't figure out, you will > be no end of glad you have this. No triggering a core dump and then > doing a post-mortem, just examine state live, using the language's > natural syntax. > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Embedded python 'scripting engine' inside Python app
I think this is the way I’ll take it, and for all the same reasons. The only way they can break it is if they really want to. I guess anything other Franken-apps would be interesting to hear about too. And I’ll still stick it on the app store. > On Nov 23, 2014, at 1:35 AM, Chris Angelico wrote: > > On Sun, Nov 23, 2014 at 9:28 PM, Patrick Stinson > wrote: >> Thanks for your great reply. I even augmented the reloading with the same >> dict by clearing all of the non-standard symbols from the dict. This >> effectively resets the dict: > > You may as well start with an empty dict and then pick up the few > things you want, I think. > >> Is there a better and more secure way to do the python-within-python in >> order allow users to automate your app? > > More secure? Basically no. You could push the inner script into a > separate process, but I would recommend simply acknowledging the > insecurity. Embrace the lack of security and call it a debugging > feature - make it possible to introspect, control, manipulate internal > structures. Feature, not flaw. :) > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Embedded python 'scripting engine' inside Python app
Thanks for your great reply. I even augmented the reloading with the same dict
by clearing all of the non-standard symbols from the dict. This effectively
resets the dict:
# try to clear out the module by deleting all global refs
d = self.module.__dict__
for k in dict(d).keys():
if not k in ['__spec__', '__name__', '__loader__', '__package__',
'__doc__', '__builtins__']:
del d[k]
self.module.__dict__['sendMessage'] = self.sendMessage
try:
exec(self.source, self.module.__dict__)
except Exception:
import traceback
traceback.print_exc(file=sys.stdout)
Is there a better and more secure way to do the python-within-python in order
allow users to automate your app?
Thanks!
> On Nov 23, 2014, at 12:24 AM, Chris Angelico wrote:
>
> On Sun, Nov 23, 2014 at 4:48 PM, Patrick Stinson
> wrote:
>> I am writing a python app (using PyQt, but that’s not important here), and
>> want my users to be able to write their own scripts to automate the app’s
>> functioning using an engine API hat I expose. I have extensive experience
>> doing this in a C++ app with the CPython api, but have no idea how to do
>> this outside of calling exec() from with in Python :)
>>
>> Ideally their script would compile when the source changes and retain it’s
>> state and respond to callbacks from the api object. It appears this won’t
>> work with exec() because the script’s definitions and state disappear as
>> soon as the exec() call is complete, and the script doesn’t seem to be able
>> to access it’s own defined functions and classes.
>>
>> Thoughts? Fun stuff!
>
> First off, a cautionary note: Security-wise, this is absolutely
> equivalent to your users editing your source code. Be aware that
> you're giving them complete control.
>
> What you should be able to do is exec the script in a specific global
> dictionary. Here's some example code (Python 3.4):
>
script = """
> def init():
>print("Initializing")
>
> def on_some_event(status):
>trigger_some_action("Status is now "+status)
> """
def trigger_some_action(msg):
>print("Action triggered.",msg)
globl = {"trigger_some_action":trigger_some_action}
exec(script,globl)
globl["init"]()
> Initializing
globl["on_some_event"]("Something happened")
> Action triggered. Status is now Something happened
>
> You can provide globals like this, or you can create an importable
> module for the scripts to call on. (Or both. Create a module, and
> pre-import it automatically.) The script defines functions with
> specific names and/or calls your functions to register hooks; you can
> reach into the globals to trigger functions.
>
> One way to handle updates to the code would be to exec it in the same
> globals dictionary. That has its complexities (for instance, if you
> rename a function, the old version will still exist under the old name
> unless you explicitly del it), but it can be very convenient.
> Alternatively, you could have the new version run in a new dictionary,
> but important state can be kept in separate dictionaries. That's how I
> manage things with a Pike program - all code gets reloaded cleanly,
> but retained state is stored separately in a mutable object that gets
> passed around.
>
> There are several ways this sort of thing can be done. It's reasonably
> easy, as long as you take a bit of care across the reload boundaries.
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: Embedded python 'scripting engine' inside Python app
On Mon, Nov 24, 2014 at 12:20 AM, Patrick Stinson wrote: > I think this is the way I’ll take it, and for all the same reasons. The only > way they can break it is if they really want to. I guess anything other > Franken-apps would be interesting to hear about too. And I’ll still stick it > on the app store. > (Please note, the convention on this list/newsgroup - as with most tech lists - is to put your text underneath what you're quoting - so-called "bottom-posting" or "interleaved" style.) Yep. I got in some trouble for doing this at my last salaried employment, because the boss couldn't get his head around the idea that an XKCD 936-compliant password is secure enough for a code executor, and that the absence of a code executor makes debugging much harder. (In the end, I just renamed it to "calculator" so it wasn't so obvious, and left it there. Debugging is important.) In my open-source MUD client, Gypsum, there's a code executor built-in, which I use regularly as a simple calculator, and also to manipulate the program's internals. Some examples: > /x 123+456*789 359907 > /x asin(.5)*180/3.14159 30.253399374 > /x "Test "*5 + "Haha" "Test Test Test Test Test Haha" > /x window.mainwindow.iconify() GTK2.Window (and the main window has just been minimized, aka iconified) Basically, any expression that I type after "/x" will be parsed and executed as code. There's no sandboxing at all; it's executed in its own globals() and locals(), but if it wants to tinker with external state, it's free to do that. Since all the code is there for the user to peruse anyway, I don't distinguish between "this you may do" and "this you may not do", but instead between "this is pledged to be supported throughout this major version" and "this is undocumented, use at your own risk" - on the understanding that most of the latter category is actually pretty stable, and can happily be used anyway. Interpreted languages tend to make this kind of thing fairly easy, and on the day you have a weird error that you can't figure out, you will be no end of glad you have this. No triggering a core dump and then doing a post-mortem, just examine state live, using the language's natural syntax. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Embedded python 'scripting engine' inside Python app
Chris Angelico schrieb am 23.11.2014 um 11:35: > On Sun, Nov 23, 2014 at 9:28 PM, Patrick Stinson wrote: >> Is there a better and more secure way to do the python-within-python in >> order allow users to automate your app? > > More secure? Basically no. You could push the inner script into a > separate process, but I would recommend simply acknowledging the > insecurity. Embrace the lack of security and call it a debugging > feature - make it possible to introspect, control, manipulate internal > structures. Feature, not flaw. :) As the author of Lupa, I know that some people have successfully and safely embedded Lua in Python as a simple, small and object-oriented scripting language in a sandbox. https://pypi.python.org/pypi/lupa The overall syntax isn't quite as great as that of Python, but as long as you mostly stick to "here's a bunch of functions you can call" or "here's an object, go and call some methods on it" kind of APIs, there isn't all that much of a difference either. Stefan -- https://mail.python.org/mailman/listinfo/python-list
Re: Embedded python 'scripting engine' inside Python app
On Sun, Nov 23, 2014 at 9:28 PM, Patrick Stinson wrote: > Thanks for your great reply. I even augmented the reloading with the same > dict by clearing all of the non-standard symbols from the dict. This > effectively resets the dict: You may as well start with an empty dict and then pick up the few things you want, I think. > Is there a better and more secure way to do the python-within-python in > order allow users to automate your app? More secure? Basically no. You could push the inner script into a separate process, but I would recommend simply acknowledging the insecurity. Embrace the lack of security and call it a debugging feature - make it possible to introspect, control, manipulate internal structures. Feature, not flaw. :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Embedded python 'scripting engine' inside Python app
On Sun, Nov 23, 2014 at 4:48 PM, Patrick Stinson wrote:
> I am writing a python app (using PyQt, but that’s not important here), and
> want my users to be able to write their own scripts to automate the app’s
> functioning using an engine API hat I expose. I have extensive experience
> doing this in a C++ app with the CPython api, but have no idea how to do this
> outside of calling exec() from with in Python :)
>
> Ideally their script would compile when the source changes and retain it’s
> state and respond to callbacks from the api object. It appears this won’t
> work with exec() because the script’s definitions and state disappear as soon
> as the exec() call is complete, and the script doesn’t seem to be able to
> access it’s own defined functions and classes.
>
> Thoughts? Fun stuff!
First off, a cautionary note: Security-wise, this is absolutely
equivalent to your users editing your source code. Be aware that
you're giving them complete control.
What you should be able to do is exec the script in a specific global
dictionary. Here's some example code (Python 3.4):
>>> script = """
def init():
print("Initializing")
def on_some_event(status):
trigger_some_action("Status is now "+status)
"""
>>> def trigger_some_action(msg):
print("Action triggered.",msg)
>>> globl = {"trigger_some_action":trigger_some_action}
>>> exec(script,globl)
>>> globl["init"]()
Initializing
>>> globl["on_some_event"]("Something happened")
Action triggered. Status is now Something happened
You can provide globals like this, or you can create an importable
module for the scripts to call on. (Or both. Create a module, and
pre-import it automatically.) The script defines functions with
specific names and/or calls your functions to register hooks; you can
reach into the globals to trigger functions.
One way to handle updates to the code would be to exec it in the same
globals dictionary. That has its complexities (for instance, if you
rename a function, the old version will still exist under the old name
unless you explicitly del it), but it can be very convenient.
Alternatively, you could have the new version run in a new dictionary,
but important state can be kept in separate dictionaries. That's how I
manage things with a Pike program - all code gets reloaded cleanly,
but retained state is stored separately in a mutable object that gets
passed around.
There are several ways this sort of thing can be done. It's reasonably
easy, as long as you take a bit of care across the reload boundaries.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Embedded python 'scripting engine' inside Python app
I am writing a python app (using PyQt, but that’s not important here), and want my users to be able to write their own scripts to automate the app’s functioning using an engine API hat I expose. I have extensive experience doing this in a C++ app with the CPython api, but have no idea how to do this outside of calling exec() from with in Python :) Ideally their script would compile when the source changes and retain it’s state and respond to callbacks from the api object. It appears this won’t work with exec() because the script’s definitions and state disappear as soon as the exec() call is complete, and the script doesn’t seem to be able to access it’s own defined functions and classes. Thoughts? Fun stuff! Thanks! -- https://mail.python.org/mailman/listinfo/python-list
Re: can't get utf8 / unicode strings from embedded python
Thank you for your thoughtful and thorough response. I now understand much better what you (and apparently the others) were warning me against and I will certainly consider that moving forward. I very much appreciate your help as I learn about python and embedding and all these crazy encoding problems. > What do kids have to do with this? When a person has children, they quickly learn that the best way to deal with some one who seems to be not listening or having a tantrum: show understanding and compassion, restraint and patience, as you, in the most neutral way that you can, gently bit firmly guide said person back on track. You learn that if you instead express your frustration at said person, that it never, ever helps the situation, and only causes more hurt to be spread around to the very people you are ostensibly attempting to help. > Are you an adult or a child? Perhaps my comment was lost in translation, but this is rather the question that I was obliquely asking you. *wink right back* In any case I thank you for your help, which has in fact been quite great! My demo script is working, and I know now to properly advise my script writers regarding how to properly encode strings. -- http://mail.python.org/mailman/listinfo/python-list
Re: can't get utf8 / unicode strings from embedded python
On Tue, 27 Aug 2013 22:57:45 -0700, David M. Cotter wrote: > I am very sorry that I have offended you to such a degree you feel it > necessary to publicly eviscerate me. You know David, you are right. I did over-react. And I apologise for that. I am sorry, I was excessively confrontational. (Although I think "eviscerate" is a bit strong.) Putting aside my earlier sarcasm, the basic message remains the same: Python byte strings are not designed to work with Unicode characters, and if they do work, it is an accident, not defined behaviour. > Perhaps I could have worded it like this: "So far I have not seen any > troubles including unicode characters in my strings, they *seem* to be > fine for my use-case. What kind of trouble has been seen with this by > others?" Exactly the same sort of trouble you were having earlier when you were inadvertently decoding the source file as MacRoman rather than UTF-8. Mojibake, garbage characters in your text, corrupted data. http://en.wikipedia.org/wiki/Mojibake The point is, you might not see these errors, because by accident all the relevant factors conspire to give you the correct result. You might test it on a Mac and on Windows and it all works well. You might even test it on a dozen different machines, and it works fine on all of them. But since you're relying on an accident of implementation, none of this is guaranteed. And then in eighteen months time, *something* changes -- a minor update to Python, a different version of Mac OS/X, an unusual Registry setting in Windows, who knows what?, and all of a sudden the factors no longer line up to give you the correct results and it all comes tumbling down in a big stinking mess. If you are lucky you will get a nice clear exception telling you something is broken, but more likely you'll just get corrupted data and mojibake and you, or the poor guy who maintains the code after you, will have no idea why. And you'll probably come here asking for our help to solve it. If you came back and said "I tried it with the u prefix, and it broke a bunch of other code, and I don't have time to fix it now so I'm reverting to the u-less byte string form" I wouldn't *like* it but I could *accept* it as one of those sub-optimal compromises people make in Real Life. I've done the same thing myself, we probably all have: written code we knew was broken, but fixing it was too hard or too low a priority. > Really, I wonder why you are so angry at me for having made a mistake? > I'm going to guess that you don't have kids. What do kids have to do with this? Are you an adult or a child? *wink* You didn't offend me so much as frustrate me. You had multiple people telling you the same thing, don't embed Unicode characters in a byte string, but you choose to not just ignore them but effectively declare that they were all wrong to give that advice, not just the people here but essentially the entire Python development community responsible for adding Unicode strings to the language. Can you blame me for feeling that your reply seemed rather arrogant? In any case, I'm glad you responded with a little more restraint than I did, and I hope you can see my point of view and hopefully I haven't soured you on this forum. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: can't get utf8 / unicode strings from embedded python
I am very sorry that I have offended you to such a degree you feel it necessary to publicly eviscerate me. Perhaps I could have worded it like this: "So far I have not seen any troubles including unicode characters in my strings, they *seem* to be fine for my use-case. What kind of trouble has been seen with this by others?" Really, I wonder why you are so angry at me for having made a mistake? I'm going to guess that you don't have kids. -- http://mail.python.org/mailman/listinfo/python-list
Re: can't get utf8 / unicode strings from embedded python
On Tue, 27 Aug 2013 15:21:00 -0700, David M. Cotter wrote: > i am already doing (3), and all is working perfectly. bytestring > literals are fine, i'm not sure what this trouble is that you speak of. Neither is anyone else, because your post is completely devoid of any context. Who are you talking to? Wait, let me see if I can peer into my crystal ball and see if the spirits tell me what you are talking about... I see a post... no, repeated posts, by many people, telling you not to embed Unicode characters in Python 2.x plain byte strings... You know what? You obviously know so much more about Unicode and Python than the entire Python community, you must be right. There is no possible way that misusing byte strings in this manner could possibly go wrong. Since byte strings literals containing Unicode data are "fine", it was clearly a complete waste of time to introduce Unicode strings in the first place. Why bother using the official interface designed to work correctly with Unicode, when you can rely on an accident of implementation that just happens to work correctly in your environment but no guarantee it will work correctly anywhere else? What could *possibly* go wrong by relying on code working by accident like this? -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: can't get utf8 / unicode strings from embedded python
i am already doing (3), and all is working perfectly. bytestring literals are fine, i'm not sure what this trouble is that you speak of. note that i'm not using PyRun_AnyFile(), i'm loading the script myself, assumed as utf8 (which was my original problem, i had assumed it was macRoman), then calling PyRun_SimpleString(). it works flawlessly now, on both mac and windows. -- http://mail.python.org/mailman/listinfo/python-list
Re: can't get utf8 / unicode strings from embedded python
On 25/08/2013 23:32, David M. Cotter wrote: i got it!! OMG! so sorry for the confusion, but i learned a lot, and i can share the result: the CORRECT code *was* what i had assumed. the Python side has always been correct (no need to put "u" in front of strings, it is known that the bytes are utf8 bytes) it was my "run script" function which read in the file. THAT was what was "reinterpreting" the utf8 bytes as macRoman (on both platforms). correct code below: When working with Unicode, what you should be doing is: 1. Specifying the encoding line in the special comment. 2. Setting the encoding of the source file. 3. Using Unicode string literals in the source file. You're doing (1) and (2), but not (3). If you want to pass UTF-8 to the the C++, then encode the Unicode string to bytes when you pass it. Using bytestring literals and relying on the source file being UTF-8, like you doing, is just asking for trouble, as you've found out! :-) -- http://mail.python.org/mailman/listinfo/python-list
Re: can't get utf8 / unicode strings from embedded python
i got it!! OMG! so sorry for the confusion, but i learned a lot, and i can
share the result:
the CORRECT code *was* what i had assumed. the Python side has always been
correct (no need to put "u" in front of strings, it is known that the bytes are
utf8 bytes)
it was my "run script" function which read in the file. THAT was what was
"reinterpreting" the utf8 bytes as macRoman (on both platforms). correct code
below:
SuperString ScPyObject::GetAs_String()
{
SuperString str;
if (PyUnicode_Check(i_objP)) {
ScPyObject utf8Str(PyUnicode_AsUTF8String(i_objP));
str = utf8Str.GetAs_String();
} else {
// calling "uc" on this means "assume this is utf8"
str.Set(uc(PyString_AsString(i_objP)));
}
return str;
}
PyObject* PyString_FromString(const SuperString& str)
{
return PyString_FromString(str.utf8Z());
}
--
http://mail.python.org/mailman/listinfo/python-list
Re: can't get utf8 / unicode strings from embedded python
fair enough. I can provide further proof of strangeness.
here is my latest script: this is saved on disk as a UTF8 encoded file, and
when viewing as UTF8, it shows the correct characters.
==
# -*- coding: utf-8 -*-
import time, kjams, kjams_lib
def log_success(msg, successB, str):
if successB:
print msg + " worked: " + str
else:
print msg + "failed: " + str
def do_test(orig_str):
cmd_enum = kjams.enum_cmds()
print "---"
print "Original string: " + orig_str
print "converting..."
oldstr = orig_str;
newstr = kjams_lib.do_command(cmd_enum.kScriptCommand_Unicode_Test,
oldstr)
log_success("first", oldstr == newstr, newstr);
oldstr = unicode(orig_str, "UTF-8")
newstr = kjams_lib.do_command(cmd_enum.kScriptCommand_Unicode_Test,
oldstr)
newstr = unicode(newstr, "UTF-8")
log_success("second", oldstr == newstr, newstr);
oldstr = unicode(orig_str, "UTF-8")
oldstr.encode("UTF-8")
newstr = kjams_lib.do_command(cmd_enum.kScriptCommand_Unicode_Test,
oldstr)
newstr = unicode(newstr, "UTF-8")
log_success("third", oldstr == newstr, newstr);
print "---"
def main():
do_test("frøânçïé")
do_test("控件")
#-
if __name__ == "__main__":
main()
==
and the latest results:
20: ---
20: Original string: frøânçïé
20: converting...
20: first worked: frøânçïé
20: second worked: frøânçïé
20: third worked: frøânçïé
20: ---
20: ---
20: Original string: 控件
20: converting...
20: first worked: 控件
20: second worked: 控件
20: third worked: 控件
20: ---
now, given the C++ source code, this should NOT work, given that i'm doing some
crazy re-coding of the bytes.
so, you see, it does not matter whether i pass "unicode" strings or regular
"strings", they all translate to the same, weird macroman.
for completeness, here is the C++ code that the script calls:
===
case kScriptCommand_Unicode_Test: {
pyArg = iterP.NextArg_OrSyntaxError();
if (pyArg.get()) {
SuperString str = pyArg.GetAs_String();
resultObjP = PyString_FromString(str);
}
break;
}
===
--
http://mail.python.org/mailman/listinfo/python-list
Re: can't get utf8 / unicode strings from embedded python
On 8/25/2013 1:57 PM, David M. Cotter wrote: i'm sorry this is so confusing, let me try to re-state the problem in as clear a way as i can. I have a C++ program, with very well tested unicode support. All logging is done in utf8. I have conversion routines that work flawlessly, so i can assure you there is nothing wrong with logging and unicode support in the underlying program. I am embedding python 2.7 into the program, and extending python with routines in my C++ program. If you want 'well-tested' (correct) unicode support from Python, use 3.3. Unicode in 2.x is somewhat buggy and definitely flakey. The first fix was to make unicode *the* text type, in 3.0. The second was to redesign the internals in 3.3. It is possible that 2.7 is too broken for what you want to do. I have a script, encoded in utf8, and *marked* as utf8 with this line: # -*- coding: utf-8 -*- In that script, i have inline unicode text. The example scripts that you posted pictures of do *not* have unicode text. They have bytestring literals with (encoded) non-ascii chars inside them. This is not a great idea. I am not sure what bytes you end up with. Apparently, not what you expect. To make them 'unicode text', you must prepend the literals with 'u'. Didn't someone say this before? When I pass that text to my C++ program, the Python interpreter decides that these bytes are macRoman, and handily "converts" them to unicode. To compensate, i must "convert" these "macRoman" characters encoded as utf8, back to macRoman, then "interpret" them as utf8. In this way i can recover the original unicode. When i return a unicode string back to python, i must do the reverse so that Python gets back what it expects. This is not related to printing, or sys.stdout, it does happen with that too but focusing on that is a red-herring. Let's focus on just passing a string into C++ then back out. This would all actually make sense IF my script was marked as being "macRoman" even tho i entered UTF8 Characters, but that is not the case. Let's prove my statements. Here is the script, *interpreted* as MacRoman: http://karaoke.kjams.com/screenshots/bugs/python_unicode/script_as_macroman.png Why are you posting pictures of code, instead of the (runnable) code itself, as you did with C code? -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: can't get utf8 / unicode strings from embedded python
2013/8/25 David M. Cotter :
> i'm sorry this is so confusing, let me try to re-state the problem in as
> clear a way as i can.
>
> I have a C++ program, with very well tested unicode support. All logging is
> done in utf8. I have conversion routines that work flawlessly, so i can
> assure you there is nothing wrong with logging and unicode support in the
> underlying program.
>
> I am embedding python 2.7 into the program, and extending python with
> routines in my C++ program.
>
> I have a script, encoded in utf8, and *marked* as utf8 with this line:
> # -*- coding: utf-8 -*-
>
> In that script, i have inline unicode text. When I pass that text to my C++
> program, the Python interpreter decides that these bytes are macRoman, and
> handily "converts" them to unicode. To compensate, i must "convert" these
> "macRoman" characters encoded as utf8, back to macRoman, then "interpret"
> them as utf8. In this way i can recover the original unicode.
>
> When i return a unicode string back to python, i must do the reverse so that
> Python gets back what it expects.
>
> This is not related to printing, or sys.stdout, it does happen with that too
> but focusing on that is a red-herring. Let's focus on just passing a string
> into C++ then back out.
>
> This would all actually make sense IF my script was marked as being
> "macRoman" even tho i entered UTF8 Characters, but that is not the case.
>
> Let's prove my statements. Here is the script, *interpreted* as MacRoman:
> http://karaoke.kjams.com/screenshots/bugs/python_unicode/script_as_macroman.png
>
> and here it is again *interpreted* as utf8:
> http://karaoke.kjams.com/screenshots/bugs/python_unicode/script_as_utf8.png
>
> here is the string conversion code:
>
> SuperString ScPyObject::GetAs_String()
> {
> SuperString str;// underlying format of
> SuperString is unicode
>
> if (PyUnicode_Check(i_objP)) {
> ScPyObject
> utf8Str(PyUnicode_AsUTF8String(i_objP));
>
> str = utf8Str.GetAs_String();
> } else {
> const UTF8Char *bytes_to_interpetZ =
> uc(PyString_AsString(i_objP));
>
> // the "Set" call *interprets*, does not *convert*
> str.Set(bytes_to_interpetZ, kCFStringEncodingUTF8);
>
> // str is now unicode characters which *represent*
> macRoman characters
> // so *convert* these to actual macRoman
>
> // fyi: Update_utf8 means "convert to this encoding and
> // store the resulting bytes in the variable named "utf8"
> str.Update_utf8(kCFStringEncodingMacRoman);
>
> // str is now unicode characters converted from macRoman
> // so *reinterpret* them as UTF8
>
> // FYI, we're just taking the pure bytes that are stored
> in the utf8 variable
> // and *interpreting* them to this encoding
> bytes_to_interpetZ = str.utf8().c_str();
>
> str.Set(bytes_to_interpetZ, kCFStringEncodingUTF8);
> }
>
> return str;
> }
>
> PyObject* PyString_FromString(const SuperString& str)
> {
> SuperString localStr(str);
>
> // localStr is the real, actual unicode string
> // but we must *interpret* it as macRoman, then take these
> "macRoman" characters
> // and "convert" them to unicode for Python to "get it"
> const UTF8Char *bytes_to_interpetZ = localStr.utf8().c_str();
>
> // take the utf8 bytes (actual utf8 prepresentation of string)
> // and say "no, these bytes are macRoman"
> localStr.Set(bytes_to_interpetZ, kCFStringEncodingMacRoman);
>
> // okay so now we have unicode of MacRoman characters (!?)
> // return the underlying utf8 bytes of THAT as our string
> return PyString_FromString(localStr.utf8Z());
> }
>
> And here is the results from running the script:
>18: ---
>18: Original string: frøânçïé
>18: converting...
>18: it worked: frøânçïé
>18: ---
>18: ---
>18: Original string: 控件
>18: converting...
>18: it worked: 控件
>18: ---
>
> Now the thing that absolutely utterly baffles me (if i'm not baffled enough)
> is that i get the EXACT same results on both Mac and Windows. Why do they
> both insist on interpreting my script's bytes as MacRoman?
> --
> http://mail.python.org/mailman/listinfo/python-list
Hi,
unfortunately, I don't have experience with embedding python and C++,
but he python (for python 2) part seems to be missing the u prefix in
the unicode literals.
like
u"frøânçïé"
Is the c++ part prepared for python unicode object, or does it require
utf-8 encoded string (or the respective bytes)?
would
oldstr.encode("
Re: can't get utf8 / unicode strings from embedded python
i'm sorry this is so confusing, let me try to re-state the problem in as clear
a way as i can.
I have a C++ program, with very well tested unicode support. All logging is
done in utf8. I have conversion routines that work flawlessly, so i can assure
you there is nothing wrong with logging and unicode support in the underlying
program.
I am embedding python 2.7 into the program, and extending python with routines
in my C++ program.
I have a script, encoded in utf8, and *marked* as utf8 with this line:
# -*- coding: utf-8 -*-
In that script, i have inline unicode text. When I pass that text to my C++
program, the Python interpreter decides that these bytes are macRoman, and
handily "converts" them to unicode. To compensate, i must "convert" these
"macRoman" characters encoded as utf8, back to macRoman, then "interpret" them
as utf8. In this way i can recover the original unicode.
When i return a unicode string back to python, i must do the reverse so that
Python gets back what it expects.
This is not related to printing, or sys.stdout, it does happen with that too
but focusing on that is a red-herring. Let's focus on just passing a string
into C++ then back out.
This would all actually make sense IF my script was marked as being "macRoman"
even tho i entered UTF8 Characters, but that is not the case.
Let's prove my statements. Here is the script, *interpreted* as MacRoman:
http://karaoke.kjams.com/screenshots/bugs/python_unicode/script_as_macroman.png
and here it is again *interpreted* as utf8:
http://karaoke.kjams.com/screenshots/bugs/python_unicode/script_as_utf8.png
here is the string conversion code:
SuperString ScPyObject::GetAs_String()
{
SuperString str;// underlying format of
SuperString is unicode
if (PyUnicode_Check(i_objP)) {
ScPyObject utf8Str(PyUnicode_AsUTF8String(i_objP));
str = utf8Str.GetAs_String();
} else {
const UTF8Char *bytes_to_interpetZ =
uc(PyString_AsString(i_objP));
// the "Set" call *interprets*, does not *convert*
str.Set(bytes_to_interpetZ, kCFStringEncodingUTF8);
// str is now unicode characters which *represent*
macRoman characters
// so *convert* these to actual macRoman
// fyi: Update_utf8 means "convert to this encoding and
// store the resulting bytes in the variable named "utf8"
str.Update_utf8(kCFStringEncodingMacRoman);
// str is now unicode characters converted from macRoman
// so *reinterpret* them as UTF8
// FYI, we're just taking the pure bytes that are stored
in the utf8 variable
// and *interpreting* them to this encoding
bytes_to_interpetZ = str.utf8().c_str();
str.Set(bytes_to_interpetZ, kCFStringEncodingUTF8);
}
return str;
}
PyObject* PyString_FromString(const SuperString& str)
{
SuperString localStr(str);
// localStr is the real, actual unicode string
// but we must *interpret* it as macRoman, then take these
"macRoman" characters
// and "convert" them to unicode for Python to "get it"
const UTF8Char *bytes_to_interpetZ = localStr.utf8().c_str();
// take the utf8 bytes (actual utf8 prepresentation of string)
// and say "no, these bytes are macRoman"
localStr.Set(bytes_to_interpetZ, kCFStringEncodingMacRoman);
// okay so now we have unicode of MacRoman characters (!?)
// return the underlying utf8 bytes of THAT as our string
return PyString_FromString(localStr.utf8Z());
}
And here is the results from running the script:
18: ---
18: Original string: frøânçïé
18: converting...
18: it worked: frøânçïé
18: ---
18: ---
18: Original string: 控件
18: converting...
18: it worked: 控件
18: ---
Now the thing that absolutely utterly baffles me (if i'm not baffled enough) is
that i get the EXACT same results on both Mac and Windows. Why do they both
insist on interpreting my script's bytes as MacRoman?
--
http://mail.python.org/mailman/listinfo/python-list
Re: can't get utf8 / unicode strings from embedded python
On Sat, Aug 24, 2013, at 12:47, David M. Cotter wrote: > > What _are_ you using? > i have scripts in a file, that i am invoking into my embedded python > within a C++ program. there is no terminal involved. the "print" > statement has been redirected (via sys.stdout) to my custom print class, > which does not specify "encoding", so i tried the suggestion above to set > it: That doesn't answer my real question. What does your "custom print class" do with the text? -- http://mail.python.org/mailman/listinfo/python-list
Re: can't get utf8 / unicode strings from embedded python
On Sat, Aug 24, 2013 at 9:47 AM, David M. Cotter wrote: > > > What _are_ you using? > i have scripts in a file, that i am invoking into my embedded python within a > C++ program. there is no terminal involved. the "print" statement has been > redirected (via sys.stdout) to my custom print class, which does not specify > "encoding", so i tried the suggestion above to set it: > > static const char *s_RedirectScript = > "import " kEmbeddedModuleName "\n" > "import sys\n" > "\n" > "class CustomPrintClass:\n" > " def write(self, stuff):\n" > " " kEmbeddedModuleName "." kCustomPrint "(stuff)\n" > "class CustomErrClass:\n" > " def write(self, stuff):\n" > " " kEmbeddedModuleName "." kCustomErr "(stuff)\n" > "sys.stdout = CustomPrintClass()\n" > "sys.stderr = CustomErrClass()\n" > "sys.stdout.encoding = 'UTF-8'\n" > "sys.stderr.encoding = 'UTF-8'\n"; > > > but it didn't help. > > I'm still getting back a string that is a utf-8 string of characters that, if > converted to "macRoman" and then interpreted as UTF8, shows the original, > correct string. who is specifying macRoman, and where, and how do i tell > whoever that is that i really *really* want utf8? > -- If you're running this from a C++ program, then you aren't getting back characters. You're getting back bytes. If you treat them as UTF-8, they'll work properly. The only thing wrong is the text editor you're using to open the file afterwards- since you aren't specifying an encoding, it's assuming MacRoman. You can try putting the UTF-8 BOM (it's not really a BOM) at the front of the file- the bytes 0xEF 0xBB 0xBF are used by some editors to identify a file as UTF-8. -- http://mail.python.org/mailman/listinfo/python-list
Re: can't get utf8 / unicode strings from embedded python
Le samedi 24 août 2013 18:47:19 UTC+2, David M. Cotter a écrit :
> > What _are_ you using?
>
> i have scripts in a file, that i am invoking into my embedded python within a
> C++ program. there is no terminal involved. the "print" statement has been
> redirected (via sys.stdout) to my custom print class, which does not specify
> "encoding", so i tried the suggestion above to set it:
>
>
>
> static const char *s_RedirectScript =
>
> "import " kEmbeddedModuleName "\n"
>
> "import sys\n"
>
> "\n"
>
> "class CustomPrintClass:\n"
>
> " def write(self, stuff):\n"
>
> " " kEmbeddedModuleName "." kCustomPrint "(stuff)\n"
>
> "class CustomErrClass:\n"
>
> " def write(self, stuff):\n"
>
> " " kEmbeddedModuleName "." kCustomErr "(stuff)\n"
>
> "sys.stdout = CustomPrintClass()\n"
>
> "sys.stderr = CustomErrClass()\n"
>
> "sys.stdout.encoding = 'UTF-8'\n"
>
> "sys.stderr.encoding = 'UTF-8'\n";
>
>
>
>
>
> but it didn't help.
>
>
>
> I'm still getting back a string that is a utf-8 string of characters that, if
> converted to "macRoman" and then interpreted as UTF8, shows the original,
> correct string. who is specifying macRoman, and where, and how do i tell
> whoever that is that i really *really* want utf8?
Always encode a "unicode" into the coding of the "system"
which will host it.
Adapting the hosting system to your "unicode" (encoded
unicode) is not a valid solution. A non sense.
sys.std***.encodings do nothing. They only give you
information about the coding of the hosting system.
The "system" can be anything, a db, a terminal, a gui, ...
Shortly, your "writer" should encode your "stuff"
to your "host" in a adequate way. It is up to you to
manage coherence. If your passive "writer" support only one
coding, adapt "stuff", if "stuff" lives in its own coding
(due to c++ ?) adapt your "writer".
Example from my interactive interpreter. It is in Python 3,
not important, basically the job is the same in Python 2.
This interpreter has the capability to support many codings,
and the coding of this host system can be changed on the
fly.
A commented session.
By default, a string, type str, is a unicode. The
host accepts "unicode". So, by default the sys.stdout
coding is ''.
>>> sys.stdout.encoding = ''
>>> print("frøânçïé")
frøânçïé
>>>
Setting the host to utf-8 and printing the above string gives
"something", but encoding into utf-8 works fine.
>>> sys.stdout.encoding = 'utf-8'
>>> sys.stdout.encoding
'utf-8'
>>> print("frøânçïé")
frøânçïé
>>> print("frøânçïé".encode('utf-8'))
'frøânçïé'
Setting the host to 'mac-roman' works fine too,
as long it is properly encoded!
>>> sys.stdout.encoding = 'mac-roman'
>>> print("frøânçïé".encode('mac-roman'))
'frøânçïé'
But
>>> print("frøânçïé".encode('utf-8'))
'frøânçïé'
Ditto for cp850
>>> sys.stdout.encoding = 'cp850'
>>> print("frøânçïé".encode('cp850'))
'frøânçïé'
If the repertoire of characters of a coding scheme does not
contain the characters -> replace
>>> sys.stdout.encoding = 'cp437'
>>> print("frøânçïé".encode('cp437'))
Traceback (most recent call last):
File "", line 1, in
File "c:\python32\lib\encodings\cp437.py", line 12, in encode
return codecs.charmap_encode(input,errors,encoding_map)
UnicodeEncodeError: 'charmap' codec can't encode character '\xf8' in position
2: character maps to
>>> print("frøânçïé".encode('cp437', 'replace'))
'fr?ânçïé'
Curiousities
>>> sys.stdout.encoding = 'utf-16-be'
>>> print("frøânçïé")
f r ø â n ç ï é
>>> print("frøânçïé".encode('utf-16-be'))
'frøânçïé'
>>> sys.stdout.encoding = 'utf-32-be'
>>> print("frøânçïé".encode('utf-32-be'))
'frøânçïé'
jmf
--
http://mail.python.org/mailman/listinfo/python-list
Re: can't get utf8 / unicode strings from embedded python
> What _are_ you using? i have scripts in a file, that i am invoking into my embedded python within a C++ program. there is no terminal involved. the "print" statement has been redirected (via sys.stdout) to my custom print class, which does not specify "encoding", so i tried the suggestion above to set it: static const char *s_RedirectScript = "import " kEmbeddedModuleName "\n" "import sys\n" "\n" "class CustomPrintClass:\n" " def write(self, stuff):\n" " " kEmbeddedModuleName "." kCustomPrint "(stuff)\n" "class CustomErrClass:\n" " def write(self, stuff):\n" " " kEmbeddedModuleName "." kCustomErr "(stuff)\n" "sys.stdout = CustomPrintClass()\n" "sys.stderr = CustomErrClass()\n" "sys.stdout.encoding = 'UTF-8'\n" "sys.stderr.encoding = 'UTF-8'\n"; but it didn't help. I'm still getting back a string that is a utf-8 string of characters that, if converted to "macRoman" and then interpreted as UTF8, shows the original, correct string. who is specifying macRoman, and where, and how do i tell whoever that is that i really *really* want utf8? -- http://mail.python.org/mailman/listinfo/python-list
Re: can't get utf8 / unicode strings from embedded python
On Sat, Aug 24, 2013, at 2:45, David M. Cotter wrote: > > you need to use u" ... " delimiters for Unicode, otherwise the results you > > get are completely arbitrary and depend on the encoding of your terminal. > okay, well, i'm on a mac, and not using "terminal" at all. but if i > were, it would be utf8 > but it's still not flying :( > so, it seems that i AM getting my utf8 bytes, but i'm getting them > converted to macRoman. huh? where is macRoman specified, and how to i > change that to utf8? i think that's the missing golden ticket You say you're not using terminal. What _are_ you using? -- http://mail.python.org/mailman/listinfo/python-list
Re: can't get utf8 / unicode strings from embedded python
David M. Cotter wrote: > Steven wrote: >> I see you are using Python 2 > correct > >>It's hard to say what *exactly* is happening here, because you don't explain >>how the python print statement somehow gets into your C++ Log code. Do I >>guess right that it catches stdout? > yes, i'm redirecting stdout to my own custom print class, and then from that > function i call into my embedded C++ print function > I don't know much about embedding Python, but each file object has an encoding property. Why not examine sys.stdout.encoding ? And change it to "UTF-8" ? print "encoding is", sys.stdout.encoding sys.stdout.encoding = "UTF-8" -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: can't get utf8 / unicode strings from embedded python
> I see you are using Python 2 correct > Firstly, in Python 2, the compiler assumes that the source code is encoded in > ASCII gar, i must have been looking at doc for v3, as i thought it was all assumed to be utf8 > # -*- coding: utf-8 -*- okay, did that, still no change > you need to use u" ... " delimiters for Unicode, otherwise the results you > get are completely arbitrary and depend on the encoding of your terminal. okay, well, i'm on a mac, and not using "terminal" at all. but if i were, it would be utf8 but it's still not flying :( > For example, if I set my terminal encoding to IBM-850 okay how do you even do that? this is not an interactive session, this is embedded python, within a C++ app, so there's no terminal. but that is a good question: all the docs say "default encoding" everywhere (as in "If string is a Unicode object, this function computes the default encoding of string and operates on that"), but fail to specify just HOW i can set the default encoding. if i could just say "hey, default encoding is utf8", i think i'd be done? > So change the line of code to: > print u"frøânçïé" okay, sure... but i get the exact same results > Those two changes ought to fix the problem, but if they don't, try setting > your terminal encoding to UTF-8 as well well, i'm not sure what you mean by that. i don't have a terminal here. i'm logging to a utf8 log file (when i print) > but what it *actually* prints is this: > >print "frøânçïé" > --> fr√∏√¢n√ß√Ø√© >It's hard to say what *exactly* is happening here, because you don't explain >how the python print statement somehow gets into your C++ Log code. Do I guess >right that it catches stdout? yes, i'm redirecting stdout to my own custom print class, and then from that function i call into my embedded C++ print function >If so, then what I expect is happening is that Python has read in the source >code of >print "~" >with ~ as a bunch of junk bytes, and then your terminal is displaying >those junk bytes according to whatever encoding it happens to be using. >Since you are seeing this: >fr√∏√¢n√ß√Ø√© >my guess is that you're using a Mac, and the encoding is set to the MacRoman >encoding. Am I close? you hit the nail on the head there, i think. using that as a hint, i took this text "fr√∏√¢n√ß√Ø√©" and pasted that into a "macRoman" document, then *reinterpreted* it as UTF8, and voala: "frøânçïé" so, it seems that i AM getting my utf8 bytes, but i'm getting them converted to macRoman. huh? where is macRoman specified, and how to i change that to utf8? i think that's the missing golden ticket -- http://mail.python.org/mailman/listinfo/python-list
Re: can't get utf8 / unicode strings from embedded python
On Fri, 23 Aug 2013 13:49:23 -0700, David M. Cotter wrote:
> note everything works great if i use Ascii, but:
>
> in my utf8-encoded script i have this:
>
>> print "frøânçïé"
I see you are using Python 2, in which case there are probably two or
three errors being made here.
Firstly, in Python 2, the compiler assumes that the source code is
encoded in ASCII, actually ASCII plus arbitrary bytes. Since your source
code is *actually* UTF-8, the bytes in the file are:
70 72 69 6E 74 20 22 66 72 C3 B8 C3 A2 6E C3 A7 C3 AF C3 A9 22
But Python doesn't know the file is encoded in UTF-8, it thinks it is
reading ASCII plus junk, so when it reads the file it parses those bytes
into a line of code:
print "~"
where the ~ represents a bunch of 13 rubbish junk bytes. So that's
the first problem to fix. You can fix this by adding an encoding cookie
at the beginning of your module, in the first or second line:
# -*- coding: utf-8 -*-
The second problem is that even once you've fixed the source encoding,
you're still not dealing with a proper Unicode string. In Python 2, you
need to use u" ... " delimiters for Unicode, otherwise the results you
get are completely arbitrary and depend on the encoding of your terminal.
For example, if I set my terminal encoding to IBM-850, I get:
fr°Ônþ´Ú
from those bytes. If I set it to Central European ISO-8859-3 I get this:
frĝânçïé
Clearly not what I intended. So change the line of code to:
print u"frøânçïé"
Those two changes ought to fix the problem, but if they don't, try
setting your terminal encoding to UTF-8 as well and see if that helps.
[...]
> but what it *actually* prints is this:
>
>> print "frøânçïé"
> --> frøânçïé
It's hard to say what *exactly* is happening here, because you don't
explain how the python print statement somehow gets into your C++ Log
code. Do I guess right that it catches stdout?
If so, then what I expect is happening is that Python has read in the
source code of
print "~"
with ~ as a bunch of junk bytes, and then your terminal is displaying
those junk bytes according to whatever encoding it happens to be using.
Since you are seeing this:
frøânçïé
my guess is that you're using a Mac, and the encoding is set to the
MacRoman encoding. Am I close?
To summarise:
* Add an encoding cookie, to tell Python to use UTF-8 when parsing your
source file.
* Use a Unicode string u"frøânçïé".
* Consider setting your terminal to use UTF-8, otherwise it may not be
able to print all the characters you would like.
* You may need to change the way data gets into your C++ Log function. If
it expects bytes, you may need to use u"...".encode('utf-8') rather than
just u"...". But since I don't understand how data is getting into your
Log function, I can't be sure about this.
I think that is everything. Does that fix your problem?
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
can't get utf8 / unicode strings from embedded python
note everything works great if i use Ascii, but:
in my utf8-encoded script i have this:
> print "frøânçïé"
in my embedded C++ i have this:
PyObject* CPython_Script::print(PyObject *args)
{
PyObject*resultObjP = NULL;
const char *utf8_strZ = NULL;
if (PyArg_ParseTuple(args, "s", &utf8_strZ)) {
Log(utf8_strZ, false);
resultObjP = Py_None;
Py_INCREF(resultObjP);
}
return resultObjP;
}
Now, i know that my Log() can print utf8 (has for years, very well debugged)
but what it *actually* prints is this:
> print "frøânçïé"
--> frøânçïé
another method i use looks like this:
> kj_commands.menu("控件", "同步滑帧", "全局无滑帧")
or
> kj_commands.menu(u"控件", u"同步滑帧", u"全局无滑帧")
and in my C++ i have:
SuperString ScPyObject::GetAs_String()
{
SuperString str;
if (PyUnicode_Check(i_objP)) {
#if 1
// method 1
{
ScPyObject
utf8Str(PyUnicode_AsUTF8String(i_objP));
str = utf8Str.GetAs_String();
}
#elif 0
// method 2
{
UTF8Char*uniZ = (UTF8Char
*)PyUnicode_AS_UNICODE(i_objP);
str.assign(&uniZ[0],
&uniZ[PyUnicode_GET_DATA_SIZE(i_objP)], kCFStringEncodingUTF16);
}
#else
// method 3
{
UTF32VeccharVec(32768);
CF_ASSERT(sizeof(UTF32Vec::value_type) == sizeof(wchar_t));
PyUnicodeObject *uniObjP = (PyUnicodeObject
*)(i_objP);
Py_ssize_t
sizeL(PyUnicode_AsWideChar(uniObjP, (wchar_t *)&charVec[0], charVec.size()));
charVec.resize(sizeL);
charVec.push_back(0);
str.Set(SuperString(&charVec[0]));
}
#endif
} else {
str.Set(uc(PyString_AsString(i_objP)));
}
Log(str.utf8Z());
return str;
}
for the string, "控件", i get:
--> 控件
for the *unicode* string, u"控件", Methods 1, 2, and 3, i get the same thing:
--> 控件
okay so what am i doing wrong???
--
http://mail.python.org/mailman/listinfo/python-list
Re: how to package embedded python?
On Wednesday, July 31, 2013 11:47:19 AM UTC-4, David M. Cotter wrote: > okay, well that might turn out to be useful, except i don't quite know how to > use it, and there are no "from scratch" instructions. > > > > i managed to download "py2exe-0.6.9.zip" and unzip it, but how does one > "install" this package? (yes, still a newb at that) What's your OS? For Windows there is an exe that installs it. Did you download that? Or you could, in the command line, do: cd c:\some_folder\the_py2exe_folder python setup.py install (The Python site has a huge page of details on that route. http://docs.python.org/2/install/) Or you could probably get away with just moving the py2exe folder to Python27\site-packages (or whatever Python you have) Or you could first install pip (recommended!) and then just: pip install py2exe (http://dubroy.com/blog/so-you-want-to-install-a-python-package) > then, once installed, how do i say "include the entire world" instead of just > "mymodule" ? cuz the point of embedding python on my app is that the > end-user can run any script at all, not just one module. I don't know. As I understand it, py2exe will pull whatever is needed in Python to run your module. If your module requires all of Python, I guess that will work. I think regardless of the module, even if it is a "Hello, World" program, py2exe has to include the Python interpreter. Unfortunately, and surprisingly, the py2exe site is still down. Odd. If it returns, that should help. There is also a py2exe users mailing list that you could find by Googling for it. In terms of -- http://mail.python.org/mailman/listinfo/python-list
Re: how to package embedded python?
okay, well that might turn out to be useful, except i don't quite know how to use it, and there are no "from scratch" instructions. i managed to download "py2exe-0.6.9.zip" and unzip it, but how does one "install" this package? (yes, still a newb at that) then, once installed, how do i say "include the entire world" instead of just "mymodule" ? cuz the point of embedding python on my app is that the end-user can run any script at all, not just one module. -- http://mail.python.org/mailman/listinfo/python-list
Re: how to package embedded python?
On Tuesday, July 30, 2013 4:23:06 PM UTC-4, David M. Cotter wrote: > yes, i've looked there, and all over google. i'm quite expert at embedding > at this point. > > > > however nowhere i have looked has had instructions for "this this is how you > package up your .exe with all the necessary python modules necessary to > actually run on a user's system that does not have python installed". > > > > on mac, it's trivial: all macs come with python, there is nothing i need to > include with my app and it "just works" > > > > on windows: if you don't include the proper DLLs and/or whatnot, then the app > will complain about missing DLLs on startup. > > > > What DLLs must i include? where are the instructions? I know nothing about embedding, but in terms of packaging up a Python interpreter with an application that needs it, could you use py2exe to do that? If, so is this helpful (gotten from Google's cache since page doesn't appear at the moment...later try embedded Python py2exe search)?: http://webcache.googleusercontent.com/search?q=cache:x3lrdFT5OF0J:www.py2exe.org/index.cgi/ShippingEmbedded+&cd=2&hl=en&ct=clnk&gl=us&client=firefox-a -- http://mail.python.org/mailman/listinfo/python-list
Re: how to package embedded python?
yes, i've looked there, and all over google. i'm quite expert at embedding at this point. however nowhere i have looked has had instructions for "this this is how you package up your .exe with all the necessary python modules necessary to actually run on a user's system that does not have python installed". on mac, it's trivial: all macs come with python, there is nothing i need to include with my app and it "just works" on windows: if you don't include the proper DLLs and/or whatnot, then the app will complain about missing DLLs on startup. What DLLs must i include? where are the instructions? -- http://mail.python.org/mailman/listinfo/python-list
Re: how to package embedded python?
On 7/25/13 5:05 PM, David M. Cotter wrote: what must i include in my app package if i'm embedding python? i tried including *everything* in the "DLLs" directory, but my app still crashes as soon as i attempt to initialize python. this is on a system that does not have python installed, as most of my users won't have it. is it actually a requirement that they first install python? (cuz it does work then) Have you looked at these docs? http://docs.python.org/2/extending/embedding.html Lots of other hits on Google for ""embedding Python in C app." -- Kevin Walzer Code by Kevin/Mobile Code by Kevin http://www.codebykevin.com http://www.wtmobilesoftware.com -- http://mail.python.org/mailman/listinfo/python-list
Re: how to package embedded python?
nooobody knw the trouble a s... -- http://mail.python.org/mailman/listinfo/python-list
Re: embedded python and threading
David M. Cotter, 26.07.2013 19:28: > DOH! as my second thread, i had been using a sample script that i had > copy-pasted without much looking at it. guess what? it prints the time. and > yes, it did "from time import time", which explains it all. Ah, and you were using the same globals dict for both scripts, I guess? That explains it then. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: embedded python and threading
DOH! as my second thread, i had been using a sample script that i had copy-pasted without much looking at it. guess what? it prints the time. and yes, it did "from time import time", which explains it all. thanks for the hints here, that helped me figure it out! -- http://mail.python.org/mailman/listinfo/python-list
Re: embedded python and threading
no, there is no "time.py" anywhere (except perhaps as the actual python library originally imported) did you understand that the function works perfectly, looping as it should, up until the time i run a second script on a separate thread? -- http://mail.python.org/mailman/listinfo/python-list
Re: embedded python and threading
In <965b463e-e5bf-4ccd-9a3c-b0cb964b3...@googlegroups.com> "David M. Cotter" writes: > == > 9: Traceback (most recent call last): > 9: File "", line 10, in ? > 9: File "", line 6, in main > 9: AttributeError: 'builtin_function_or_method' object has no attribute > 'sleep' > == You must have a file named 'time.py' in the current directory, and the import statement is getting that module instead of the system time module. -- John Gordon A is for Amy, who fell down the stairs gor...@panix.com B is for Basil, assaulted by bears -- Edward Gorey, "The Gashlycrumb Tinies" -- http://mail.python.org/mailman/listinfo/python-list
