C extension custom types in abi3 module
I am trying to split off reportlab C extensions to simplify installations and
make use of more advanced packages.
A simple extension is easily converted to being an abi3 module. However, another has a custom type which uses the old
style mechanisms here
https://docs.python.org/3.7/extending/newtypes_tutorial.html#the-basics
I made a simple setup based on this abi3 example modified to allow switching
between abi3 and normal build
https://github.com/joerick/python-abi3-package-sample
I made a tiny change to the code example in the newtypes tutorial page the code
is here
https://github.com/MrBitBucket/custom-type
In a python 3.7 - 3.12a3 environment I find I can build the wheel OK with
ABI3_WHEEL=0 pip wheel -w dist .
but I get lots of compile errors if I switch to an abi3 build with
ABI3_WHEEL=1 pip wheel -w dist .
looking at the errors
src/_custom.c:10:1: error: variable ‘CustomType’ has initializer but
incomplete type
10 | static PyTypeObject CustomType = {
| ^~
In file included from
/home/robin/LOCAL/3.7.16/include/python3.7m/Python.h:90,
from src/_custom.c:2:
/home/robin/LOCAL/3.7.16/include/python3.7m/object.h:90:5: error: extra
brace group at end of initializer
90 | { PyObject_HEAD_INIT(type) size },
| ^
src/_custom.c:11:9: note: in expansion of macro ‘PyVarObject_HEAD_INIT’
11 | PyVarObject_HEAD_INIT(NULL, 0)
it looks like I have to use a different mechanism to setup custom types in the
abi3 world.
I looked in Modules/xxlimited_35.c, but that seems much more complex and
provides for a type which supports GC.
Are there any ABI3 examples using the old style strategy?
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to convert a raw string r'\xdd' to '\xdd' more gracefully?
I’m not understanding the task. The sample code given is converting the input
r’\x0a’ to a newline, it appears.
import re
def exam(z):
print(f"examine {type(z)} {z}")
for c in z:
print(f"{ord(c)} {c}")
s0 = r'\x0a'
def to1byte(matchobj):
return chr(int('0x' + matchobj.group(1), 16))
s1 = re.sub(r'\\x([0-9a-fA-F]{2})', to1byte, s0)
exam(s0)
exam(s1)
---
examine \x0a
92 \
120 x
48 0
97 a
examine
10
From: Python-list on
behalf of Jach Feng
Date: Wednesday, December 7, 2022 at 9:27 PM
To: python-list@python.org
Subject: Re: How to convert a raw string r'xdd' to 'xdd' more gracefully?
*** Attention: This is an external email. Use caution responding, opening
attachments or clicking on links. ***
Peter Otten 在 2022年12月8日 星期四清晨5:17:59 [UTC+8] 的信中寫道：
> On 07/12/2022 03:23, Jach Feng wrote:
> > s0 = r'\x0a'
> > At this moment it was done by
> >
> > def to1byte(matchobj):
> > return chr(int('0x' + matchobj.group(1), 16))
> > s1 = re.sub(r'\\x([0-9a-fA-F]{2})', to1byte, s0)
> >
> > But, is it that difficult on doing this simple thing?
> >>> import codecs
> >>> codecs.decode(r"\x68\x65\x6c\x6c\x6f\x0a", "unicode-escape")
> 'hello\n'
Thank you. What I really want to handle is to any r'\xdd'. The r'\x0a' is for
example. Sorry, didn't describe it clearly:-)
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!kUnextA7_cF7EoP_4hGzC5Jq2wRvn8nwLwT8wmeNkgVjK_n6VG19fxb-4SwmDMwepWe8_bGaH9Y2LlkSvFRz$
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to convert a raw string r'\xdd' to '\xdd' more gracefully?
The original post started out with r'\x0a' but then talked about '\xdd'.
I assumed that there was a pattern here, a raw string containing "\x"
and two more characters, and made a suggestion for converting any string
with that pattern. But the OP was very unclear what the task really
was, so here we all are, making a variety of guesses.
On 12/8/2022 8:23 AM, Weatherby,Gerard wrote:
I’m not understanding the task. The sample code given is converting the input
r’\x0a’ to a newline, it appears.
import re
def exam(z):
print(f"examine {type(z)} {z}")
for c in z:
print(f"{ord(c)} {c}")
s0 = r'\x0a'
def to1byte(matchobj):
return chr(int('0x' + matchobj.group(1), 16))
s1 = re.sub(r'\\x([0-9a-fA-F]{2})', to1byte, s0)
exam(s0)
exam(s1)
---
examine \x0a
92 \
120 x
48 0
97 a
examine
10
From: Python-list on behalf of
Jach Feng
Date: Wednesday, December 7, 2022 at 9:27 PM
To: python-list@python.org
Subject: Re: How to convert a raw string r'xdd' to 'xdd' more gracefully?
*** Attention: This is an external email. Use caution responding, opening
attachments or clicking on links. ***
Peter Otten 在 2022年12月8日 星期四清晨5:17:59 [UTC+8] 的信中寫道：
On 07/12/2022 03:23, Jach Feng wrote:
s0 = r'\x0a'
At this moment it was done by
def to1byte(matchobj):
return chr(int('0x' + matchobj.group(1), 16))
s1 = re.sub(r'\\x([0-9a-fA-F]{2})', to1byte, s0)
But, is it that difficult on doing this simple thing?
import codecs
codecs.decode(r"\x68\x65\x6c\x6c\x6f\x0a", "unicode-escape")
'hello\n'
Thank you. What I really want to handle is to any r'\xdd'. The r'\x0a' is for
example. Sorry, didn't describe it clearly:-)
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!kUnextA7_cF7EoP_4hGzC5Jq2wRvn8nwLwT8wmeNkgVjK_n6VG19fxb-4SwmDMwepWe8_bGaH9Y2LlkSvFRz$
--
https://mail.python.org/mailman/listinfo/python-list
Re: C extension custom types in abi3 module
On 08/12/2022 12:52, Robin Becker wrote: I am trying to split off reportlab C extensions to simplify installations and make use of more advanced packages. A simple extension is easily converted to being an abi3 module. However, another has a custom type which uses the old style mechanisms here it looks like I have to use a different mechanism to setup custom types in the abi3 world. In the docs I see this "Also, since PyTypeObject is only part of the Limited API as an opaque struct, any extension modules using static types must be compiled for a specific Python minor version." So it seems I must switch to using a heap allocated type or keep compiling in the old way. I looked in Modules/xxlimited_35.c, but that seems much more complex and provides for a type which supports GC. Are there any ABI3 examples using the old style strategy? -- Robin Becker -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: How to convert a raw string r'\xdd' to '\xdd' more gracefully?
On 08/12/2022 02:17, Jach Feng wrote:
Peter Otten 在 2022年12月8日 星期四清晨5:17:59 [UTC+8] 的信中寫道：
On 07/12/2022 03:23, Jach Feng wrote:
s0 = r'\x0a'
At this moment it was done by
def to1byte(matchobj):
return chr(int('0x' + matchobj.group(1), 16))
s1 = re.sub(r'\\x([0-9a-fA-F]{2})', to1byte, s0)
But, is it that difficult on doing this simple thing?
import codecs
codecs.decode(r"\x68\x65\x6c\x6c\x6f\x0a", "unicode-escape")
'hello\n'
Thank you. What I really want to handle is to any r'\xdd'. The r'\x0a' is for
example. Sorry, didn't describe it clearly:-)
Hm, codecs.decode() does work for arbitrary escapes. It will produce the
same result for r"\xdd"-type raw strings where d is in the range 0...F.
It will also convert other escapes like
>>> codecs.decode(r"\t", "unicode-escape")
'\t'
>>> codecs.decode(r"\u5728", "unicode-escape")
'在'
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to convert a raw string r'\xdd' to '\xdd' more gracefully?
Jach Feng 在 2022年12月7日 星期三上午10:23:20 [UTC+8] 的信中寫道：
> s0 = r'\x0a'
> At this moment it was done by
>
> def to1byte(matchobj):
> return chr(int('0x' + matchobj.group(1), 16))
> s1 = re.sub(r'\\x([0-9a-fA-F]{2})', to1byte, s0)
>
> But, is it that difficult on doing this simple thing?
>
> --Jach
I find another answer on the web.
>>> s0 = r'\x0a'
>>> s0.encode('Latin-1').decode('unicode-escape')
'\n'
--
https://mail.python.org/mailman/listinfo/python-list
Panoptisch - A way to understand your project's dependencies and find malicious packages
Hello all, I would like to share Panoptisch, a FOSS(Free and Open Source Software) tool I've been working on. We all may have encountered the issue of not having a clear dependency tree or not being sure of the modules our dependencies and sub-dependencies are using. Some of us may have also heard of supply chain attacks, where open source projects are hijacked to distribute malicious code masquerading as the original package. This can happen deep down in the dependency chain. Panoptisch was born out of the need to accurately verify the modules used in my project. It recursively scans a Python module or file to find modules used and exports a report in JSON which can be parsed for analysis. For example, should your yaml parser, or it's sub-dependencies import socket/os? should your markdown renderer or it's sub-dependencies import sys/importlib? *Probably not.* Panoptisch is in early stages, has known limitations and is looking for help! I would love feedback, contributions, and most important of all, rigorous testing! I would also love to help you integrate this tool in your workflow to write more secure software. Link: https://github.com/R9295/panoptisch Short Demo: https://www.youtube.com/watch?v=bDJWl_odXx0 Thanks and Regards, aarnav -- https://mail.python.org/mailman/listinfo/python-list
MinecraftEdu
Hello, I'm new to MinecraftEDU programming with Python. I'm wondering if anyone can recommend how to get started with creating lessons and how to get started programming in MinecraftEDU? -- https://mail.python.org/mailman/listinfo/python-list
Re: MinecraftEdu
On 08Dec2022 12:12, Jelena Ilić wrote: I'm new to MinecraftEDU programming with Python. I'm wondering if anyone can recommend how to get started with creating lessons and how to get started programming in MinecraftEDU? Had you started here? https://education.minecraft.net/en-us/resources/computer-science-subject-kit/python-101 That's just from a web search, I've not used it. Cheers, Cameron Simpson -- https://mail.python.org/mailman/listinfo/python-list
Re: How to convert a raw string r'\xdd' to '\xdd' more gracefully?
Jach Feng 在 2022年12月7日 星期三上午10:23:20 [UTC+8] 的信中寫道：
> s0 = r'\x0a'
> At this moment it was done by
>
> def to1byte(matchobj):
> return chr(int('0x' + matchobj.group(1), 16))
> s1 = re.sub(r'\\x([0-9a-fA-F]{2})', to1byte, s0)
>
> But, is it that difficult on doing this simple thing?
>
> --Jach
The whold story is,
I had a script which accepts an argparse's positional argument. I like this
argument may have control character embedded in when required. So I make a post
"How to enter escape character in a positional string argument from the command
line? on DEC05. But there is no response. I assume that there is no way of
doing it and I have to convert it later after I get the whole string from the
command line.
I made this convertion using the chr(int(...)) method but not satisfied with.
That why this post came out.
At this moment the conversion is done almost the same as Peter's
codecs.decode() method but without the need of importing codecs module:-)
def to1byte(matchobj):
return matchobj.group(0).encode().decode("unicode-escape")
--
https://mail.python.org/mailman/listinfo/python-list
Re: Panoptisch - A way to understand your project's dependencies and find malicious packages
On 08/12/2022 17:52, Aarnav Mahavir Bos wrote: Hello all, I would like to share Panoptisch, a FOSS(Free and Open Source Software) tool I've been working on. Hi there, I added your project to my watch list, keep on your work. A couple of points: First, I glanced at the code and in the very first file I opened, https://github.com/R9295/panoptisch/blob/master/panoptisch/__init__.py, I see main(). I usually place such a code in __main__.py Second, in addition to AST analysis it would be nice to implement a sandbox with import hooks. Axy. -- https://mail.python.org/mailman/listinfo/python-list
