Re: [Pythonmac-SIG] PyObjC - Apple sample code

[Glyph]

Relying on the system Python for this sort of stuff has always guaranteed you'd 
have an out-of-date version of all of your dependencies.

The availability of wheels (thanks again Ronald!!!) for pyobjc means that you 
don't need the biggest impediment to users installing stuff, which is a C 
compiler.  If you're building stuff for distribution to non-technical folks, 
build it with your own version of python (3) and ship it with py2app, not by 
copying scripts around.

If there are things that make this more painful than just copying individual 
scripts, it's probably best to figure out how to get those addressed with the 
PyPA community.

-glyph

> On Apr 28, 2017, at 4:20 AM, Ben Byram-Wigfield  wrote:
> 
> I ran the installer package for the latest python 2.7, and then used 
> easy_install to install PyObjC.
> I get the same errors on two separate Macs. What should I do to fix the 
> installation?
> 
> Does Apple not have plans to include (all of) the latest PyObjC? That’s 
> rather sad. The reason I was attracted to creating ObjC python scripts was 
> that they could run on any Mac.
> 
> Thanks
> 
> Ben
> 
> 
>> On 28 Apr 2017, at 08:07, Ronald Oussoren  wrote:
>> 
>> 
>>> On 27 Apr 2017, at 10:17, Ben Byram-Wigfield  wrote:
>>> 
>>> I tried the repository browser there, and the version of 
>>> parse_page_contents.py still doesn’t work for me. I’m using the latest 
>>> downloaded versions of python 2.7 and PyObjC. I also tried using the 
>>> default OS X versions.
>>> The errors are in the attached file.
>>> 
>> The PyObjC 3.2 error at the top of the file seems to indicate that your 
>> Python installation is broken, PyObjC imports the stdlib io module and that 
>> causes and error.
>> 
>> The PyObjC 2.5 error is due to general brokeness of the system installation 
>> of PyObjC. PyObjC 2.5 is ancient and not something I support anymore, 
>> furthermore (IIRC) Apple doesn’t ship all of PyObjC. The error you’re 
>> getting indicates that the framework wrappers are incomplete.
>> 
>> BTW. If you are new to Python I’d look into using Python 3.6 instead, both 
>> because that has less change to run into problems due to interference 
>> between the system install of Python 2.7 and a manual installation, and 
>> because the Python community is moving ever faster to Python 3.
>> 
>> Ronald
>> 
> 
> ___
> Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
> https://mail.python.org/mailman/listinfo/pythonmac-sig
> unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG
___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

Re: [Pythonmac-SIG] Uninstalling

[Glyph]

> On Jul 20, 2017, at 3:24 AM, Piet van Oostrum  wrote:
> 
> Normally you should never touch anything in /usr/bin or /usr/lib. That is 
> Apple territory.

You can't any more.

$ touch /usr/bin/just-testing
touch: /usr/bin/just-testing: Operation not permitted
$ sudo touch /usr/bin/just-testing
Password:
touch: /usr/bin/just-testing: Operation not permitted

Since System Integrity Protection came out with El Capitan, unless you reboot 
into recovery mode and type some "void your warranty" commands into the 
terminal app on the recovery partition, it's simply no longer possible for 
3rd-party applications and installers to write files into /usr/bin.  Attempting 
to install over /usr/bin/python on any macOS released in the last 2 years will 
just fail.

-glyph___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

Re: [Pythonmac-SIG] Failure on pip upgrade attempt

[Glyph]

PYTHONPATH itself is a bit of an anachronism :).  It's the past, virtualenvs 
are the future!  https://orbifold.xyz/pythonpath.html 
<https://orbifold.xyz/pythonpath.html>

(If you're using PyObjC and avoiding virtualenvs for that reason, check out 
https://github.com/glyph/venvdotapp <https://github.com/glyph/venvdotapp> )

-g

> On Feb 13, 2018, at 9:51 AM, Gerrie Shults  wrote:
> 
> Thank you, Barry. Your PYTHONPATH suspicion was correct.
> 
> I have been away from Python for many years. My PYTHONPATH pointed to 
> personal directories, which happened to include a 15-year old module named 
> ‘html’.
> 
> Clearing that solved my problem.
> 
> Thanks again,
> Gerrie
> 
>> On Feb 12, 2018, at 11:56, Barry Scott > <mailto:ba...@barrys-emacs.org>> wrote:
>> 
>> html module is shipped with python 3.6.4.
>> 
>> Only thing I can think of is that your PYTHONPATH is interfering in some way.
>> 
>> $ python3
>> Python 3.6.4 (v3.6.4:d48ecebad5, Dec 18 2017, 21:07:28)
>> [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
>> Type "help", "copyright", "credits" or "license" for more information.
>> >>> import html
>> >>> html
>> > '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/html/__init__.py'>
>> >>> from html.parser import HTMLParser
>> >>> HTMLParser
>> 
>> >>> import sys
>> >>> sys.path
>> ['', '/Library/Frameworks/Python.framework/Versions/3.6/lib/python36.zip', 
>> '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6', 
>> '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib-dynload',
>>  
>> '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages']
>> >>>
>> 
>> Barry
>> 
>> 
>>> On 12 Feb 2018, at 16:05, Gerrie Shults >> <mailto:gshu...@icloud.com>> wrote:
>>> 
>>> Before my original post, I looked up html.parser in the Python 
>>> documentation, and everything I saw indicated to me that it was already 
>>> installed.
>>> 
>>> I have now looked again at the documentation and can find examples using 
>>> html.parser, but nothing that tells me how to install it.
>>> 
>>> How do I install it? And how can I find what is already installed, and what 
>>> is available to install without having to download more?
>>> 
>>> Thanks,
>>> Gerrie Shults
>>> 
>>>> On Feb 11, 2018, at 18:59, 樊佳亮 mailto:fjl2...@163.com>> 
>>>> wrote:
>>>> 
>>>> You should first install html.parser this error report is about html.parser
>>>> Then rerun pip command
>>>> 
>>>> 发自我的 iPhone
>>>> 
>>>> 在 2018年2月12日，上午5:32，Gerrie Shults >>> <mailto:gshu...@icloud.com>> 写道：
>>>> 
>>>>> I’m trying to get the latest version of pip to use to install Flask.
>>>>> 
>>>>> I have a fresh install of Python 3.6.4, and I executed "pip3 install -U 
>>>>> pip”, per the instructions from "https://pip.pypa.io/en/stable/installing 
>>>>> <https://pip.pypa.io/en/stable/installing>/“, and got the result below.
>>>>> 
>>>>> Why isn’t html.parser available as part of the standard install? What do 
>>>>> I need to do to make this work?
>>>>> 
>>>>> Thanks for any help.
>>>>> 
>>>>> Gerrie Shults
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> $ pip3 install -U pip
>>>>> Traceback (most recent call last):
>>>>>   File "/Library/Frameworks/Python.framework/Versions/3.6/bin/pip3", line 
>>>>> 7, in 
>>>>> from pip import main
>>>>>   File 
>>>>> "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/__init__.py",
>>>>>  line 28, in 
>>>>> from pip.vcs import git, mercurial, subversion, bazaar  # noqa
>>>>>   File 
>>>>> "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/vcs/subversion.py",
>>>>>  line 9, in 
>>>>> from pip.index import Link
>>>>>   File 
>>>>> "/Library/Frameworks/Python.framework/

Re: [Pythonmac-SIG] [ANN] PyObjC 5.0

[Glyph]

On Sep 18, 2018, at 1:22 PM, Ronald Oussoren via Pythonmac-SIG 
 wrote:
> 
> PyObjC 5.0 is out

Thanks again for your tireless (or at least apparently tireless) maintenance, 
Ronald!

> * Added bindings for the “CarbonCore” subframework of the “CoreServices” 
> framework.
> 
> Most APIs in this subframework are not available to Python, only those APIs 
> that are not deprecated and seem interesting are exposed.

One thing I wondered about when I saw "carbon" here is that I wonder if the 
APIs that could make 
https://pythonhosted.org/pyobjc/examples/Cocoa/AppKit/HotKeyPython/index.html 
<https://pythonhosted.org/pyobjc/examples/Cocoa/AppKit/HotKeyPython/index.html> 
are exposed again?  The cautions that the relevant Carbon APIs require 32-bit 
mode are inaccurate; 
https://blog.shpakovski.com/2012/07/global-keyboard-shortcuts-in-cocoa.html 
<https://blog.shpakovski.com/2012/07/global-keyboard-shortcuts-in-cocoa.html> 
explains that they're still supported in sandboxed Mac App Store apps, so 
they're just poorly documented, not deprecated.  (It seems that there's never 
been a proper Cocoa replacement for this functionality, just a couple of 
popular wrappers floating around out there...)

This is something I have wanted to do in Python many times, but never quite 
badly enough to figure out all the build gymnastics required to get the 
bindings for https://github.com/shpakovski/MASShortcut 
<https://github.com/shpakovski/MASShortcut> built and bundled with my 
application.

-glyph

P.S.: speaking of build gymnastics, does anyone on this list happen to have a 
way to use CocoaPods or something similar to pull in open source Cocoa 
frameworks to a PyObjC application?___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

Re: [Pythonmac-SIG] [ANN] PyObjC 5.0

[Glyph]

> On Sep 21, 2018, at 4:50 AM, Ronald Oussoren  wrote:
> 
> 
> 
>> On 21 Sep 2018, at 05:59, Glyph > <mailto:gl...@twistedmatrix.com>> wrote:
>> 
>> On Sep 18, 2018, at 1:22 PM, Ronald Oussoren via Pythonmac-SIG 
>> mailto:pythonmac-sig@python.org>> wrote:
>>> 
>>> PyObjC 5.0 is out
>> 
>> Thanks again for your tireless (or at least apparently tireless) 
>> maintenance, Ronald!
>> 
>>> * Added bindings for the “CarbonCore” subframework of the “CoreServices” 
>>> framework.
>>> 
>>> Most APIs in this subframework are not available to Python, only those APIs 
>>> that are not deprecated and seem interesting are exposed.
>> 
>> One thing I wondered about when I saw "carbon" here is that I wonder if the 
>> APIs that could make 
>> https://pythonhosted.org/pyobjc/examples/Cocoa/AppKit/HotKeyPython/index.html
>>  
>> <https://pythonhosted.org/pyobjc/examples/Cocoa/AppKit/HotKeyPython/index.html>
>>  are exposed again?  
> 
> The API used in that example is not exposed through PyObjC at this time 
> (AFAIK it is part of “Carbon.framework”, not the CarbonCore sub framework of 
> “CoreServices.framework”. 
> 
> I’m open to adding bindings to interesting APIs, but note that 
> “RegisterEventHotKey” is not documented on Apple’s website and might not be 
> the right solution for modern code. I don’t know yet what the right solution 
> is though (a number of APIs that are exposed through PyObjC can be used to 
> implement similar functionality).

If there were a more modern solution I'd gladly use it rather than asking you 
to expose gross and ancient Carbon APIs :).  Which APIs that are already 
exposed can do this?

>> The cautions that the relevant Carbon APIs require 32-bit mode are 
>> inaccurate; 
>> https://blog.shpakovski.com/2012/07/global-keyboard-shortcuts-in-cocoa.html 
>> <https://blog.shpakovski.com/2012/07/global-keyboard-shortcuts-in-cocoa.html>
>>  explains that they're still supported in sandboxed Mac App Store apps, so 
>> they're just poorly documented, not deprecated.  (It seems that there's 
>> never been a proper Cocoa replacement for this functionality, just a couple 
>> of popular wrappers floating around out there…)
>> 
>> This is something I have wanted to do in Python many times, but never quite 
>> badly enough to figure out all the build gymnastics required to get the 
>> bindings for https://github.com/shpakovski/MASShortcut 
>> <https://github.com/shpakovski/MASShortcut> built and bundled with my 
>> application.
> 
> I’ll look into exposing this API for PyObjC 5.1, even if that means adding a 
> very limited set of bindings for Carbon.framework.

Thanks!

>> -glyph
>> 
>> P.S.: speaking of build gymnastics, does anyone on this list happen to have 
>> a way to use CocoaPods or something similar to pull in open source Cocoa 
>> frameworks to a PyObjC application?
> 
> I haven’t looked into this yet.  I guess the hardest part is creating the 
> metadata for constants and pointer arguments. Could you file and issue for 
> this? 

Sure, sometime this weekend I'll try to write it up.

___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

[Pythonmac-SIG] ctypes MemoryError when attempting to launch notarized app

[Glyph]

I adjusted my code-signing to use the new, stricter requirements imposed by app 
notarization.  I managed to get it successfully notarized, but the app is now 
non-functional as a result: at startup, I get:

Traceback (most recent call last):
  File "my.app/Contents/Resources/__boot__.py", line 93, in 
_setup_ctypes()
  File "my.app/Contents/Resources/__boot__.py", line 86, in _setup_ctypes
from ctypes.macholib import dyld
  File "", line 971, in _find_and_load
  File "", line 955, in _find_and_load_unlocked
  File "", line 656, in _load_unlocked
  File "", line 626, in _load_backward_compatible
  File "ctypes/__init__.pyc", line 538, in 
  File "ctypes/__init__.pyc", line 273, in _reset_cache

(If anyone wants to follow along in the traceback, this is using python.org 
 3.6.6.)

This happens before any of my code even runs, so I can't just try to avoid 
ctypes.

Curiously, this is the same traceback that comes from 
https://forum.kodi.tv/showthread.php?tid=329171 
, which suggests it's 
something fundamental to strict shared-library sandboxing that ctypes trips 
over when trying to initialize itself.

Does anyone have experience with this, or ideas about what to do?

-glyph___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

[Pythonmac-SIG] py2app apps codesigned with "--options runtime", i.e. any notorizable app, fails to launch with ctypes error (was Re: [Pyobjc-dev] ctypes MemoryError when attempting to launch notarize

[Glyph]

> On Oct 28, 2018, at 2:27 AM, Ronald Oussoren  wrote:
> 
> 
> 
>> On 28 Oct 2018, at 06:27, Glyph  wrote:
>> 
>> I adjusted my code-signing to use the new, stricter requirements imposed by 
>> app notarization. I managed to get it successfully notarized, but the app is 
>> now non-functional as a result: at startup, I get:
>> 
>> Traceback (most recent call last):
>>  File "my.app/Contents/Resources/__boot__.py", line 93, in 
>>_setup_ctypes()
>>  File "my.app/Contents/Resources/__boot__.py", line 86, in _setup_ctypes
>>from ctypes.macholib import dyld
>>  File "", line 971, in _find_and_load
>>  File "", line 955, in _find_and_load_unlocked
>>  File "", line 656, in _load_unlocked
>>  File "", line 626, in _load_backward_compatible
>>  File "ctypes/__init__.pyc", line 538, in 
>>  File "ctypes/__init__.pyc", line 273, in _reset_cache
>> 
>> (If anyone wants to follow along in the traceback, this is using python.org 
>> 3.6.6.)
> 
> On what version of macOS? I expect 10.14 because that’s the only release that 
> actually knows about notarization, but enabling this feature might also 
> affect how the app is signed.

ProductName:Mac OS X
ProductVersion: 10.14
BuildVersion:   18A391

>> This happens before any of my code even runs, so I can't just try to avoid 
>> ctypes.
> 
> You could patch the __boot__.py file before signing to see if that helps. 
> Although this should cause problems further on, the call to _setup_ctypes 
> should only be created when some code in your app bundle has a dependency on 
> ctypes. 

I'll give that a shot.

>> 
>> Curiously, this is the same traceback that comes from 
>> https://forum.kodi.tv/showthread.php?tid=329171 
>> <https://forum.kodi.tv/showthread.php?tid=329171>, which suggests it's 
>> something fundamental to strict shared-library sandboxing that ctypes trips 
>> over when trying to initialize itself.
>> 
>> Does anyone have experience with this, or ideas about what to do?
> 
> I’m afraid not. I currently get away with not signing apps at all, although 
> properly supporting signing is on my way too long wish list for py2app.  

The ability to distribute unsigned apps is not-so-slowly going away; even the 
ability to distribute non-notarized apps has a very limited shelf-life at this 
point.  So this ought to be an alarming development for everyone - having 
Python apps effectively banned from macOS distribution is a big potential 
problem :-\.

The good news here is that aside from having to write a little for loop in 
shell (shown below) getting the app codesigned previously was easy, and my app 
*did* pass notarization, so nothing that py2app is doing is breaking things on 
apple's end.  It's just a matter of a ctypes bug.

As I see it, there's 2 problems here:

py2app's __boot__.py should fail more gracefully if initializing ctypes doesn't 
work, since not everybody needs ctypes.  Shall I file this on the tracker?
ctypes itself should address whatever eldritch hideousness is causing this; in 
addition to the windows security layer stuff I found, grsecurity TPE causes the 
same traceback: https://bugs.python.org/issue28429

> With some luck there’s some entitlement or code signing option that causes 
> this problem.  What is the output of "codesign --display --verbose=4” for the 
> application? Both with and without notarisation? 

Sorry, my original message was not clear.  App notarization itself is not the 
problem, it's the "stricter requirements" that I ambiguously referenced.  The 
requirement in question is the '--options runtime' flag passed to 'codesign'.  
So you can just codesign an app (even with an ad-hoc identity, you technically 
could do this without even having a valid cert, although the way one generates 
one of those escapes me) with the 'runtime' option, you can reproduce this.

So if I sign my app like this:

#!/bin/bash
find "${NAME}.app" -iname '*.so' -or -iname '*.dylib' |
while read libfile; do
  codesign --sign "${IDENTITY}" \
   --deep "${libfile}" \
   --force \
   --options runtime;
done;

codesign --sign "${IDENTITY}" \
 --deep "${NAME}.app" \
 --force \
 --options runtime;

and then run it as "./${NAME}.app/Contents/MacOS/${NAME}".  I immediately get 
the traceback given above.

-glyph___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

Re: [Pythonmac-SIG] py2app apps codesigned with "--options runtime", i.e. any notorizable app, fails to launch with ctypes error (was Re: [Pyobjc-dev] ctypes MemoryError when attempting to launch nota

[Glyph]

> On Oct 28, 2018, at 11:20 AM, Glyph  wrote:
> 
> 
> 
>> On Oct 28, 2018, at 2:27 AM, Ronald Oussoren > <mailto:ronaldousso...@mac.com>> wrote:
> 
>>> 
>>> Curiously, this is the same traceback that comes from 
>>> https://forum.kodi.tv/showthread.php?tid=329171 
>>> <https://forum.kodi.tv/showthread.php?tid=329171>, which suggests it's 
>>> something fundamental to strict shared-library sandboxing that ctypes trips 
>>> over when trying to initialize itself.
>>> 
>>> Does anyone have experience with this, or ideas about what to do?
>> 
>> I’m afraid not. I currently get away with not signing apps at all, although 
>> properly supporting signing is on my way too long wish list for py2app.  
> 
> The ability to distribute unsigned apps is not-so-slowly going away; even the 
> ability to distribute non-notarized apps has a very limited shelf-life at 
> this point.  So this ought to be an alarming development for everyone - 
> having Python apps effectively banned from macOS distribution is a big 
> potential problem :-\.
> 
> The good news here is that aside from having to write a little for loop in 
> shell (shown below) getting the app codesigned previously was easy, and my 
> app *did* pass notarization, so nothing that py2app is doing is breaking 
> things on apple's end.  It's just a matter of a ctypes bug.

On that note: more good news.  While I haven't round-tripped through 
notarization again yet, this is a bit less dire than it first appeared.  If I 
prevent the import of ctypes with an `import sys; sys.modules['ctypes'] = 
None`, and add a 'sed' script to my build process to prevent _setup_ctypes from 
running in __boot__, then the app launches again.

Apparently my app doesn't actually need ctypes.

The problem seems to be that Twisted includes a ctypes import; modulegraph sees 
this and thinks there is a hard dependency, and inserts the ctypes setup blob 
into __boot__.  However, this is a conditional import, and it's for Windows 
support anyway.

(There also seem to be problems with cffi-using libraries, but not other shared 
objects, so maybe this is a bug in libffi; however, these don't interfere with 
py2app itself starting up.)

-glyph___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

Re: [Pythonmac-SIG] py2app apps codesigned with "--options runtime", i.e. any notorizable app, fails to launch with ctypes error (was Re: [Pyobjc-dev] ctypes MemoryError when attempting to launch nota

[Glyph]

> On Oct 28, 2018, at 1:48 PM, Ronald Oussoren  wrote:
> 
> 
> 
>> On 28 Oct 2018, at 19:47, Glyph  wrote:
>> 
>> 
>> 
>>> On Oct 28, 2018, at 11:20 AM, Glyph  wrote:
>>> 
>>> 
>>> 
>>>> On Oct 28, 2018, at 2:27 AM, Ronald Oussoren  
>>>> wrote:
>>> 
>>>>> 
>>>>> Curiously, this is the same traceback that comes from 
>>>>> https://forum.kodi.tv/showthread.php?tid=329171, which suggests it's 
>>>>> something fundamental to strict shared-library sandboxing that ctypes 
>>>>> trips over when trying to initialize itself.
>>>>> 
>>>>> Does anyone have experience with this, or ideas about what to do?
>>>> 
>>>> I’m afraid not. I currently get away with not signing apps at all, 
>>>> although properly supporting signing is on my way too long wish list for 
>>>> py2app.  
>>> 
>>> The ability to distribute unsigned apps is not-so-slowly going away; even 
>>> the ability to distribute non-notarized apps has a very limited shelf-life 
>>> at this point.  So this ought to be an alarming development for everyone - 
>>> having Python apps effectively banned from macOS distribution is a big 
>>> potential problem :-\.
>>> 
>>> The good news here is that aside from having to write a little for loop in 
>>> shell (shown below) getting the app codesigned previously was easy, and my 
>>> app *did* pass notarization, so nothing that py2app is doing is breaking 
>>> things on apple's end.  It's just a matter of a ctypes bug.
>> 
>> On that note: more good news.  While I haven't round-tripped through 
>> notarization again yet, this is a bit less dire than it first appeared.  If 
>> I prevent the import of ctypes with an `import sys; sys.modules['ctypes'] = 
>> None`, and add a 'sed' script to my build process to prevent _setup_ctypes 
>> from running in __boot__, then the app launches again.
>> 
>> Apparently my app doesn't actually need ctypes.
> 
> Good to hear that. 
> 
>> 
>> The problem seems to be that Twisted includes a ctypes import; modulegraph 
>> sees this and thinks there is a hard dependency, and inserts the ctypes 
>> setup blob into __boot__.  However, this is a conditional import, and it's 
>> for Windows support anyway.
> 
> Hmm…. I wonder what’s the best way forward here. I could add on option to 
> disable ctypes support, but that is a kludge.  A weak importing hook 
> (something like the never withdrawn PEP 369) could execute this code only 
> when actually needed, but I have no idea how hard it would be to implement 
> this.
> 
> 
>> 
>> (There also seem to be problems with cffi-using libraries, but not other 
>> shared objects, so maybe this is a bug in libffi; however, these don't 
>> interfere with py2app itself starting up.)
> 
> Interesting…  I haven’t had complaints about PyObjC yet, and that also uses 
> libffi.  
> 
> I wonder what the “hardened runtime” option actually does and enforces.   In 
> 3.7 the line in ctypes/__init__.py that causes the exception is a call that 
> creates a dummy C function, and likely triggers the first allocation for 
> storing a libffi closure which could be something the hardened runtime 
> doesn’t like (being writeable + executable memory). 

Interesting. Perhaps what I want is simply 
https://developer.apple.com/documentation/security/com_apple_security_cs_allow-unsigned-executable-memory
 then?  Any chance you know how to jam that into a `codesign` command line 
somehow? :-)

> P.S. I just noticed that the traceback in your initial message doesn’t 
> include the actual exception, just the traceback. 

Oh; it’s “MemoryError”, no exception message.

> Ronald
> 
___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

[Pythonmac-SIG] notarization works! (was Re: not that)

[Glyph]

> On Oct 28, 2018, at 2:57 PM, Glyph  wrote:
> 
>> I wonder what the “hardened runtime” option actually does and enforces.   In 
>> 3.7 the line in ctypes/__init__.py that causes the exception is a call that 
>> creates a dummy C function, and likely triggers the first allocation for 
>> storing a libffi closure which could be something the hardened runtime 
>> doesn’t like (being writeable + executable memory). 
> 
> Interesting. Perhaps what I want is simply 
> https://developer.apple.com/documentation/security/com_apple_security_cs_allow-unsigned-executable-memory
>  
> <https://developer.apple.com/documentation/security/com_apple_security_cs_allow-unsigned-executable-memory>
>  then?  Any chance you know how to jam that into a `codesign` command line 
> somehow? :-)
> 

Thank you so much for this tip, Ronald!  This was much easier than I 
anticipated, and things are working now!

The relevant entitlements file is literally just:


http://www.apple.com/DTDs/PropertyList-1.0.dtd";>


com.apple.security.cs.allow-unsigned-executable-memory




I dropped that in a file, added `--entitlements=$THAT_FILE.plist` to my 
codesign invocations, removed all my workarounds for ctypes et. al. (except for 
the hard-coded 'import _cffi_backend' still necessary to convince modulegraph 
to include enough code for SSL to work), and then tried launching my app.  
Success!  Then I tried notarizing it: also success!  Time permitting, I'll be 
updating my blog post at 
https://glyph.twistedmatrix.com/2018/01/shipping-pygame-mac-app.html 
<https://glyph.twistedmatrix.com/2018/01/shipping-pygame-mac-app.html> with 
this information, and possibly publishing the now unfortunately somewhat 
complex tooling I use to do signing now.

So I don't know if I'm the first to do this, but looking at the archives for 
these lists I seem to be the first to report it: you can successfully codesign 
and notarize apps created with py2app and python 3.6!

It seems to me that whatever "MAP_JIT" is (an mmap flag, I'm guessing?) libffi 
needs to be using it for the memory it places synthetic closures into, so that 
this entitlement won't be necessary with some future version of Python.  But it 
looks like Apple is not pushing particularly hard to deprecate this one right 
now, thank goodness :-).

-glyph___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

Re: [Pythonmac-SIG] [ANN] PyObjC 5.0

[Glyph]

> On Sep 21, 2018, at 4:50 AM, Ronald Oussoren  wrote:
> 
>> P.S.: speaking of build gymnastics, does anyone on this list happen to have 
>> a way to use CocoaPods or something similar to pull in open source Cocoa 
>> frameworks to a PyObjC application?
> 
> I haven’t looked into this yet.  I guess the hardest part is creating the 
> metadata for constants and pointer arguments. Could you file and issue for 
> this?  
> 

Sorry it took a minute to get to this :).  Filed here: 
https://bitbucket.org/ronaldoussoren/py2app/issues/255/tooling-and-or-documentation-to-integrate
 


-glyph___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

Re: [Pythonmac-SIG] [Pyobjc-dev] notarization works! (was Re: not that)

[Glyph]

On Sun, Oct 28, 2018, at 11:58 PM, Ronald Oussoren wrote:
> 
> 
>> On 29 Oct 2018, at 00:56, Glyph  wrote:
>> 
>> 
>> 
>>> On Oct 28, 2018, at 2:57 PM, Glyph  wrote:>>> 
>>>> I wonder what the “hardened runtime” option actually does and
>>>> enforces.   In 3.7 the line in ctypes/__init__.py that causes the
>>>> exception is a call that creates a dummy C function, and likely
>>>> triggers the first allocation for storing a libffi closure which
>>>> could be something the hardened runtime doesn’t like (being
>>>> writeable + executable memory).>>> 
>>> Interesting. Perhaps what I want is simply
>>> https://developer.apple.com/documentation/security/com_apple_security_cs_allow-unsigned-executable-memory
>>> then?  Any chance you know how to jam that into a `codesign` command
>>> line somehow? :-)>>> 
>> 
>> Thank you so much for this tip, Ronald!  This was much easier than I
>> anticipated, and things are working now!> 
> Great.
> 
>> 
>> The relevant entitlements file is literally just:
>> 
>>> 
>>> >> http://www.apple.com/DTDs/PropertyList-1.0.dtd";>>>> 
>>> 
>>> com.apple.security.cs.allow-unsigned-executable-memory
>>> 
>>> 
>>> 
>> 
>> I dropped that in a file, added `--entitlements=$THAT_FILE.plist` to
>> my codesign invocations, removed all my workarounds for ctypes et.
>> al. (except for the hard-coded 'import _cffi_backend' still necessary
>> to convince modulegraph to include enough code for SSL to work), and
>> then tried launching my app.> 
> Which package needs _cffi_backend? I can add a recipe for that to
> py2app to do this automagically.
This may sound obvious, but: cffi :-).  In my case, pyOpenSSL ->
cryptography -> cffi.
> 
>> Success!  Then I tried notarizing it: also success!  Time permitting,
>> I'll be updating my blog post at
>> https://glyph.twistedmatrix.com/2018/01/shipping-pygame-mac-app.html
>> with this information, and possibly publishing the now unfortunately
>> somewhat complex tooling I use to do signing now.>> 
>> So I don't know if I'm the first to *do* this, but looking at the
>> archives for these lists I seem to be the first to *report* it: you
>> can successfully codesign and notarize apps created with py2app and
>> python 3.6!>> 
>> It seems to me that whatever "MAP_JIT" is (an mmap flag, I'm
>> guessing?) libffi needs to be using it for the memory it places
>> synthetic closures into, so that this entitlement won't be necessary
>> with some future version of Python.  But it looks like Apple is not
>> pushing particularly hard to deprecate this one right now, thank
>> goodness :-).> 
> MAP_JIT is a mmap flag that’s apparently introduced in 10.14. The
> slides at https://developer.apple.com/videos/play/wwdc2018/702/
> mention this flag and the hardened runtime.> 
> I guess we should add this flag to the code in
> Modules/_ctypes/malloc_closure.c CPython) and in the similar code in
> PyObjC.  The annoying bit is that the flag is new in 10.14, and
> CPython installers are created on 10.9 which means those won’t include
> the new flag for a long time.> 
> I’ll have to check if using MAP_JIT is ok when deploying on older
> macOS versions, or if the code should do a runtime version check.
I can't shed any light on this, but I suspect the cffi folks will also
have to figure this out, and may already have some sense of how this
works.  I filed an issue with them here:
https://bitbucket.org/cffi/cffi/issues/391/cffi-doesnt-work-inside-a-macos-app-bundle
___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

Re: [Pythonmac-SIG] ANN: PyObjC 11.0

[Glyph]

Thanks Ronald! Congratulations! Definitely been looking forward to this one.

> On Jan 14, 2025, at 12:50 PM, Ronald Oussoren via Pythonmac-SIG 
>  wrote:
> 
> I'm happy to announce the 11.0 release of PyObjC, slightly delayed from the 
> normal major release in October.
> 
> This release has two major features:
> 
> Support for the macOS 15.2 SDK, including new bindings for the frameworks 
> MediaExtension and DeviceDiscoveryExtension
> 
> Experimental support for GIL-less operation in the free-threaded build of 
> Python 3.13
> 
> The latter feature is an important reason for the delay: Supporting GIL-less 
> operation required reworking parts of the internals of PyObjC, both to rely 
> on other locks than the GIL and to avoid CPython APIs that are known to be 
> problematic when the GIL isn't present ("borrowed references" for anyone 
> familiar with the CPython API).
> 
> Free-threaded support is experimental at this point in time and has seen only 
> limited testing for multi-threading. I've also focussed on getting the basics 
> right, and have not worked at optimizing free-threading support. Expect 
> improvements in future releases.
> 
> With PyObjC 11 I'm dropping support for Python 3.8, given that that's gone 
> out of support by the CPython team.
> 
> Finally, there's a number of bug fixes and smaller features mentioned in the 
> changelog at https://pyobjc.readthedocs.io/en/latest/changelog.html
> 
> Ronald
> 
> --
> 
> Twitter / micro.blog: @ronaldoussoren
> Mastodon: @ron...@blog.ronaldoussoren.net.
> Blog: https://blog.ronaldoussoren.net/
> 
> ___
> Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
> https://mail.python.org/mailman/listinfo/pythonmac-sig
> unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

[Pythonmac-SIG] Re: Do we want to keep pythonmac-sig?

[glyph]

> On May 16, 2025, at 6:52 PM, Matthew Dixon Cowles  wrote:
> 
> Jack,
> 
>> I just got a mail from the Python mailman server admin that they
>> will shortly be upgrading the mailman server, and the question is
>> whether to keep the Pythonmac-SIG mailing list alive or archive it.
> 
>> Personally I don’t mind archiving it, we've had about 2-4
>> messages per year over the last years.
> 
> I think that the fact that there isn't a lot of traffic should be
> taken as a sign of success. You and the other people who work on the
> releases deserve a lot of credit.
> 
> For myself, I'm glad that there's a place where it's possible to get
> the attention of the platform experts because very occasionally a
> platform issue comes up. For example, a month ago we got a question
> over on the python-help list about Python for Windows on ARM and it
> was a little trouble tracking down someone who could answer it. (In
> case anyone's curious: Python for Windows on ARM isn't on Microsoft's
> store because few binary modules are compiled for it and Microsoft
> doesn't want users to be mad and/or confused when, for example, "pip
> install pandas" doesn't work for it.)
> 
> That said, I don't much mind whether that place is here on this list
> or over on a section of the PSF's web board. I tend to like mailing
> lists a little more, but that's a very minor thing.

Similarly, I'd like for there to be somewhere to go, and the list is 
convenient, but as long as the archiving comes along with an official 
designation of a thread on the Discourse board or something, that's all I would 
need to be happy.

-g
___
Pythonmac-SIG mailing list -- pythonmac-sig@python.org
To unsubscribe send an email to pythonmac-sig-le...@python.org
https://mail.python.org/mailman3//lists/pythonmac-sig.python.org
Member address: arch...@mail-archive.com

Re: [Pythonmac-SIG] PyObjC and macOS 10.12 (Sierra)

[Glyph Lefkowitz]

> On Sep 13, 2016, at 3:35 PM, Andrew Jaffe  wrote:
> 
> Aha!
> 
>  $ ls -lt /Library/Python/2.7/site-packages/
>  total 0
>  -rwxr-xr-x  1 root  wheel  157 31 Jul 02:36 Extras.pth*
>  -rw-r--r--  1 root  wheel  119 31 Jul 02:36 README
>  $ more /Library/Python/2.7/site-packages/Extras.pth 
>  /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
>  
> /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC
> 
> Now I wonder how those got there?!
> 

Hah!  Thanks for sharing.  Very satisfying to actually make a *correct* 
prediction about setuptools' behavior :)

-glyph___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

Re: [Pythonmac-SIG] PyObjC and macOS 10.12 (Sierra)

[Glyph Lefkowitz]

> On Sep 13, 2016, at 12:05 PM, Jack Jansen  wrote:
> 
> I think 
> /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages 
> is a very old location for storing Python packages. Recently things have been 
> installed in /Library/Python/2.7/site-packages.
> 
> Could it be that you’ve installed pyobjc a couple of OSX releases ago?

This is always worth checking ;).  Particularly if it was a few Setuptools 
releases ago.  Also worth checking: ~/Library/Python.

> And could it be that the OSX upgrade that introduced SIP somehow didn’t clean 
> out user-installed things from /Library/Frameworks before turning off write 
> permission?

SIP locks down /System, not /Library.

> A possible workaround is to turn off SIP (or boot from the recovery 
> partition), record what is in 
> /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages 
> and then clean it out. Then after a reboot re-install the packages you’re 
> still using.


This should be an _absolute_ last resort, though.  You should be able to clean 
out /Library just fine.  If you have a /Library/Frameworks/Python.framework, 
that's probably Python.org  python, not system python.

-glyph___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

Re: [Pythonmac-SIG] PyObjC and macOS 10.12 (Sierra)

[Glyph Lefkowitz]

> On Sep 13, 2016, at 3:05 PM, Andrew Jaffe  wrote:
> 
> But this is the framework (non-apple!) build!…

"framework build" refers to the way that Python is built.  Apple's python, 
Python.org's python, and Homebrew's python are all framework builds.  So, to be 
clear: this is python.org python?

> And, again: why isn’t everyone seeing this all the time? (And why didn’t I 
> see it before?)

There's probably a .pth file somewhere that is stuffing Extras on your 
sys.path.  easy_install (especially older versions) is notorious for creating 
such things in ways that are hard to inspect for.  My usual recommendation at 
this point is to blow away _everything_ in /Library/Python, ~/Library/Python, 
~/.local/lib/python*, and /Library/Frameworks/Python.framework and then 
reinstall from scratch.  (Thanks to SIP you don't need to blow away anything in 
/System anymore; hooray!)

-glyph___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

Re: [Pythonmac-SIG] PyObjC and macOS 10.12 (Sierra)

[Glyph Lefkowitz]

> On Sep 20, 2016, at 10:29 AM, Andrew Jaffe  wrote:
> 
> On 17/09/2016 18:59, Glyph Lefkowitz wrote:
>> 
>>> On Sep 17, 2016, at 9:27 AM, Ned Deily  wrote:
>>> 
>>> On 2016-09-13 19:33, Glyph Lefkowitz wrote:
>>>>> On Sep 13, 2016, at 3:35 PM, Andrew Jaffe >>>> <mailto:a.h.ja...@gmail.com>> wrote:
>>>>> 
>>>>> Aha!
>>>>> 
>>>>> $ ls -lt /Library/Python/2.7/site-packages/
>>>>> total 0
>>>>> -rwxr-xr-x  1 root  wheel  157 31 Jul 02:36 Extras.pth*
>>>>> -rw-r--r--  1 root  wheel  119 31 Jul 02:36 README
>>>>> $ more /Library/Python/2.7/site-packages/Extras.pth
>>>>> /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
>>>>> /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC
>>>>> 
>>>>> Now I wonder how those got there?!
>>>>> 
>>>> 
>>>> Hah!  Thanks for sharing.  Very satisfying to actually make a *correct*
>>>> prediction about setuptools' behavior :)
>>> 
>>> This seems to be Apple's doing.  AFAICT, 10.12 is shipping with this
>>> Extras.pth file in /Library/Python/2.7; it's something new.  And,
>>> unfortunately, due to https://bugs.python.org/issue4865, the
>>> site-packages directory for the system Python 2.7 is included in
>>> sys.path along with the non-system framework Python site-packages.
>> 
>> Hrm.  I guess everyone I knew on the beta was using homebrew python :(.
>> 
>> I'm surprised that Apple is putting stuff in /Library.  I don't have a 
>> Sierra box handy - /Library isn't SIP-protected now, is it?
> 
> Nope, that's not a problem.
>> 
>> This seems wrong; someone should file a radar (and probably share on 
>> http://www.openradar.me <http://www.openradar.me/> for further discussion).
> 
> In the meantime, what's the recommended workaround?

Looking back over the thread, my first reply was: "Make a virtualenv and 
install pyobjc there", but I didn't see a direct response to that.  That would 
still be my first choice for a workaround.  Is there some reason that doesn't 
work for you?

-glyph___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

Re: [Pythonmac-SIG] [Pyobjc-dev] PyObjC and macOS 10.12 (Sierra)

[Glyph Lefkowitz]

> On Oct 11, 2016, at 2:26 PM, Ronald Oussoren  wrote:
> 
> BTW. Has anyone experience with using LLDB on Sierra? I’m currently running 
> Sierra in a VM and for some reason LLDB doesn’t appear to work, in an SSH 
> session I cannot start programs at all and on the console a crashing bug in 
> the interpreter exists the interpreter instead of breaking into the debugger. 
> That’s rather annoying when you’re trying to a debug a crash on 10.12 that 
> doesn’t happen on 10.11.

Ever since the introduction of SIP, system-level debugging tools like LLDB have 
worked very poorly for me, even on self-built executables.  I don't see a 
difference in its behavior on 10.11 vs. 10.12 though.  I can run it on `curl` 
from Homebrew, but not `python`; my guess being that Python is trying to 
dlopen() some SIP-protected thing whereas `curl` is loading only things from 
Homebrew?

If it's a VM, then system integrity is less of a concern; have you tried just 
blanket disabling SIP to see if that improves the situation?  I've had some 
luck with other tools (dtrace, mostly) working in that configuration.  (I had 
to totally disable SIP though, disabling one flag at a time didn't work.)

-glyph

___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

Re: [Pythonmac-SIG] PyObjC and macOS 10.12 (Sierra)

[Glyph Lefkowitz]

> On Sep 17, 2016, at 9:27 AM, Ned Deily  wrote:
> 
> On 2016-09-13 19:33, Glyph Lefkowitz wrote:
>>> On Sep 13, 2016, at 3:35 PM, Andrew Jaffe >> <mailto:a.h.ja...@gmail.com>> wrote:
>>> 
>>> Aha!
>>> 
>>> $ ls -lt /Library/Python/2.7/site-packages/
>>> total 0
>>> -rwxr-xr-x  1 root  wheel  157 31 Jul 02:36 Extras.pth*
>>> -rw-r--r--  1 root  wheel  119 31 Jul 02:36 README
>>> $ more /Library/Python/2.7/site-packages/Extras.pth 
>>> /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
>>> /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC
>>> 
>>> Now I wonder how those got there?!
>>> 
>> 
>> Hah!  Thanks for sharing.  Very satisfying to actually make a *correct*
>> prediction about setuptools' behavior :)
> 
> This seems to be Apple's doing.  AFAICT, 10.12 is shipping with this
> Extras.pth file in /Library/Python/2.7; it's something new.  And,
> unfortunately, due to https://bugs.python.org/issue4865, the
> site-packages directory for the system Python 2.7 is included in
> sys.path along with the non-system framework Python site-packages.

Hrm.  I guess everyone I knew on the beta was using homebrew python :(.

I'm surprised that Apple is putting stuff in /Library.  I don't have a Sierra 
box handy - /Library isn't SIP-protected now, is it?

This seems wrong; someone should file a radar (and probably share on 
http://www.openradar.me for further discussion).

-glyph

___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

Re: [Pythonmac-SIG] PyObjC and macOS 10.12 (Sierra)

[Glyph Lefkowitz]

> On Oct 12, 2016, at 11:49 AM, Andrew Jaffe  wrote:
> 
> Not me. If I understand correctly, Glyph -- who undoubtedly understand the 
> situation better than I do -- still thinks that there's no actual bug here, 
> since we shouldn't be using the framework build this way, but I'm not sure I 
> understand/agree...

To be honest, I'm not 100% sure this is a good idea either; it's just that I 
know Donald Stufft has had a terrible time with Apple system python for several 
years, and he regards this as a positive change.  From my personal perspective, 
there's a good case to be made that a python in /System should just load from 
/System and one in /Library should load only from /Library, similar to the way 
--prefix works on "regular" UNIX.  But, this is what we've got :).

-glyph___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

Re: [Pythonmac-SIG] PyObjC and macOS 10.12 (Sierra)

[Glyph Lefkowitz]

> On Sep 21, 2016, at 01:52, Andrew Jaffe  wrote:
> 
> That would work, and in fact I don't really need PyObjC (sorry, Ronald!) but 
> I've got my whole setup working with the "global" python.org 
>  framework build, so I am used to that... and the Sierra 
> status quo does seem ugly (and quite possibly is a bug!).

Well, Apple ships certain libraries, and they want them to be on your path.

The nice thing about virtualenv is that, along with pip wheel caching, 
re-creating them is really fast, so you don't have to just do one setup; you 
just make a requirements.txt for each separate environment you want, and 
regularly re-create it, so you know that if you give directions to anyone else, 
your setup will work for them as well.

One bad habit we get into is getting a full-on environment set up on our own 
workstation, and then not knowing exactly what we installed, and therefore what 
we need in order for someone else to build and work on our software.  
Virtualenv helps reduce that problem a bit.

-glyph___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

Re: [Pythonmac-SIG] PyObjC and macOS 10.12 (Sierra)

[Glyph Lefkowitz]

> On Oct 6, 2016, at 2:56 AM, Andrew Jaffe  wrote:
> 
> On 17/09/2016 18:59, Glyph Lefkowitz wrote:
>> 
>>> On Sep 17, 2016, at 9:27 AM, Ned Deily  wrote:
>>> 
>>> On 2016-09-13 19:33, Glyph Lefkowitz wrote:
>>>>> On Sep 13, 2016, at 3:35 PM, Andrew Jaffe >>>> <mailto:a.h.ja...@gmail.com>> wrote:
>>>>> 
>>>>> Aha!
>>>>> 
>>>>> $ ls -lt /Library/Python/2.7/site-packages/
>>>>> total 0
>>>>> -rwxr-xr-x  1 root  wheel  157 31 Jul 02:36 Extras.pth*
>>>>> -rw-r--r--  1 root  wheel  119 31 Jul 02:36 README
>>>>> $ more /Library/Python/2.7/site-packages/Extras.pth
>>>>> /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
>>>>> /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC
>>>>> 
>>>>> Now I wonder how those got there?!
>>>>> 
>>>> 
>>>> Hah!  Thanks for sharing.  Very satisfying to actually make a *correct*
>>>> prediction about setuptools' behavior :)
>>> 
>>> This seems to be Apple's doing.  AFAICT, 10.12 is shipping with this
>>> Extras.pth file in /Library/Python/2.7; it's something new.  And,
>>> unfortunately, due to https://bugs.python.org/issue4865, the
>>> site-packages directory for the system Python 2.7 is included in
>>> sys.path along with the non-system framework Python site-packages.
>> 
> 
> >
> 
> So, a little more data:
> 
> If you rename or remove /Library/Python/2.7/site-packages/Extras.pth then 
> pip2 works.

What do you mean by "works"?  Your original error is pip refusing to upgrade 
pyObjC because to upgrade pyObjC it has to delete the old version, and pyObjC 
is part of the operating system, and it can't delete the installed version.  
This is correct; the error reporting could be nicer, but pip is not broken.  
Don't mess with files in /System.

The suggestion to use virtualenv isn't really optional.  If you really, really 
want things to be importable by a bare 'python', not inside a venv, `pip 
install --user` is another option you might have.

> *However*, lots of other stuff breaks -- anything that uses Apple's python 
> and relies on access to pyobjc and the frameworks (e.g., TextMate's latex 
> package).

Yep, that's expected behavior.  This is exactly why Apple is making it 
increasingly difficult to screw up /System.  Apps can, should, and do rely upon 
the libraries installed on the system.

> What I don't understand is: what changed from Yosemite? This file did not 
> exist before Sierra, but there were no problems with (Apple) python accessing 
> these packages.

Do you mean from El Capitan?

This file previously existed in a different location, and (while the 
particulars stubbornly refuse to stick to memory for me, for some reason) this 
new location is the one generally preferred by the python packaging maintainers.

> (Or is there something unique in my setup that is causing this? I kind of 
> doubt it, but it's possible...)
> 
> Does anyone have any insight?

If you really, really, really want to do this in such a way that /System stuff 
is only present for /System's python and not for python.org's, you can take 
advantage of the 'import' hack <https://docs.python.org/2/library/site.html>, 
and rewrite /Library/Python/2.7/site-packages/Extras.pth to _conditionally_ add 
those entries to sys.path, checking sys.executable or some other fingerprint.

But: don't.  Given that system python and python.org python share /Library and 
~/Library, they're not really distinct environments, and things installed into 
one will show up in the other, so excluding the /System directories on one but 
not the other will just cause other, more confusing issues.

In conclusion: just use virtualenv.  This is not a problem that should be fixed.

-glyph___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

Re: [Pythonmac-SIG] PyObjC and macOS 10.12 (Sierra)

[Glyph Lefkowitz]

> On Oct 7, 2016, at 1:36 AM, Andrew Jaffe  wrote:
> 
> On 06/10/2016 20:26, Glyph Lefkowitz wrote:
>>> On Oct 6, 2016, at 2:56 AM, Andrew Jaffe  wrote:
>>> On 17/09/2016 18:59, Glyph Lefkowitz wrote:
>>>>> On Sep 17, 2016, at 9:27 AM, Ned Deily  wrote:
>>>>> On 2016-09-13 19:33, Glyph Lefkowitz wrote:
>>>>>>> On Sep 13, 2016, at 3:35 PM, Andrew Jaffe wrote:
>>>>>>> 
>>>>>>> $ ls -lt /Library/Python/2.7/site-packages/
>>>>>>> total 0
>>>>>>> -rwxr-xr-x  1 root  wheel  157 31 Jul 02:36 Extras.pth*
>>>>>>> -rw-r--r--  1 root  wheel  119 31 Jul 02:36 README
>>>>>>> $ more /Library/Python/2.7/site-packages/Extras.pth
>>>>>>> /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
>>>>>>> /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC
>>>>>>> 
>>>>>> Hah!  Thanks for sharing.  Very satisfying to actually make a *correct*
>>>>>> prediction about setuptools' behavior :)
>>>>> 
>>>>> This seems to be Apple's doing.  AFAICT, 10.12 is shipping with this
>>>>> Extras.pth file in /Library/Python/2.7; it's something new.  And,
>>>>> unfortunately, due to https://bugs.python.org/issue4865, the
>>>>> site-packages directory for the system Python 2.7 is included in
>>>>> sys.path along with the non-system framework Python site-packages.
>>>> 
>>> So, a little more data:
>>> 
>>> If you rename or remove /Library/Python/2.7/site-packages/Extras.pth
>>> then pip2 works.
>> 
>> What do you mean by "works"?  Your original error is pip refusing to
>> upgrade pyObjC because to upgrade pyObjC it has to delete the old
>> version, and pyObjC is part of the operating system, and it can't delete
>> the installed version.  This is correct; the error reporting could be
>> nicer, but pip is not broken.  Don't mess with files in /System.
>> 
>> The suggestion to use virtualenv isn't really optional.  If you really,
>> really want things to be importable by a bare 'python', not inside a
>> venv, `pip install --user` is another option you might have.
>> 
>>> *However*, lots of other stuff breaks -- anything that uses Apple's
>>> python and relies on access to pyobjc and the frameworks (e.g.,
>>> TextMate's latex package).
>> 
>> Yep, that's expected behavior.  This is exactly why Apple is making it
>> increasingly difficult to screw up /System.  Apps can, should, and do
>> rely upon the libraries installed on the system.
>> 
>>> What I don't understand is: what changed from Yosemite? This file did
>>> not exist before Sierra, but there were no problems with (Apple)
>>> python accessing these packages.
>> 
>> Do you mean from El Capitan?
> 
> Yes, sorry.
> 
>> This file previously existed in a different location, and (while the
>> particulars stubbornly refuse to stick to memory for me, for some
>> reason) this new location is the one generally preferred by the python
>> packaging maintainers.
>> 
>>> (Or is there something unique in my setup that is causing this? I kind
>>> of doubt it, but it's possible...)
>>> 
>>> Does anyone have any insight?
>> 
>> If you really, really, really want to do this in such a way that /System
>> stuff is only present for /System's python and not for python.org
>> <http://python.org>'s, you can take advantage of the 'import' hack
>> <https://docs.python.org/2/library/site.html>, and
>> rewrite /Library/Python/2.7/site-packages/Extras.pth to _conditionally_
>> add those entries to sys.path, checking sys.executable or some other
>> fingerprint.
>> 
>> But: don't.  Given that system python and python.org <http://python.org>
>> python share /Library and ~/Library, they're not /really/ distinct
>> environments, and things installed into one will show up in the other,
>> so excluding the /System directories on one but not the other will just
>> cause other, more confusing issues.
>> 
>> In conclusion: just use virtualenv.  This is not a problem that should
>> be fixed.
> 
> I agree that this would solve all of the problems, at the cost of some minor 
> startup pain.
> 
> But one nice thing about the python.org builds is that, for the last few 
> releases,

Re: [Pythonmac-SIG] PyObjC 3.2 released

[Glyph Lefkowitz]

> On Dec 6, 2016, at 8:59 AM, Ronald Oussoren  wrote:
> 
> Hi,
> 
> I’ve pushed PyObjC 3.2 to PyPI (finally…). The major new feature in this 
> version is support for OSX 10.12 (Sierra) and the new APIs introduced in this 
> version of OSX.

🎉😃🎉

> There is also a backward incompatible change: the default method signature is 
> calculated differently than before. This will primarily affect code that adds 
> python-only method to a class, like so:
> 
>   class MyObject (NSObject):
>def amethod(self, a, b): pass

Huh, I didn't even realize that was possible before.  Hopefully none of my code 
will be affected as a result :-).

Is there any chance that now this release is done, wheel builds might be on the 
horizon? :)

-glyph

___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

Re: [Pythonmac-SIG] PyObjC 3.2 released

[Glyph Lefkowitz]

> On Dec 6, 2016, at 10:36 AM, Ronald Oussoren  wrote:
> 
> 
>> On 6 Dec 2016, at 19:19, Glyph Lefkowitz  wrote:
>> 
>> 
>>> On Dec 6, 2016, at 8:59 AM, Ronald Oussoren  wrote:
>>> 
>>> Hi,
>>> 
>>> I’ve pushed PyObjC 3.2 to PyPI (finally…). The major new feature in this 
>>> version is support for OSX 10.12 (Sierra) and the new APIs introduced in 
>>> this version of OSX.
>> 
>> 🎉😃🎉
>> 
>>> There is also a backward incompatible change: the default method signature 
>>> is calculated differently than before. This will primarily affect code that 
>>> adds python-only method to a class, like so:
>>> 
>>> class MyObject (NSObject):
>>>  def amethod(self, a, b): pass
>> 
>> Huh, I didn't even realize that was possible before.  Hopefully none of my 
>> code will be affected as a result :-).
> 
> It wasn’t really documented, but I wouldn’t be surprised if someone relies on 
> the previous behavior…
> 
>> 
>> Is there any chance that now this release is done, wheel builds might be on 
>> the horizon? :)
> 
> Maybe.  The important bit w.r.t. wheel builds is that they must be automated, 
> otherwise releases take too much time and I end up not doing releases at all. 
>  That said, automating this shouldn’t be too hard and should be able to save 
> me some time in the end (I currently also test manually…)

If manual source releasing is `python setup.py sdist && twine upload dist/*`, 
then "manual" wheel releasing is `python setup.py sdist bdist_wheel && twine 
upload dist/*`.  It doesn't seem like that should be a crushing maintenance 
burden :-).

Although perhaps there's something I'm missing?  The subtleties of binary 
compatibility often escape me.

-glyph

___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

Re: [Pythonmac-SIG] PyObjC 3.2 released

[Glyph Lefkowitz]

> On Dec 6, 2016, at 12:17 PM, Ronald Oussoren  wrote:
> 
>> 
>> On 6 Dec 2016, at 20:30, Glyph Lefkowitz > <mailto:gl...@twistedmatrix.com>> wrote:
>> 
>> 
>>> On Dec 6, 2016, at 10:36 AM, Ronald Oussoren >> <mailto:ronaldousso...@mac.com>> wrote:
>>> 
>>> 
>>>> On 6 Dec 2016, at 19:19, Glyph Lefkowitz >>> <mailto:gl...@twistedmatrix.com>> wrote:
>>>> 
>>>> 
>>>>> On Dec 6, 2016, at 8:59 AM, Ronald Oussoren >>>> <mailto:ronaldousso...@mac.com>> wrote:
>>>>> 
>>>>> Hi,
>>>>> 
>>>>> I’ve pushed PyObjC 3.2 to PyPI (finally…). The major new feature in this 
>>>>> version is support for OSX 10.12 (Sierra) and the new APIs introduced in 
>>>>> this version of OSX.
>>>> 
>>>> 🎉😃🎉
>>>> 
>>>>> There is also a backward incompatible change: the default method 
>>>>> signature is calculated differently than before. This will primarily 
>>>>> affect code that adds python-only method to a class, like so:
>>>>> 
>>>>> class MyObject (NSObject):
>>>>> def amethod(self, a, b): pass
>>>> 
>>>> Huh, I didn't even realize that was possible before.  Hopefully none of my 
>>>> code will be affected as a result :-).
>>> 
>>> It wasn’t really documented, but I wouldn’t be surprised if someone relies 
>>> on the previous behavior…
>>> 
>>>> 
>>>> Is there any chance that now this release is done, wheel builds might be 
>>>> on the horizon? :)
>>> 
>>> Maybe.  The important bit w.r.t. wheel builds is that they must be 
>>> automated, otherwise releases take too much time and I end up not doing 
>>> releases at all.  That said, automating this shouldn’t be too hard and 
>>> should be able to save me some time in the end (I currently also test 
>>> manually…)
>> 
>> If manual source releasing is `python setup.py sdist && twine upload 
>> dist/*`, then "manual" wheel releasing is `python setup.py sdist bdist_wheel 
>> && twine upload dist/*`.  It doesn't seem like that should be a crushing 
>> maintenance burden :-).
>> 
>> Although perhaps there's something I'm missing?  The subtleties of binary 
>> compatibility often escape me.
> 
> That’s about it, although I don’t use twine (see 
> <https://bitbucket.org/ronaldoussoren/pyobjc/src/3e6dce3dcfa0a3a72f9a204855d83644eaf6afc0/development-support/upload-release?at=default&fileviewer=file-view-default
>  
> <https://bitbucket.org/ronaldoussoren/pyobjc/src/3e6dce3dcfa0a3a72f9a204855d83644eaf6afc0/development-support/upload-release?at=default&fileviewer=file-view-default>>).
>   I’m slightly worried
> about binary compatibility and want to have some way to do smoke tests of the 
> generated wheels on various platforms.

Twine is strongly recommended by the packaging community these days.  Mostly 
the security problems with `setup.py upload` have been fixed, but there's no 
easy way to check, especially on the command line, so it remains somewhat risky 
to use, especially in a scripted context.

That said, the process is mostly the same; just stick `bdist_wheel` in on that 
command line somewhere before `upload` and you're good to go.

> In theory binary compatibility shouldn’t be a problem, I used to do builds on 
> newer OSX releases when the system where the application was running but that 
> was a couple of releases ago (both of PyObjC and OSX).
> 
> Binary compatibility is likely a larger problem for CPython itself, due to 
> configure picking up some definitions that aren’t available on older OSX 
> releases (IIRC functions like openat(2) are new in 10.10, that causes 
> problems with CPython 3.5 that uses there functions when avaiable.  Nothing 
> too hard, but also not something I have a real need for at the moment (and 
> adding weak-linking code-paths could result in patches that won’t be 
> accepted…).  
> 
> BTW. The reason I’m avoiding this work is more that I’d like to be able to 
> kick of a script that runs a long time and reports on test results on a 
> number of OSX and Python versions, instead of me doing that work. I think I 
> have a plan to get there, and that should make it fairly trivial to generate 
> wheels (and sdists) as a side effect of the test run.

IMHO the pareto principle applies here.  Even with potentially "bad" (untested) 
wheels, for most people, using recent OS X, `pip instal pyobjc` will go a

Re: [Pythonmac-SIG] PyObjC 3.2.1 released

[Glyph Lefkowitz]

> On Dec 12, 2016, at 1:25 AM, Ronald Oussoren  wrote:
> 
> Hi,
> 
> I’ve just pushed PyObjC 3.2.1 to PyPI. This fixes a number of small issues in 
> PyObjC 3.2, but the primary new feature is that there are now wheels on PyPI. 
> 
> I’ve tested the wheels on OSX 10.12 with the Python.org <http://python.org/> 
> “intel” installers (the default download option for the OSX installers) and 
> Homebrew. The wheels should also work fine with older OSX releases. 

Thank you *so* much, Ronald :D.  I installed pyobjc onto my laptop just now and 
didn't even have to go get an icepack out of the freezer first :-).  In under 
ten seconds, no less, for both python 3.5 and python 2.7.  A glorious day, to 
be sure.

One minor note - I do notice that the 'pyobjc' package itself has no wheel, 
even though all its dependents do.  Any chance you could upload 2.7/3.5 wheels 
for that as well?  It would just complete the set :-).

Thanks again; this is fabulous,

-glyph

___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

Re: [Pythonmac-SIG] [Pyobjc-dev] PyObjC 3.2 released

[Glyph Lefkowitz]

> On Dec 6, 2016, at 12:55 PM, Ronald Oussoren  wrote:
> 
> 
>> On 6 Dec 2016, at 21:43, Glyph Lefkowitz > <mailto:gl...@twistedmatrix.com>> wrote:
>> 
>> 
>>> Anyways, there may be a 3.2.1 release by the end of the week, I just 
>>> noticed that building PyObjC on 10.12 doesn’t work when using python 2.7, I 
>>> apparently only tested python 2.7 on older OSX releases :-(
>> 
>> Thanks for saving me some time - I was just about to test that configuration 
>> :).  It works fine on 10.11, which is what this machine is running!
> 
> What version of Xcode do you use?  And do you use a framework build of 
> Python?  The issue I saw earlier should be present on 10.11 as well when 
> using Xcode 8.1 (which includes the 10.12 SDK)

So... technically the answer for Xcode is "8.1" but when I launched it to find 
out it asked me to "install required components"... but even after installing 
them it builds fine (with pip --no-use-wheel to avoid using my 
previously-cached wheels from the earlier attempt).  I also tried pip wheel 
--no-cache-dir just in case; it works that way as well.

I use a framework build of 2.7, from Homebrew.

Given the confusion you've expressed, I went ahead and tried to build this for 
my Sierra machine as well, and it worked there too.  So as far as I can tell 
there is no bug, but I haven't tried with the system-provided Python.

Happy to report any other salient details though; let me know.

-glyph

___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

Re: [Pythonmac-SIG] PyObjC 3.2 released

[Glyph Lefkowitz]

> On Dec 7, 2016, at 12:21 AM, Ronald Oussoren  wrote:
> 
> 
>> On 7 Dec 2016, at 00:28, Glyph Lefkowitz > <mailto:gl...@twistedmatrix.com>> wrote:
>> 
>> Was this intentionally off-list?  (Feel free to forward my reply back to the 
>> list if not).
> 
> It wasn’t and I’ll continue the conversation on-list.
> 
>> 
>>> On Dec 6, 2016, at 2:31 PM, Ronald Oussoren >> <mailto:ronaldousso...@mac.com>> wrote:
>>> 
>>>> 
>>>> On 6 Dec 2016, at 21:43, Glyph Lefkowitz >>> <mailto:gl...@twistedmatrix.com>> wrote:
>>>> 
>>>>> 
>>>>> On Dec 6, 2016, at 12:17 PM, Ronald Oussoren >>>> <mailto:ronaldousso...@mac.com>> wrote:
>>>>> 
>>>>>> 
>>>>>> On 6 Dec 2016, at 20:30, Glyph Lefkowitz >>>>> <mailto:gl...@twistedmatrix.com>> wrote:
>>>>>> 
>>>>>> 
>>>>>>> On Dec 6, 2016, at 10:36 AM, Ronald Oussoren >>>>>> <mailto:ronaldousso...@mac.com>> wrote:
>>>>>>> 
>>>>>>> 
>>>>>>>> On 6 Dec 2016, at 19:19, Glyph Lefkowitz >>>>>>> <mailto:gl...@twistedmatrix.com>> wrote:
>>>>>>>> 
>>>>>>>> 
>>>>>>>>> On Dec 6, 2016, at 8:59 AM, Ronald Oussoren >>>>>>>> <mailto:ronaldousso...@mac.com>> wrote:
>>>>>>>>> 
>>>>>>>>> Hi,
>>>>>>>>> 
>>>>>>>>> I’ve pushed PyObjC 3.2 to PyPI (finally…). The major new feature in 
>>>>>>>>> this version is support for OSX 10.12 (Sierra) and the new APIs 
>>>>>>>>> introduced in this version of OSX.
>>>>>>>> 
>>>>>>>> 🎉😃🎉
>>>>>>>> 
>>>>>>>>> There is also a backward incompatible change: the default method 
>>>>>>>>> signature is calculated differently than before. This will primarily 
>>>>>>>>> affect code that adds python-only method to a class, like so:
>>>>>>>>> 
>>>>>>>>> class MyObject (NSObject):
>>>>>>>>> def amethod(self, a, b): pass
>>>>>>>> 
>>>>>>>> Huh, I didn't even realize that was possible before.  Hopefully none 
>>>>>>>> of my code will be affected as a result :-).
>>>>>>> 
>>>>>>> It wasn’t really documented, but I wouldn’t be surprised if someone 
>>>>>>> relies on the previous behavior…
>>>>>>> 
>>>>>>>> 
>>>>>>>> Is there any chance that now this release is done, wheel builds might 
>>>>>>>> be on the horizon? :)
>>>>>>> 
>>>>>>> Maybe.  The important bit w.r.t. wheel builds is that they must be 
>>>>>>> automated, otherwise releases take too much time and I end up not doing 
>>>>>>> releases at all.  That said, automating this shouldn’t be too hard and 
>>>>>>> should be able to save me some time in the end (I currently also test 
>>>>>>> manually…)
>>>>>> 
>>>>>> If manual source releasing is `python setup.py sdist && twine upload 
>>>>>> dist/*`, then "manual" wheel releasing is `python setup.py sdist 
>>>>>> bdist_wheel && twine upload dist/*`.  It doesn't seem like that should 
>>>>>> be a crushing maintenance burden :-).
>>>>>> 
>>>>>> Although perhaps there's something I'm missing?  The subtleties of 
>>>>>> binary compatibility often escape me.
>>>>> 
>>>>> That’s about it, although I don’t use twine (see 
>>>>> <https://bitbucket.org/ronaldoussoren/pyobjc/src/3e6dce3dcfa0a3a72f9a204855d83644eaf6afc0/development-support/upload-release?at=default&fileviewer=file-view-default
>>>>>  
>>>>> <https://bitbucket.org/ronaldoussoren/pyobjc/src/3e6dce3dcfa0a3a72f9a204855d83644eaf6afc0/development-support/upload-release?at=default&fileviewer=file-view-default>>).
>>>>>   I’m slightly worried
>>>>> about binary compatibility and want to have some way to do smoke tests of 
>>>>

Re: [Pythonmac-SIG] PyObjC 3.2 released

[Glyph Lefkowitz]

> On Dec 8, 2016, at 3:37 AM, Ronald Oussoren  wrote:
> 
> 
>> On 8 Dec 2016, at 00:40, Glyph Lefkowitz > <mailto:gl...@twistedmatrix.com>> wrote:
>>> 
>>> 
>>> Last time I checked systems like Homebrew only build 64-bit binaries (on 
>>> anything resembling modern hardware). It should be possible to support both 
>>> i386/x86_64 python.org <http://python.org/> and homebrew using the same 
>>> wheel file by tweaking the platform tag. The filename of the wheels will 
>>> get annoyingly long, but that’s something users won’t see anyway.   If that 
>>> doesn’t work out I’ll install homebrew on my build machine and have the 
>>> build script generate wheels for that as well. 
>> 
>> You shouldn't need to do this; and in fact you shouldn't do it :).  If you 
>> tweak the platform tag, you'll break Pip's ability to discover the wheel as 
>> valid-to-install for the current platform.  Platform tags are a descriptive 
>> mechanism for Python, Pip, and Setuptools, not a point for user 
>> extensibility.
> 
> I know how platform tags work, I’ve read the relevant PEPs when the were 
> proposed. The “tweak” I wanted to use is documented as “Compressed Tag Sets” 
> in PEP 425, and is how the python version is added to the wheel filename for 
> wheels supporting both python2 and python3, as in:
> 
>pyobjc_framework_XgridFoundation-3.3a0-py2.py3-none-any.whl
> 
> This should work in the platform tag as well, and is already used by wheels 
> for matplotlib (see https://pypi.python.org/pypi/matplotlib 
> <https://pypi.python.org/pypi/matplotlib>).

Aah, I was reading you backwards.  I thought you were saying something about 
inventing a platform tag for Homebrew (since they don't have one of their own; 
they build specifically to be compatible with system python).  This makes sense.

>> There are a lot of fiddly nuances here, but basically, "intel" is the 
>> architecture for "fat binary, intel", and "x86_64" is the architecture for 
>> "64-bit".  (I have no idea what the story is on 10.5 and previous but IMHO 
>> you can skip bothering to build wheels for them for now).  If you want to 
>> build release wheels, just build "intel" (fat binary) and any Python will be 
>> happy with them.
> 
> I wrote the distutils code that sets the platform to a sane value for fat 
> binaries on OSX ;-).  

Heh. I think we both know a little too much about this for our own good :-).

> That’s cool, this means pip did implement the logic that’s needed to 
> understand that “intel” is an acceptable architecture when looking for a 
> wheel for an “x86_64” build of CPython.

I believe it understands the tags pretty thoroughly; I haven't tested, but I 
believe it also knows that an x86_64 wheel won't cut it on an 'intel' python 
because it might blow up later.

>> 
>> This is probably worth reading, as it explains the above in a lot more 
>> detail: https://github.com/MacPython/wiki/wiki/Spinning-wheels 
>> <https://github.com/MacPython/wiki/wiki/Spinning-wheels>.  But the lede is: 
>> just use the default behavior of a fat binary Python, it will do the right 
>> thing :).
> 
> Looking at that page the relevant logic was added in pip 6, that’s old enough 
> to not worry about using a compressed tag set for the platform. 

Cool.  I'm really looking forward to never building pyobjc again :-).

-glyph___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

Re: [Pythonmac-SIG] PyObjC and macOS 10.12 (Sierra)

[Glyph Lefkowitz]

> On Dec 13, 2016, at 8:37 AM, Christopher Barker  wrote:
> 
> Personally, I have avoided all this mess the last couple years by using conda 
> (miniconda install). It does a nice job of keeping entirely separate from the 
> system ( or any other) python, and it can manage non-python libs as well, so 
> you don't need Brew. And it has environments that are like virtualenv, but 
> with the C libs isolated as well.
> 
> Between the default channel, conda-forge, and pip for pure python packages, 
> there is a lot available.
> 
> If we were to add s PyObjC build to conda-forge, the full stack should "just 
> work".
> 
> And conda forge has s CI system set up to auto build for various python 
> versions
> 
> http://conda-forge.github.io/#add_recipe 
> 
Why would this be necessary?  Can't conda just install the wheels from PyPI?

-glyph___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

Re: [Pythonmac-SIG] PyObjC and macOS 10.12 (Sierra)

[Glyph Lefkowitz]

> On Dec 14, 2016, at 9:44 AM, Chris Barker  wrote:
> 
> conda also has a non-framework build of Python -- not sure if that would 
> cause any issues with the wheels.

I am not up on all the technical specifics, but this suggests to me that Conda 
would be generally unsuitable for use as a Mac native development environment, 
and hence not a Python you'd want to use pyobjc with.

If you don't have a framework build, you don't have an app bundle; if you don't 
have an app bundle then many frameworks will fail to initialize, or will start 
behaving in bizarre ways.  APIs like NSUserNotificationCenter will just 
silently do nothing, CoreLocation won't respond... it's always very confusing 
and hard to troubleshoot.

That said, I don't think the framework/non-framework-ness of the build is 
relevant to the ABI; the wheels should work fine just from a "will they link 
and run" perspective.  And many CoreFoundation APIs definitely work without an 
app bundle, so, not completely useless :-).

-glyph

___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

Re: [Pythonmac-SIG] py2app release delayed

[Glyph Lefkowitz]

> On Dec 18, 2016, at 12:39 PM, Ronald Oussoren  wrote:
> 
> Hi,
> 
> I had hoped to release a new version of py2app today, but didn’t due to 
> problems with macholib that resulted in broken application bundles. The good 
> news is that I found the cause of that breakage and reverted the patch that 
> caused it.
> 
> I did fix a number of bugs on the tracker last week (the number of open 
> issues decreased from 94 to 72), and implemented a useful new feature: 
> support for “@loader_path” in shared libraries, which is used in wheels 
> processed with delocate 
>  >
>  such as the wheels for Pillow.
> 
> Because this is turning into a fairly large release I’d appreciate if people 
> could test the code in the repository with their applications. To install 
> first install altgraph-0.13, then install modulegraph, macholib and py2app 
> from their bitbucket repositories 
> (https://bitbucket.org/ronaldoussoren/modulegraph 
> , 
> https://bitbucket.org/ronaldoussoren/macholib 
>  and 
> https://bitbucket.org/ronaldoussoren/py2app 
> ).

Can you translate these URLs into (pinned) URLs suitable for a 
requirements.txt?  I'm not used to doing this with hg or bitbucket.  If so I'd 
be happy to test with Mimic.

-g


___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

Re: [Pythonmac-SIG] apple-Python and TLS 1.0

[Glyph Lefkowitz]

> On Jan 10, 2017, at 9:21 AM, Kevin Ollivier  wrote:
> 
> Hi Jack,
> 
> No, I think you're spot on, this is a big problem. Actually, 2.7.9-2.7.12, 
> even the Python.org ones, are already somewhat broken because they use 
> Apple's ancient OpenSSL version. All the ciphers supported by that version of 
> OpenSSL are ones that are regarded as insecure now, so most modern servers, 
> including big ones like AWS, don't allow them anymore. Because of this you 
> can't even download a newer OpenSSL from the OpenSSL web site using Python. 
> :( 

Let's remember that the actual end-of-life here is 18 months out.  It would not 
be desirable to "fix" this problem by clinging to TLS 1.0 even longer; Fastly 
is shutting it off for a lot of very good reasons.

In between here and there are a variety of events, including a new OS release 
from apple.  When it enters beta we can loudly encourage Apple to include new 
and better crypto with Python - that probably does not mean a new 'ssl' module, 
but something more like the new 'tls' module PEP that Cory Benfield has been 
championing (in large part as a response to this).

You can see that discussion here: 
https://mail.python.org/pipermail/security-sig/2017-January/000126.html 
.

Finally, system Python has a lot of issues (unrelated to this, for example, it 
can't be 'dtrace'd, which eliminates a ton of super-useful debugging tools, 
including the OS X equivalent of 'strace'), and I've been gently discouraging 
users from using it for years.  (I say this as an ex- myself.)  Tell users to 
install Python (and PyPy!) from Homebrew, and they will generally have a much 
better time.

If we do nothing, this is going to be a huge disaster.  But the responsible 
people are highly motivated and are quite actively doing things.  If you want 
to make sure users have a good experience, helping them out would definitely be 
advised :).

> It surprised me to find that the Python community wasn't really aware of this 
> problem. For one project I worked on we actually re-coded all our download 
> code to use the Cocoa HTTPS classes via PyObjC, and this was a couple years 
> back. Don't know how many others out there have been fighting with it. 


Cryptography ships with its own OpenSSL, which means you can side-step a lot of 
these issues; install requests[security] and you get better HTTPS.  Or install 
from Homebrew, or python.org , or Anaconda.

-glyph___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

Re: [Pythonmac-SIG] apple-Python and TLS 1.0

[Glyph Lefkowitz]

> On Jan 12, 2017, at 6:10 AM, Jack Jansen  wrote:
> 
> Ok, so this is a real problem:-(
> 
> Again, I’m not deep enough into the SSL stuff to really understand this (and 
> specifically whether it needs a new openssl module, a new libssl, both, 
> something else, ….), but I’d like to think of ways to fix this before the 
> shit hits the fan for all poor mac Python users out there, if possible. And 
> that includes people who aren’t even aware they’re macPython users because 
> they use some app that uses Python under the hood…..
> 
> So a couple of questions:
> 
> 1. Is this fixable by Apple, by providing a security update to various OSX 
> versions that would include a newer python/libssl/whatever? 

Yes.  Apple should really be including a private OpenSSL with Python.  But they 
probably won't do that.

> 1a. Would this still fall under Apple’s idea of “security update”?

No, it would probably be a Major-OS sort of thing.

> 1b. Do we have any chance of making Apple interested in fixing this?

It's doubtful that they will update OpenSSL directly.  But they might include a 
new version of Python with a different approach to basic TLS (see the 'tls' 
module in my other response)

> 2. Is this fixable with an installer that would somehow override the openssl 
> module, so that installing this one thing would make the whole Apple-Python 
> installation work again?

Yes and no.  You could fix this by replacing the 'ssl' module with a wrapper 
that replicates it based on 'pyOpenSSL', which is a layer over 'cryptography', 
but this doesn't help users bootstrap via Pip; it would be easier to just tell 
them to get a different Python.
> 
> 3. Failing that, I assume its the end of the line for Apple-Python, and we’ll 
> have to steer end users towards installing a python.org version. Right?

Homebrew is by far my most preferred option (easier to get updates, easier to 
get PyPy, easier to make multiple versions play nicely, more tuned for 
developers, etc etc etc).  But python.org <http://python.org/> is a reasonable 
option as well.  And there are lots of other reasons not to tell users to use 
the system-provided Python.

> 3a. If that’s the case, is there something we could ask of the pip 
> developers, the PyPi maintainers, whoever else to help the poor end users? 
> I.e. get them to release a version that would not say “ssl v1 invalid foobar 
> get lost” but in stead “you appear to be using Apple Python which does not 
> support current security measures, please see www.example.com for more 
> information”.

There's no way to do this purely from PyPI's side, but with a little help from 
Apple it's doable.

Another option here is to build the mac Python installers differently so 
they're more user-friendly, and rather than a .pkg make a .app, so that users 
who have sufficiently little command-line expertise to be able to get something 
like Homebrew to behave properly would be able to get on-screen instructions 
and prompts that would help them get set up with a correct command linee.

-glyph

___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

Re: [Pythonmac-SIG] py2app 0.12 and modulegraph 0.14 released

[Glyph Lefkowitz]

> On Feb 22, 2017, at 2:39 AM, Ronald Oussoren  wrote:
> 
> Hi,
> 
> I’ve just pushed py2app 0.12 and modulegraph 0.14 to PyPI. Both primarily 
> contain bugfixes.
> 
> I had hoped to fix more issues in these releases, but that hasn’t worked out 
> and the fixed bugs are serious enough to not delay the release any more. On 
> to the next release…

Thanks for the release, Ronald!  Looking forward to some of those fixes, and 
good luck on the next batch :-).

-glyph
___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG