I’ve uploaded PyObjC 8.1 to PyPI. This is a minor bugfix release.

A new macOS release means a new major release of PyObjC. I’m happy to announce that PyObjC 8 is available on PyPI with full support for macOS 12, including APIs introduced in that release.

I’ve also been working on performance and code quality of the core bridge. There are significant speedups of the core bridge in PyObjC 8.0, and I have more ideas to reduce the performance difference between regular Python methods and calling Objective-C APIs.

The release is a bit later than I would have liked, in part because I’ve been worrying about the covid-19 situation in Holland. Because of that I’ve spend some more time away from the computer.

A full change log is included below.

Backward incompatible changes

• In PyObjC 7 and earlier it was possible to leave out “output” arguments when calling a method with a Python implementation:

  class MyClass(NSObject):
      @objc.typedSelector(b"@@:o^@")
      def methodWithOutput_(self, a):
          return 1, 2
  
  
  o = MyClass.alloc().init()
  print(o.methodWithOutput_())
  

  This no longer works, it is always necessary to pass in all arguments, which was already true for methods implemented in Objective-C. That is:

  print(o.methodWithOutput_(None))
  

  This change both simplifies the PyObjC code base and was required to cleanly implement vectorcall support (see the section on performance below).

• Removed bindings for InterfaceBuilderKit. This was a way to integrate with InterfaceBuilder in old versions of Xcode, but support for that was dropped before the release of Mac OS X 10.8.

• Bindings for the Objective-C runtime API related to “associated objects” is now enabled unconditionally. This will cause problems when running or building on macOS 10.6 or earlier.

• It is no longer possible to deploy to macOS 10.7 or earlier when you attempt to create a formal protocol. Protocol creation already failed on those platform due to lack of the required runtime API, and that will now result in a crash because PyObjC no longer checks for availability of that runtime API.

• GH-371: Remove manual bindings for a number of old CoreGraphics APIs

  The following functions are no longer available:

  • CGDataProviderCreate

  • CGDataProviderCreateDirectAccess

  These functions were removed as a public API in macOS 10.8, but were still available through PyObjC through old backward compatibility code. That code has now been removed.

• For compatibility with Python’s socket APIs functions that return a “struct sockaddr” (either by reference or as a function result) will now encode the IP address as a string and not a bytes object.

• The (undocumented) API in pyobjc-api.h (used in some framework bindings to integratie with pyobjc-core) has changed in an incompatible way, in particular the API for “caller” functions now mostly mirrors the vectorcall convention.

• Adding a method with a double underscore name will now raise an exception at class definition time instead of silently not creating the Objective-C method.

   class AClass (NSObject):
       ...
  
   def __foo_bar__(self, a, b, c):
       pass
  
   MethodNamesClass.__foo_bar__ = __foo_bar__
  

  Before PyObjC 8 this would add a __foo_bar__ selector to the Python representation of the class without adding a selector to the Objective-C class.

  Use objc.python_method to mark this as a python-only function.

Upcoming incompatible changes

• The module PyObjCTools.Signals is deprecated and will be removed in PyObjC 9.

• objc.initFrameworkWrapper and objc.parseBridgeSupport are deprecated and will be removed in PyObjC 9.

These functions implement support for “.bridgesupport” XML files, something that PyObjC hasn’t used itself in a number of releases (in part because system versions of those files are at best incomplete).

Performance

Most performance changes use features introduced in Python 3.9, performance in older Python versions is unchanged except for the effects of general cleanup.

• Implement the “vectorcall” protocol for objc.function, objc.WeakRef, objc.selector, objc.IMP, objc.python_method.

  This reduces the interpreter overhead for calling instances of these objects.

• Implement Py_TPFLAGS_METHOD_DESCRIPTOR for objc.selector, objc.python_method.

• Use vectorcall in the method stub that forwards Objective-C calls to Python.

• Convert internal calls into Python to the vectorcall protocol (pyobjc-core)

• Add more optimized vectorcall implementation to :class:objc.function, objc.IMP and objc.selector for simpler callables.

  “Simpler” methods are those with a small number of plain arguments, although the definition of what’s simple will evolve over time.

Generic Implementation Quality

• GH-391: Fix some spelling errors found by the codespell tool.

  The codespell tool is also run as part of pre-commit hooks.

• GH-296: use clang-format for Objective-C code

  The Objective-C code for the various extensions has been reformatted using clang-format, and this enforced by a pre-commit hook.

• GH-374: Use pyupgrade to modernize the code base

  This is enforced by a pre-commit hook.

• GH-388: Added “nullability” attributes to Objectice-C sources for pyobjc-core.

  This gives the compiler and clang static analyzer more information that can be used to pinpoint possible bugs in the implementation. As a side effect of this a number of internal checks were strengthened, although most of them were for error conditions that should never happen.

  That said, this change also found a number of places where Python reference counts weren’t updated properly, which may have led to refcount overflows in long running programs.

• Add more error checking to pyobjc-core to catch (very) unlikely error conditions.

This is a side effect of the previous item.

New features

• Updated framework bindings for macOS 12

• New framework bindings for the following frameworks:

  • AudioVideoBridging (introduced in macOS 10.8)

  • DataDetection (introduced in macOS 12.0)

  • IntentsUI (introduced in macOS 12.0)

  • LocalAuthenticationEmbeddedUI (introduced in macOS 12.0)

  • MailKit (introduced in macOS 12.0)

  • MetricKit (introduced in macOS 12.0)

  • ShazamKit (introduced in macOS 12.0)

• GH-318: Implement support for __class_getitem__ for Objective-C classes

  The result of this is that effectively all Objective-C classes can be used as generic classes, without runtime type checking. This is meant to be used with optional type checking (for example MyPy)

  Usage:

    def create_integers(count: int) -> NSArray[int]:
          return NSArray[int].arrayWithArray_([i for i in range(count)])
  

  This requires typing stubs for framework bindings to be really useful, and those do not yet exist.

Other changes and bugfixes

• GH-390: pyobjc-core is no longer linked with the Carbon framework.

  Due to implicit dependencies this also required a change to the Metal bindings: those now import AppKit instead of Foundation.

• PyObjC only ships “Universal 2” wheels for Python 3.8 and later. Those work with single-architecture builds of Python as well.

• PyObjC 8 only ships with source archives and “univeral2” binary wheels (Python 3.? and later). There are no longer “x86_64” binary wheels.

• The AVFoundation bindings (in pyobjc-framework-AVFoundation) now have an install dependency on the CoreAudio bindings (pyobjc-framework-CoreAudio).

  This is needed for a new API introduced in macOS 12.

• GH-371: Link extensions in the Quartz bindings to the Quartz frameworks

  A number of C extensions in the Quartz bindings package were not linked to a framework. Those now link to the Quartz framework.

• GH-378: Fix raising ImportError when doing from ApplicationServices import *

  The root cause for this were private classes in system frameworks that contain a dot in their name (for example Swift.DispatchQueueShim. Those names are both private and invalid attribute names.

• Creating protocols that contain methods that have a method signature containing PyObjC custom type encodings now works (those encodings are translated to the corresponding Objective-C encoding.

• Fix bindings for SKIndexCopyDocumentRefsForDocumentIDs, that binding didn’t work due to a typo in the metadata.

• GH-365: The PyObjCTools namespace package no longer has an __init__.py file in the source tree (that is, the tree switches to implicit namespace packages instead of the older setuptools style for namespace packages).

  This primarily affects testing with recent versions of pip/setuptools (which seem to no longer install the __init__.py file for namespace packages).

• development-support/run-testsuite now uses venv instead of virtualenv. This removes a development dependency.

• PR-367: Tweak the code that calculates PyObjC_BUILD_RELEASE in the various setup.py files to deal with versions with more than two labels (can happen when building using Xcode 13 beta)

  PR by Eric Lin (Tzu Hsiang Lin), github user eric100lin.

• PyObjCTest.TestSupport now never uses “10.16” as the OS release but always uses the actual platform version, even when Python was compiled using an old SDK.

• Adjusted PyObjC testcases to check for 11.0 instead of 10.16 now that testsupport uses the real platform version.

• GH-385: Fix race condition the lazy importer

  When two threads simultaneously try to get an attribute from a framework binding one of them might fail with an attribute error because information for resolving the name was removed before actually resolving the name.

• Fix various issues with invalid indices in :class:objc.varlist

• Fix support for AF_UNIX in the support code for struct sockaddr.

• The implementation for opaque pointer types (such as the proxy for ‘NSZone*‘) has switched to PyType_FromSpec.

• The objc.FSRef.from_path and objc.FSRef.as_pathname, methods now use the filesystem encoding instead of the default encoding. C string. This shouldn’t affect any code, both encoding default to UTF-8 on macOS.

• Inheriting directly from objc.objc_object now raises TypeError instead of objc.InternalError. User code should always inherit from a Cocoa class.

• GH-354: Add an option to install all framework bindings, including those not relevant for the current platform. To use this:

  $ pip install 'pyobjc[allbindings]'
  

I’ve just uploaded py2app 0.26.1 to PyPI. This contains a small bugfix that only affects applications using Tkinter when using the “Intel” installer on python(dot)org.

I’ve spend some more time on py2app over the last week or so, in between relaxing while being away from work. That results in a larger release for py2app than I’ve done in a long while.

The most important changes are support for Python 3.10 and (finally!) bundling package metadata (“dist-info” directories). The latter should fix issues with code that uses pkg_resources and looks for distribution information such as entry points.

As always: Py2app is supposed to be a “DWIM” tool, please file issues when you need to adjust the bundled application, either through py2app options or (especially) manually.

And a public service announcement: Don’t use the “argv_emulator” option with GUI applications. That option tends to cause options with GUI eventloops, and most GUI libraries have better options to handle “file-open” events.

I expect to work less on py2app in the coming weeks, the current plan is to continue working on PyObjC during my last week of from work. The current repository is up-to-date w.r.t. support for the upcoming macOS Monterey, and I hope to land land some interesting other improvements during the week…

The full changelog:

• Stub executables were recompiled on macOS 11

  This means support for light mode/dark mode should now work out of the box.

  The old stub executables are still used when detecting that Tkinter is used with an old build of Tk.

• Issue 1: Include “.egg-info” and “.dist-info” information in the bundled application

  This fixes any python package that uses pkg_resources to look for specific distributions.

• py2app.filters.not_stdlib_filter now knows about Python’s “venv”

• Issue 368: Add recipe detect_dunder_file

  This recipe will ensure that a Python package is stored outside of site-packages.zip when a module in that package uses the __file__ variable.

  This variable is most commonly used to load resources stored in the package (instead of the newer importlib.resources and pkg_resources libraries).

• Issue 339: Add recipe for pydantic

  The recipe is needed because pydantic uses Cython to compile all sources (including the package __init__) and therefore hides imports from the dependency analyzer.

• Issue 338: Add “imageio_ffmpeg” to autopackages

• PR367: Add recipes for pandas, pylsp, and zmq

• PR367: Add docutils and pylint to autopackages

  PR by Ryan Clary (mrclary on GitHub)

• Issue 344: Invocation of codesign on the whole bundle sometimes fails

  Py2app will now try this a number of times before giving up. This is at best a workaround for and doesn’t completely fix the problem.

• Issue 370: py2app now works with Python 3.10

  Python 3.10 no longer exports a (private) symbol used by the py2app stub executable. Switched to a public API to accomplish the same task where available.

• Issue 110: Add recipe for SQLAlchemy

  The recipe includes all dialects and connectors, including implicit dependencies, because SQLAlchemy uses __import__ to load dependencies.

• Issue 328: Add recipe for gcloud

• Issue 195: Add USER_BASE, getuserbase() and getusersitepackages() to py2app’s version of site.py.

• Issue 184: Add recipe for ‘ssl’

  This recipe is only used for Python 3.4 or later and ensures that the CA bundle used by Python’s ssl module is included in the app bundle and OpenSSL is configured to look for that bundle in the application bundle.

• Issue 371: change default error message on launch problems

  The default error message shown when the application cannot be launched is now slightly more useful and refers the py2app debug page.

• Issue 345, 169: Adjust qt5 and qt6 recipes for non-PyPI installations

  The qt5 and qt6 recipes now should work when the Qt installation prefix is outside of the PyQt package, for example when PyQt was installed through homebrew.

  I’ve tested this for PyQt5 and made the same change to the PyQt6 recipe, although I haven’t tested that change.

I’ve done some work on py2app over the weekend after mostly neglecting it for the rest of the year.

I’ve uploaded version 0.25 to PyPI with the results of that work.

The full changelog for this release:

• #358: Add recipe for multiprocessing

• PR363: Add recipe for platformdirs

PR by Ryan Clary (mrclary on GitHub)

• PR353: Add recipe for sphinx

PR by Ryan Clary (mrclary on GitHub)

• PR352: Fix for using ipython

PR by Ryan Clary (mrclary on GitHub)

• PR351: Tweak the matplotlib recipe

PR by Ryan Clary (mrclary on GitHub)

• PR348: Fix for checking for dead symlinks links in py2app

PR by Oliver Cordes (ocordes on GitHub)

• #354: Fix buggy “autopackages” and “automissing” recipes

• #350: Add sentencepiece to the autopackages list

• #359: Add recipe for PyQt6

• #349: Add recipe for OpenCV (opencv-python, import cv2)

• PR365: Add RTree recipe

PR by Ryan Clary (mrclary on GitHub)

Apple: Let’s have an API for programmaticly creating an ObjC protocol Also Apple: Let’s use more information that cannot be added programmaticly (FB9543982)

github.com/ronaldous…

So close… Monterey runs in a VM, but for some reason the network adaptor doesn’t work.

With WWDC and beta of a new major release of macOS approaching fast it is high time to push out a new release for PyObjC. PyObjC 7.3 is primarily a bugfix release, with one minor feature.

I expect this to be the last release for PyObjC 7.x and will start working on PyObjC 8 soon.

• issue 356: Explicitly error out when building for unsupported architectures

  “python setup.py build” will now fail with a clear error when trying to build PyObjC for a CPU architecture that is no longer supported (such as 32-bit Intel)

• issue 319: Use memset instead of bzero in C code to clear memory

  Based on a PR by GitHub user stbdang.

• issue 348: Fix platform version guard for using protocols in MetalPerformanceShaders bindings

• issue 344: Fix test for CFMessagePortCreateLocal

  The tests didn’t actually test calling the callback function for CFMessagePortCreateLocal.

• issue 349: Change calls to htonl in pyobjc-core to avoid compiler warning

  The original code had a 32-bit assumption (using ‘long’ to represent a 32-bit value), and that causes problems for some users build from source.

• issue 315: Fix binding for SecAddSharedWebCredential (Security framework)

  Trying to use this function will no longer crash Python.

• issue 357: Calling Metal.MTLCopyAllDevices() no longer crashes

  The reference count of the result of this function was handled incorrect, causing access to an already deallocated value when the Python reference was garbage collected.

• issue 260: Add manual bindings for AXValueCreate and AXValueGetValue in ApplicationServices

  Calling these crashed in previous versions.

• issue 320, 324: Fix the type encoding for a number of CoreFoundation types in the Security bindings

• issue 336: Add core support for “final” classes

  It is now possible to mark Objective-C classes as final, that is to disable subclassing for such classes.

  This is primarily meant to be used in framework bindings for matching Objective-C semantics.

The “final” feature adds two new APIs:

1. A keyword argument “final” when defining a new class

       class MyClass (NSObject, final=True):
           pass
   

2. An read-write attribute __objc_final__ on all subclasses of NSObject.

Note that this is a separate concept from typing.final.

Some exercise before the second day of PyCon US. #PyConUS2021Active

I’ve released PyObjC 7.2 with support for the new APIs introduced in macOS 11.3.