I posted about f2pypy my mini-project to make a new backend to f2py which generates a ctypes-based extension module for Python. The goal is to see if that is a viable path for PyPy support of the existing shared libraries already wrapped by f2py. I think it is.
Feel free to comment!
skip to main |
skip to sidebar
Blog Archive
About Me
- Andrew Dalke
- I am an American living in Göteborg, Sweden. I make my living writing software for computational chemistry and biology, usually in Python.
4 comments:
note that PyPy has also another, (faster) way to call C function: the _ffi module, which ctypes is built on top of.
On my machine:
viper tmp $ cat x.py
import ctypes
m = ctypes.cdll.LoadLibrary("libm.so")
cos = m.cos
cos.argtypes = [ctypes.c_double]
cos.restype = ctypes.c_double
viper tmp $ cat y.py
import _ffi
m = _ffi.CDLL('libm.so')
cos = m.getfunc('cos', [_ffi.types.double], _ffi.types.double)
viper tmp $ pypy-c -m timeit "from x import cos" "cos(0)"
10000000 loops, best of 3: 0.0393 usec per loop
viper tmp $ pypy-c -m timeit "from y import cos" "cos(0)"
100000000 loops, best of 3: 0.0148 usec per loop
viper tmp $ pypy-c -m timeit "from math import cos" "cos(0)"
100000000 loops, best of 3: 0.0076 usec per loop
still not as good as math.cos, but that's because _ffi calls always release the GIL, while math.cos doesn't.
Hi Andrew,
I think you're missing the point on SWIG v.s. Reflex. First of all, someone needs to come up with the manpower, and in our field, SWIG is non-existent (it isn't able to parse our C++ codes, for one). As such, I could not justify working on it, as I can with Reflex (and Cling, later). Second, SWIG does not allow the same performance improvements if it goes through the CPython ABI (cpyext). In our field, with lots of cross-calls, cpyext and SWIG itself would be the main bottleneck. With Reflex, we can peel away those layers for optimal performance.
Cheers,
Wim
Hi Wim,
I had no point on SWIG vs. Reflex so it's very easy to have missed the point. I had never heard of it outside of the single PyPy reference, which is what I wrote.
In my field (computational chemistry software), several tools are based on SWIG bindings to Python. These include the OpenEye libraries (C++), Open Babel library (C++) and my own aged PyDaylight (C). Both OpenEye and Open Babel use SWIG to also generate Java and C# bindings.
As you say, there isn't the people time/interest to do the conversion to some other system. Which is the main issue here. CPython has had decades to build up ecosystems which support it, with multiple tools in the niche. PyPy is just now getting to that point.
That PyPy isn't there *yet* is mostly because the lack of strong, specific funding for these tasks means that it must be done piecemeal over 5-10 years.
Hi Andrew,
where I saw you making a comparison between the two, is the description of Reflex as an "existing ABI" and the call for the creation of "some SWIG/ABI interface."
However, Reflex is not and does not provide an ABI for PyPy. Instead, the cppyy module generates bindings at runtime (this code is written in RPython) based on stored reflection information.
This method is more functional (once its finalized :) ) and gives better performance than a SWIG/ABI approach could ever do, since it does not have an extension layer.
Cheers,
Wim
Post a Comment