Announcing Python-Blosc2 3.0.0 (final release) ==============================================
The Blosc development team is pleased to announce the final release of Python-Blosc2 3.0.0. Now, we will be producing conda(-forge) packages, as well as providing wheels for the most common platforms, as usual. In this major release, we are adding a new compute engine tuned for compressed data that can be stored either in-memory, on-disk or on the network (via the Caterva2 library (https://github.com/ironArray/Caterva2/). You can think of Python-Blosc2 3.0 as an extension of NumPy/numexpr that: - Can deal with ndarrays (optionally) compressed using first-class codecs & filters. - Performs many kind of math expressions, including reductions, indexing and more. - Supports broadcasting operations. - Supports NumPy ufunc mechanism, allowing to mix and match NumPy and Blosc2 computations. - Integrates with Numba and Cython via UDFs (User Defined Functions). - Adheres to modern NumPy casting rules way better than numexpr. - Computes expressions lazily, only when needed, and can be stored for later use. Install it with:: pip install blosc2==3.0.0 # if you prefer wheels conda install -c conda-forge python-blosc2 mkl # if you prefer conda and MKL For more info, you can have a look at the release notes in: https://github.com/Blosc/python-blosc2/releases Code example:: from time import time import blosc2 import numpy as np # Create some data operands N = 20_000 a = blosc2.linspace(0, 1, N * N, dtype="float32", shape=(N, N)) b = blosc2.linspace(1, 2, N * N, shape=(N, N)) c = blosc2.linspace(-10, 10, N) # broadcasting is supported # Expression t0 = time() expr = ((a**3 + blosc2.sin(c * 2)) < b) & (c > 0) print(f"Time to create expression: {time()-t0:.5f}") # Evaluate while reducing (yep, reductions are in) along axis 1 t0 = time() out = blosc2.sum(expr, axis=1) t1 = time() - t0 print(f"Time to compute with Blosc2: {t1:.5f}") # Evaluate using NumPy na, nb, nc = a[:], b[:], c[:] t0 = time() nout = np.sum(((na**3 + np.sin(nc * 2)) < nb) & (nc > 0), axis=1) t2 = time() - t0 print(f"Time to compute with NumPy: {t2:.5f}") print(f"Speedup: {t2/t1:.2f}x") assert np.all(out == nout) print("All results are equal!") This will output something like (using an Intel i9-13900X CPU here):: Time to create expression: 0.00033 Time to compute with Blosc2: 0.46387 Time to compute with NumPy: 2.57469 Speedup: 5.55x All results are equal! See a more in-depth example, explaining why Python-Blosc2 is so fast, at: https://www.blosc.org/python-blosc2/getting_started/overview.html#operating-with-ndarrays Docs ---- Read some of our tutorials on how to perform advanced computations at: https://www.blosc.org/python-blosc2/getting_started/tutorials See also materials for our recent PyData Global 2024 tutorial at: https://github.com/Blosc/Python-Blosc2-3.0-tutorial And the full documentation at: https://www.blosc.org/python-blosc2 Enjoy! - Blosc Development Team Compress Better, Compute Bigger _______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
