You need to reacquire the gil, then you can get the lock and rerelease the
gil.
I think this works (on phone, so untested)
with gil:
with nogil, lock:
...
Kevin
On Mon, Dec 14, 2020, 13:37 Evgeni Burovski <evgeny.burovs...@gmail.com>
wrote:
> Hi,
>
> What would be the correct way of locking the bit generator of
> np.random.Generator in cython's nogil context?
> (This is threading 101, surely, so please forgive my ignorance).
>
> The docs for extending np.random.Generator in cython
> (https://numpy.org/doc/stable/reference/random/extending.html#cython)
> recommend the following idiom for generating uniform variates, where
> the GIL is released and a Generator-specific lock is held:
>
> x = PCG64()
> rng = <bitgen_t *> PyCapsule_GetPointer(x.capsule, capsule_name)
> with nogil, x.lock:
> rng.next_double(rng.state)
>
> What is the correct way of locking it when already in the nogil
> section (so that x.lock is not accessible)?
>
> The use case is a long-running MC process which generates random
> variates in a tight loop (so the loop is all nogil). In case it
> matters, I probably won't be using python threads, but may use
> multiprocessing.
>
> Basically,
>
> cdef double uniform(self) nogil:
> if self.idx >= self.buf.shape[0]:
> self._fill()
> cdef double value = self.buf[self.idx]
> self.idx += 1
> return value
>
> cdef void _fill(self) nogil:
> self.idx = 0
> # HERE: Lock ?
> for i in range(self.buf.shape[0]):
> self.buf[i] = self.rng.next_double(self.rng.state)
>
>
> Thanks,
> Evgeni
>
>
> P.S. The full cdef class, for completeness:
>
> cdef class RndmWrapper():
> cdef:
> double[::1] buf
> Py_ssize_t idx
> bitgen_t *rng
> object py_gen # keep the garbage collector away
>
> def __init__(self, seed=(1234, 0), buf_size=4096, bitgen_kind=None):
> if bitgen_kind is None:
> bitgen_kind = PCG64
>
> # cf Numpy-discussion list, K.~Sheppard, R.~Kern, June 29,
> 2020 and below
> #
> https://mail.python.org/pipermail/numpy-discussion/2020-June/080794.html
> entropy, num = seed
> seed_seq = SeedSequence(entropy, spawn_key=(num,))
> py_gen = bitgen_kind(seed_seq)
>
> # store the python object to avoid it being garbage collected
> self.py_gen = py_gen
>
> capsule = py_gen.capsule
> self.rng = <bitgen_t *>PyCapsule_GetPointer(capsule, capsule_name)
> if not PyCapsule_IsValid(capsule, capsule_name):
> raise ValueError("Invalid pointer to anon_func_state")
>
> self.buf = np.empty(buf_size, dtype='float64')
> self._fill()
>
> @cython.boundscheck(False)
> @cython.wraparound(False)
> cdef void _fill(self) nogil:
> self.idx = 0
> for i in range(self.buf.shape[0]):
> self.buf[i] = self.rng.next_double(self.rng.state)
>
> @cython.boundscheck(False)
> @cython.wraparound(False)
> cdef double uniform(self) nogil:
> if self.idx >= self.buf.shape[0]:
> self._fill()
> cdef double value = self.buf[self.idx]
> self.idx += 1
> return value
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion
