On 07/06/2022 00:28, Israel Brewster wrote:
I have some large (>100GB) datasets loaded into memory in a two-dimensional (X
and Y) NumPy array backed XArray dataset. At one point I want to filter the data
using a boolean array created by performing a boolean operation on the dataset
that is, I
Hi, I'm not an expert on this so this is an educated guess:
You are calling drop=True and I presume that you want to delete the rows
of your dataset that satisfy a condition.
That's a problem.
If the underlying original data is stored in a dense contiguous array,
deleting chunks of it will
On Mon, 6 Jun 2022 14:28:41 -0800, Israel Brewster
declaimed the following:
>I have some large (>100GB) datasets loaded into memory in a two-dimensional (X
>and Y) NumPy array backed
Unless you have some massive number cruncher machine, with TB RAM, you
are running with a lot of page
I have some large (>100GB) datasets loaded into memory in a two-dimensional (X
and Y) NumPy array backed XArray dataset. At one point I want to filter the
data using a boolean array created by performing a boolean operation on the
dataset that is, I want to filter the dataset for all points
