As I said, you can sum the values for each pixel so you don't store all the
differences, that gets rid of the memory problem, but of course it will
still be slow if it's not parallelized:
vals = np.array([np.sum(np.abs(y - array.flat)) for y in array.flat])
Note that I didn't check thoroughly if
On Fri, 8 Apr 2022 at 16:33, Anna Petrášová wrote:
>
> Hi Luca,
>
Hi Anna,
> I would say the biggest problem is the memory, I tried to run it and it
> consumes way too much memory. Maybe you could process the differences from
> each pixel (compute the sum) as they are computed, not collect it
On Fri, 8 Apr 2022 at 11:17, Stefan Blumentrath
wrote:
>
> Ciao Luca,
>
Ciao Stefan
> Yes, you could also consider looping over e.g. rows (maybe in combination
> with "np.apply_along_axis") so you could put results easier back together to
> a map if needed at a later stage.
>
> In addition, si
---
> From: grass-dev On Behalf Of Luca
> Delucchi
> Sent: fredag 8. april 2022 10:46
> To: Moritz Lennert
> Cc: GRASS-dev
> Subject: Re: [GRASS-dev] multiprocessing problem
>
> On Fri, 8 Apr 2022 at 09:14, Moritz Lennert
> wrote:
> >
> > Hi Luca,
> >
: fredag 8. april 2022 10:46
To: Moritz Lennert
Cc: GRASS-dev
Subject: Re: [GRASS-dev] multiprocessing problem
On Fri, 8 Apr 2022 at 09:14, Moritz Lennert
wrote:
>
> Hi Luca,
>
Hi Moritz,
> Just two brainstorming ideas:
>
> - From a rapid glance at the code it seems to m
On Fri, 8 Apr 2022 at 09:14, Moritz Lennert
wrote:
>
> Hi Luca,
>
Hi Moritz,
> Just two brainstorming ideas:
>
> - From a rapid glance at the code it seems to me that you create a separate
> worker for each row in the raster. Correct ? AFAIR, spawning workers does
> create quite a bit of overh
Hi Luca,
Just two brainstorming ideas:
- From a rapid glance at the code it seems to me that you create a separate
worker for each row in the raster. Correct ? AFAIR, spawning workers does
create quite a bit of overhead. Depending on the row to column ratio of your
raster, maybe you would be b
Hi devs,
I wrote an addons to calculate Rao's Q diversity index [0], I would
like to speed it up using multiprocessing but with multiprocessing it
took 2/3 times longer than a single process. Could someone look at the
code and explain to me what I'm doing wrong?
A colleague of mine suggested using
