Re: [Numpy-discussion] Is there a defined way to "unpad" an array, and if not, should there be?
Prior art data point: scikit-image.util.crop : https://scikit-image.org/docs/dev/api/skimage.util.html#skimage.util.crop вт, 13 апр. 2021 г., 11:37 Eric Wieser : > Some other options here that avoid the need for a new function: > > * Add a `return_view` argument to `pad`, such that for `padded, orig = > np.pad(arr, ..., return_view=True)`, `orig == arr` and `orig.base is > padded`. This is useful if `padded` is modified in place, but less useful > otherwise. It has the advantage of not having to recompute the slices, as > pad already has them. > * Accept a `slice` object directly in `np.pad`; for `sl = np.s_[2:-20, > 4:-40]`, `padded = np.pad(array, sl)`, we have `padded[sl] == array`. > > The second idea seems promising to me, but perhaps there are corner cases > I haven't thought of that it wouldn't help with. > > Eric > > On Tue, 13 Apr 2021 at 09:26, Ralf Gommers wrote: > >> >> >> On Tue, Apr 13, 2021 at 3:37 AM Jeff Gostick wrote: >> >>> It is great to hear that this might be useful. I would LOVE to create a >>> PR on this idea and contribute back to numpy...but let's not get ahead of >>> ourselves :-) >>> >>> Regarding the name, I kinda like "unpad" since it relates directly to >>> "pad", analogous to "ravel" and "unravel" for instance. Or maybe "depad". >>> Although, it's possible to use this on any array, not just a previously >>> padded one, so maybe tying it too directly to "pad" is not right, in which >>> case "trim" and "crop" are both perfect. I must admit that I find it odd >>> that these functions are not in numpy already. I just searched the docs >>> and they show up as keyword args for a few functions but are otherwise >>> conspicuously absent. Also, funnily, there is a link to "padding arrays" >>> but it is basically empty: >>> https://numpy.org/doc/stable/reference/routines.padding.html. >>> >>> Alternatively, I don't hate the idea of passing negative pad widths into >>> "pad". I actually tried this at one point to see if there was a hidden >>> functionality there, to no avail. >>> >>> BTW, we just adding a custom "unpad" function to our PoreSpy package for >>> this purpose: >>> https://github.com/PMEAL/porespy/blob/dev/porespy/tools/_unpadfunc.py >>> >>> >>> >>> On Mon, Apr 12, 2021 at 9:15 PM Stephan Hoyer wrote: >>> On Mon, Apr 12, 2021 at 5:12 PM Jeff Gostick wrote: > I guess I should have clarified that I was inquiring about proposing a > 'feature request'. The github site suggested I open a discussion on this > list first. There are several ways to effectively unpad an array as has > been pointed out, but they all require more than a little bit of thought > and care, are dependent on array shape, and honestly error prone. It > would > be very valuable to me to have such a 'predefined' function, so I was > wondering if (a) I was unaware of some function that already does this and > (b) if I'm alone in thinking this would be useful. > Indeed, this is a fair question. Given that this is not entirely trivial to write correctly, I think it would be reasonable to add the inverse operation for pad() into NumPy. This is generally better than encouraging users to write their own thing. From a naming perspective, here are some possibilities: unpad trim crop I think "trim" would be pretty descriptive, probably slightly better than "unpad." >>> >> I'm not a fan of `trim`. We already have `clip` which sounds similar. >> >> `unpad` looks like the only one that's completely unambiguous. >> >> `crop` sounds like an image processing function, and what we don't want >> is something like Pillow's `crop(left, top, right, bottom)`. >> >> Cheers, >> Ralf >> >> >> ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion >>> ___ >>> NumPy-Discussion mailing list >>> NumPy-Discussion@python.org >>> https://mail.python.org/mailman/listinfo/numpy-discussion >>> >> ___ >> NumPy-Discussion mailing list >> NumPy-Discussion@python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Is there a defined way to "unpad" an array, and if not, should there be?
Some other options here that avoid the need for a new function: * Add a `return_view` argument to `pad`, such that for `padded, orig = np.pad(arr, ..., return_view=True)`, `orig == arr` and `orig.base is padded`. This is useful if `padded` is modified in place, but less useful otherwise. It has the advantage of not having to recompute the slices, as pad already has them. * Accept a `slice` object directly in `np.pad`; for `sl = np.s_[2:-20, 4:-40]`, `padded = np.pad(array, sl)`, we have `padded[sl] == array`. The second idea seems promising to me, but perhaps there are corner cases I haven't thought of that it wouldn't help with. Eric On Tue, 13 Apr 2021 at 09:26, Ralf Gommers wrote: > > > On Tue, Apr 13, 2021 at 3:37 AM Jeff Gostick wrote: > >> It is great to hear that this might be useful. I would LOVE to create a >> PR on this idea and contribute back to numpy...but let's not get ahead of >> ourselves :-) >> >> Regarding the name, I kinda like "unpad" since it relates directly to >> "pad", analogous to "ravel" and "unravel" for instance. Or maybe "depad". >> Although, it's possible to use this on any array, not just a previously >> padded one, so maybe tying it too directly to "pad" is not right, in which >> case "trim" and "crop" are both perfect. I must admit that I find it odd >> that these functions are not in numpy already. I just searched the docs >> and they show up as keyword args for a few functions but are otherwise >> conspicuously absent. Also, funnily, there is a link to "padding arrays" >> but it is basically empty: >> https://numpy.org/doc/stable/reference/routines.padding.html. >> >> Alternatively, I don't hate the idea of passing negative pad widths into >> "pad". I actually tried this at one point to see if there was a hidden >> functionality there, to no avail. >> >> BTW, we just adding a custom "unpad" function to our PoreSpy package for >> this purpose: >> https://github.com/PMEAL/porespy/blob/dev/porespy/tools/_unpadfunc.py >> >> >> >> On Mon, Apr 12, 2021 at 9:15 PM Stephan Hoyer wrote: >> >>> On Mon, Apr 12, 2021 at 5:12 PM Jeff Gostick wrote: >>> I guess I should have clarified that I was inquiring about proposing a 'feature request'. The github site suggested I open a discussion on this list first. There are several ways to effectively unpad an array as has been pointed out, but they all require more than a little bit of thought and care, are dependent on array shape, and honestly error prone. It would be very valuable to me to have such a 'predefined' function, so I was wondering if (a) I was unaware of some function that already does this and (b) if I'm alone in thinking this would be useful. >>> >>> Indeed, this is a fair question. >>> >>> Given that this is not entirely trivial to write correctly, I think it >>> would be reasonable to add the inverse operation for pad() into NumPy. This >>> is generally better than encouraging users to write their own thing. >>> >>> From a naming perspective, here are some possibilities: >>> unpad >>> trim >>> crop >>> >>> I think "trim" would be pretty descriptive, probably slightly better >>> than "unpad." >>> >> > I'm not a fan of `trim`. We already have `clip` which sounds similar. > > `unpad` looks like the only one that's completely unambiguous. > > `crop` sounds like an image processing function, and what we don't want is > something like Pillow's `crop(left, top, right, bottom)`. > > Cheers, > Ralf > > > ___ >>> NumPy-Discussion mailing list >>> NumPy-Discussion@python.org >>> https://mail.python.org/mailman/listinfo/numpy-discussion >>> >> ___ >> NumPy-Discussion mailing list >> NumPy-Discussion@python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Is there a defined way to "unpad" an array, and if not, should there be?
It is great to hear that this might be useful. I would LOVE to create a PR on this idea and contribute back to numpy...but let's not get ahead of ourselves :-) Regarding the name, I kinda like "unpad" since it relates directly to "pad", analogous to "ravel" and "unravel" for instance. Or maybe "depad". Although, it's possible to use this on any array, not just a previously padded one, so maybe tying it too directly to "pad" is not right, in which case "trim" and "crop" are both perfect. I must admit that I find it odd that these functions are not in numpy already. I just searched the docs and they show up as keyword args for a few functions but are otherwise conspicuously absent. Also, funnily, there is a link to "padding arrays" but it is basically empty: https://numpy.org/doc/stable/reference/routines.padding.html. Alternatively, I don't hate the idea of passing negative pad widths into "pad". I actually tried this at one point to see if there was a hidden functionality there, to no avail. BTW, we just adding a custom "unpad" function to our PoreSpy package for this purpose: https://github.com/PMEAL/porespy/blob/dev/porespy/tools/_unpadfunc.py On Mon, Apr 12, 2021 at 9:15 PM Stephan Hoyer wrote: > On Mon, Apr 12, 2021 at 5:12 PM Jeff Gostick wrote: > >> I guess I should have clarified that I was inquiring about proposing a >> 'feature request'. The github site suggested I open a discussion on this >> list first. There are several ways to effectively unpad an array as has >> been pointed out, but they all require more than a little bit of thought >> and care, are dependent on array shape, and honestly error prone. It would >> be very valuable to me to have such a 'predefined' function, so I was >> wondering if (a) I was unaware of some function that already does this and >> (b) if I'm alone in thinking this would be useful. >> > > Indeed, this is a fair question. > > Given that this is not entirely trivial to write correctly, I think it > would be reasonable to add the inverse operation for pad() into NumPy. This > is generally better than encouraging users to write their own thing. > > From a naming perspective, here are some possibilities: > unpad > trim > crop > > I think "trim" would be pretty descriptive, probably slightly better than > "unpad." > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Is there a defined way to "unpad" an array, and if not, should there be?
On Mon, Apr 12, 2021 at 5:12 PM Jeff Gostick wrote: > I guess I should have clarified that I was inquiring about proposing a > 'feature request'. The github site suggested I open a discussion on this > list first. There are several ways to effectively unpad an array as has > been pointed out, but they all require more than a little bit of thought > and care, are dependent on array shape, and honestly error prone. It would > be very valuable to me to have such a 'predefined' function, so I was > wondering if (a) I was unaware of some function that already does this and > (b) if I'm alone in thinking this would be useful. > Indeed, this is a fair question. Given that this is not entirely trivial to write correctly, I think it would be reasonable to add the inverse operation for pad() into NumPy. This is generally better than encouraging users to write their own thing. >From a naming perspective, here are some possibilities: unpad trim crop I think "trim" would be pretty descriptive, probably slightly better than "unpad." ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Is there a defined way to "unpad" an array, and if not, should there be?
I guess I should have clarified that I was inquiring about proposing a 'feature request'. The github site suggested I open a discussion on this list first. There are several ways to effectively unpad an array as has been pointed out, but they all require more than a little bit of thought and care, are dependent on array shape, and honestly error prone. It would be very valuable to me to have such a 'predefined' function, so I was wondering if (a) I was unaware of some function that already does this and (b) if I'm alone in thinking this would be useful. On Mon, Apr 12, 2021 at 7:42 PM Aaron Meurer wrote: > On Mon, Apr 12, 2021 at 2:29 PM Stephan Hoyer wrote: > > > > The easy way to unpad an array is by indexing with slices, e.g., > x[20:-4] to undo a padding of [(20, 4)]. Just be careful about unpadding > "zero" elements on the right hand side, because Python interprets an ending > slice of zero differently -- you need to write something like x[20:] to > undo padding by [(20, 0)]. > > You can use x[20:x.shape[0] - 4] to avoid this inconsistency. Or > construct the slice based on the original unpadded shape > (x[20:20+orig_x.shape[0]]). > > Aaron Meurer > > > > > > > On Mon, Apr 12, 2021 at 1:15 PM Jeff Gostick wrote: > >> > >> I often find myself padding an array to do some processing on it (i.e. > to avoid edge artifacts), then I need to remove the padding. I wish there > was either a built in "unpad" function that accepted the same arguments as > "pad", or that "pad" accepted negative numbers (e.g [-20, -4] would undo a > padding of [20, 4]). This seems like a pretty obvious feature to me so > maybe I've just missed something, but I have looked through all the open > and closed issues on github and don't see anything related to this. > >> > >> > >> Jeff G > >> > >> ___ > >> NumPy-Discussion mailing list > >> NumPy-Discussion@python.org > >> https://mail.python.org/mailman/listinfo/numpy-discussion > > > > ___ > > NumPy-Discussion mailing list > > NumPy-Discussion@python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Is there a defined way to "unpad" an array, and if not, should there be?
On Mon, Apr 12, 2021 at 2:29 PM Stephan Hoyer wrote: > > The easy way to unpad an array is by indexing with slices, e.g., x[20:-4] to > undo a padding of [(20, 4)]. Just be careful about unpadding "zero" elements > on the right hand side, because Python interprets an ending slice of zero > differently -- you need to write something like x[20:] to undo padding by > [(20, 0)]. You can use x[20:x.shape[0] - 4] to avoid this inconsistency. Or construct the slice based on the original unpadded shape (x[20:20+orig_x.shape[0]]). Aaron Meurer > > > On Mon, Apr 12, 2021 at 1:15 PM Jeff Gostick wrote: >> >> I often find myself padding an array to do some processing on it (i.e. to >> avoid edge artifacts), then I need to remove the padding. I wish there was >> either a built in "unpad" function that accepted the same arguments as >> "pad", or that "pad" accepted negative numbers (e.g [-20, -4] would undo a >> padding of [20, 4]). This seems like a pretty obvious feature to me so >> maybe I've just missed something, but I have looked through all the open and >> closed issues on github and don't see anything related to this. >> >> >> Jeff G >> >> ___ >> NumPy-Discussion mailing list >> NumPy-Discussion@python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Is there a defined way to "unpad" an array, and if not, should there be?
It's definitely just "slicing", but it's a bit inconvenient. I'm thinking more like: arr = np.random.rand(10, 10, 10) W = [[3, 2], [4, 6]] # or W = 4, or W = [4, 5] arr_padded = np.pad(arr, pad_width=W) < Do some stuff to arr_padded > arr = np.unpad(arr_padded, pad_width=W) # Using W just works, no matter how odd the various pad widths were On Mon, Apr 12, 2021 at 4:29 PM Stephan Hoyer wrote: > The easy way to unpad an array is by indexing with slices, e.g., x[20:-4] > to undo a padding of [(20, 4)]. Just be careful about unpadding "zero" > elements on the right hand side, because Python interprets an ending slice > of zero differently -- you need to write something like x[20:] to undo > padding by [(20, 0)]. > > > On Mon, Apr 12, 2021 at 1:15 PM Jeff Gostick wrote: > >> I often find myself padding an array to do some processing on it (i.e. to >> avoid edge artifacts), then I need to remove the padding. I wish there >> was either a built in "unpad" function that accepted the same arguments as >> "pad", or that "pad" accepted negative numbers (e.g [-20, -4] would undo a >> padding of [20, 4]). This seems like a pretty obvious feature to me so >> maybe I've just missed something, but I have looked through all the open >> and closed issues on github and don't see anything related to this. >> >> >> Jeff G >> >> ___ >> NumPy-Discussion mailing list >> NumPy-Discussion@python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Is there a defined way to "unpad" an array, and if not, should there be?
The easy way to unpad an array is by indexing with slices, e.g., x[20:-4] to undo a padding of [(20, 4)]. Just be careful about unpadding "zero" elements on the right hand side, because Python interprets an ending slice of zero differently -- you need to write something like x[20:] to undo padding by [(20, 0)]. On Mon, Apr 12, 2021 at 1:15 PM Jeff Gostick wrote: > I often find myself padding an array to do some processing on it (i.e. to > avoid edge artifacts), then I need to remove the padding. I wish there > was either a built in "unpad" function that accepted the same arguments as > "pad", or that "pad" accepted negative numbers (e.g [-20, -4] would undo a > padding of [20, 4]). This seems like a pretty obvious feature to me so > maybe I've just missed something, but I have looked through all the open > and closed issues on github and don't see anything related to this. > > > Jeff G > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Is there a defined way to "unpad" an array, and if not, should there be?
Isn't that just slicing? Or perhaps you are looking for a way to simplify the calculation of the slice arguments from the original pad arguments? On Mon, Apr 12, 2021 at 4:15 PM Jeff Gostick wrote: > I often find myself padding an array to do some processing on it (i.e. to > avoid edge artifacts), then I need to remove the padding. I wish there > was either a built in "unpad" function that accepted the same arguments as > "pad", or that "pad" accepted negative numbers (e.g [-20, -4] would undo a > padding of [20, 4]). This seems like a pretty obvious feature to me so > maybe I've just missed something, but I have looked through all the open > and closed issues on github and don't see anything related to this. > > > Jeff G > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
