Hi Roland-
Not much time but a few ideas for helping to track problems
down and to get help:
(1) Recent versions of perl have fixed the debugger problem
if an upgrade there is possible for you.
(2) Use the PDL shell (pdl2) to test out small examples. Of
particular use is the online help and apropos commands
(abbreviated ? or ?? as well) that let you find PDL docs
that may be relevant. For example:
> bash$ pdl2
> Perldl2 Shell v0.008
> PDL comes with ABSOLUTELY NO WARRANTY. For details, see the file
> 'COPYING' in the PDL distribution. This is free software and you
> are welcome to redistribute it under certain conditions, see
> the same file for details.
>
> Loaded plugins:
>
> CleanErrors Commands Completion CompletionDriver::INC
> CompletionDriver::Keywords
> CompletionDriver::LexEnv CompletionDriver::Methods DDS FindVariable History
> Interrupt LexEnv MultiLine::PPI NiceSlice PDLCommands Packages PrintControl
> ReadLineHistory
>
> Type 'help' for online help
> Type Ctrl-D or quit to exit
>
> Loaded PDL v2.007_01
>
> load_rcfile: got $HOME = /home/chm
> load_rcfile: loading /home/chm/.perldlrc
>
> pdl> ??minimum
> PDL::Lite Module: minimum PDL module OO loader
> PDL::LiteF Module: minimum PDL module function loader
> gelsd Computes the minimum-norm solution to a real linear least
> squares problem:
> gelss Computes the minimum norm solution to a real linear least
> squares problem:
> gelsy Computes the minimum-norm solution to a real linear least
> squares problem:
> histogram Calculates a histogram for given stepsize and minimum.
> limits limits derives global limits for one or more
> multi-dimensional sets of data for display purposes. It obtains
> minimum and maximum limits for each dimension based upon one
> of several algorithms.
> min Return the minimum value generable by this RNG.
> minimum Project via minimum to N-1 dimensions
> minimum_ind Like minimum but returns the index rather than the value
> minimum_n_ind Returns the index of `m' minimum elements
> minmax Returns an array with minimum and maximum values of a piddle.
> minmaximum Find minimum and maximum and their indices for a given piddle;
> mllss Computes the minimum-norm solution to a real linear least
> squares problem using a singular value decomposition.
> mllsy Computes the minimum-norm solution to a real linear least
> squares problem using a complete orthogonal
> factorization.
> mn_def_pars The function mn_def_pars() defines the initial values of the
> parameters of the function to be minimized and the
> value of the initial steps around these values that the
> minimizer will use for the first variations of the
> parameters in the search for the minimum. There are several
> optional arguments. One allows to assign names to
> these parameters which otherwise get names (Par_0,
> Par_1,....,Par_n) by default. Another two arguments can give
> lower and upper bounds for the parameters via two piddles. If
> the lower and upper bound for a given parameter
> are both equal to 0 then the parameter is unbound. By default
> these lower and upper bound piddles are set to
> zeroes(n), where n is the number of parameters, i.e. the
> parameters are unbound by default.
> set_autopthread_size ...
> Set the minimum size (in M-elements or 2^20 elements) of the
> largest PDL involved in a function where
> auto-pthreading will be performed. For small PDLs, it
> probably isn't worth starting multiple pthreads, so this
> function is used to define a minimum threshold where
> auto-pthreading won't be attempted.
> whistogram Calculates a histogram from weighted data for given stepsize
> and minimum.
>
> pdl>
(3) In the shell, help vars, ?vars will give you a table of the non-lexical
piddles in your session. Use the $pdl->info command to get information
about a specific piddle. I often string together a sequence of methods
with p at the beginning and ->info at the end to track what the actual
dimensions are of the intermediate pdl objects are and to verify that it
is what I expect.
(4) Small test cases from the PDL shell can be used as an *excellent*
way to ask a question on the perldl list. For example, a question like
"when I run the following sequence in pdl2, I get the following output
but I expected this, any idea what might be going on?". This type of
question almost always can generate a very quick response since
if someone takes a look at the post, they can just cut and paste the
lines into a pdl2 session of their own to see what is going on.
(5) And, finally, above all, please use the perldl list for general questions
and discussion and be sure to keep your replies copied to the list.
That way help can be provided by a pool of readers and the response
can be of use to others.
For the final hole patch in the image, I picked the help command above
because it seems that the minimum_ind routine might be used to get
the index of the "least bad pixel" to patch the final sigma image holes
with.
Regards,
Chris
On Thu, Mar 13, 2014 at 1:35 PM, Chris Marshall <[email protected]> wrote:
> ---------- Forwarded message ----------
> From: Schregle Roland HSLU T&A <[email protected]>
> Date: Thu, Mar 13, 2014 at 10:27 AM
> Subject: Re: [Perldl] Implementing sigma clipping on RGB image stack
>
>
> On Tue, 11 Mar 2014 23:28:20 +0100, Chris Marshall
> <[email protected]> wrote:
>
>> Hi Roland-
>>
>> It looks like you've already gotten everything working but
>> the final selective average. I've put comments inline with
>> your text below on ways I would do things for efficiency if
>> performance (speed or memory) were important.
>
>
> Hi Chris,
>
> Looks like I'm stuck again... ;^)
>
> The script does indeed seem to spend significant time glueing the
> images to the stack. I mucked about with accumulating the images in a
> fixed chunk pdl which is then cat'ed to the image stack; not only
> didn't this improve performance noticeably, it even ran out of memory!
>
> Instead I'm now simply pre-allocating the image stack once for a (user
> specified) maximum number of iterations, albeit using floats instead
> of doubles to conserve memory. Inserting the images into the stack
> using slicing works (and noticeably faster too), but triggers the
> nasty "Can't return a temporary from lvalue subroutine" error while
> debugging. While the documented workaround with temporary assignment
> suppresses this, it fails to update the image stack! Sample code
> follows:
>
> my $imgStack = float zeroes($imgComb -> dims, $maxIter);
> ...
>
> for (my $n = 0; $n <= $#imgFiles; $n++) {
> # Read image from file and add to image stack (with bogus
> # assignment as workaround for debugger!)
> my $imgSlice = ($imgStack -> slice(",,,$i-$n-1"));
> $imgSlice .= ReadImg($imgFiles [$n]);
> }
>
> Note that $i is the iteration counter, which points to the last
> completed iteration. @imgFiles contains the image filenames generated
> in the current chunk. I was under the impression assignments to slices
> always propagate back to their parent pdls, so I don't see why this
> doesn't work. Note that I have to rely on the debugger at this stage
> to figure ANYTHING out, so I'd prefer to keep this workaround at the
> expense of readability.
>
> On another note, I tried Craig's selective averaging using badvalue(),
> which worked with monochrome output, but I botched something up when
> adding on the extra dimension for RGB. I'll look into that later.
>
> One pitfall of sigma clipping is that it'll return zero for pixels
> whose values ALL deviate outside the threshold, i.e. are all bad.
> Predictably, this leaves ugly black holes in the averaged images. I
> figure in those cases it makes sense to simply drop in the single
> value which is closest to the median. How would I go about that? For
> starters, I'd get a mask of those averaged pixels which are still
> zero, multiply it by some selection which minimises the deviation from
> the mean and add it to the combined image (which should only update
> the zero elements). Something like:
>
> $zeroMask = (!$imgComb);
> $imgComb += zeroMask * where($hdrStack, min($imgStackDiff) -> dummy(0,3))
>
> Haven't tried this yet. Sounds plausible?
>
> Sorry for nagging again, would appreciate a few hints to point me in
> the right direction.
>
> Thanks & best regards,
>
>
> --Roland
>
>
> --
> Dr. Roland Schregle
> Senior Research Associate
>
> T direct: +41 41 349 39 77
> [email protected]
>
> Lucerne University of Applied Sciences and Arts
> School of Engineering and Architecture
> CC Envelopes and Solar Energy (EASE)
> Technikumstrasse 21, CH-6048 Horw
> T +41 41 349 33 11, F +41 41 349 39 60
> www.hslu.ch/ccease
_______________________________________________
Perldl mailing list
[email protected]
http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
