On Thu, Nov 1, 2012 at 10:41 AM, Clifford Sobchuk <[email protected]> wrote: > > Nan, if this is in the interactive pdl2 shell, then you can use the > following to print out a 1d or 2d you can use "wcols $pdl, 'file.name';" to > print it to a file. The pdl2 shell will only print items to a certain size. > > If you are getting this error from a script that is already writing to a > file - I don't know.
The print size restriction is actually the object->string overload and not specific to the PDL shells. As mentioned by Craig, you can up the limit for the stringify operation but this is not what you want to do for real IO in the sense that the printed output is not exact as far as floating point values and is not very efficient either. --Chris > > CLIFF SOBCHUK > Core RF Engineering > Phone 613-667-1974 ecn: 8109-71974 > mobile 403-819-9233 > yahoo: sobchuk > www.ericsson.com > > “The author works for Telefonaktiebolaget L M Ericsson (“Ericsson”), who is > solely responsible for this email and its contents. All inquiries regarding > this email should be addressed to Ericsson. The web site for Ericsson is > www.ericsson.com.” > > This Communication is Confidential. We only send and receive email on the > basis of the terms set out at www.ericsson.com/email_disclaimer > > > > ________________________________ > From: Nan Yu [mailto:[email protected]] > Sent: Thursday, November 01, 2012 7:43 AM > To: Craig DeForest > Cc: [email protected] > > Subject: Re: [Perldl] how to work with large 2D matrix > > Hi, > > I’m having trouble with print out the big matrix (print > "$matrix"; gives “TOO LONG TO PRINT” error message when the matrix is big). > > Is there a way to do so? > > Or do I have to print individual element in the matrix? > > In perl, one can use TwoDArray[n][m] to refer to the value of individual > element in the 2D array. How could I do so in PDL? I know I could use > slice/range/dice to get a sub-matrix, but it gives an 1D/2D array > essentially. > > > > Thanks! > > Nan > > > > > > From: Craig DeForest [mailto:[email protected]] > Sent: Monday, October 29, 2012 11:38 PM > To: Nan Yu > Cc: Craig DeForest; [email protected] > Subject: Re: [Perldl] how to work with large 2D matrix > > > > You want to allocate the matrix all at once, then populate it. The ->glue > call makes a copy of > > the matrix, so by the time you're up to 4000x10000, you're copying between > 40MB and 320MB with each glue call. That can eat up some time. > > > > Far better do do something like: > > > > $PDL::BIGPDL = 1; > > $matrix = PDL::new_from_specification(byte,4000,50000); # use the smallest > type you can! > > print "Loading rows"; > > for $row(0..$maxrow) { > > $matrix->(:,($row)) .= get_a_row(); > > print "." unless($row%500); > > } > > > > > > > > On Oct 29, 2012, at 10:21 PM, Nan Yu <[email protected]> wrote: > > > > Hi, > > I’m working with a large 2D matrix: about 4000 x 50000. > > I used to use perl 2D array to do the job (when working with > smaller dimensions) but the memory became an issue when the dimension grows > into this size. So I decided to use PDL. > > After I finished my code and started testing it, I found out that just to > initiate the 2D matrix takes a long time. > > I start with an 1D array of 4000 elements and then use “$matrix = > $matrix->glue(1,$oneDarray)” within a loop (from 1 to 50000) to create the > 2D matrix. > > The first 5000-10000 were reasonable quick (within 10 mins) then the speed > slows down a lot. > > I’m wondering if I used a wrong way to do the initiate of > the matrix. > > Could someone please tell me how to work with the large 2D > matrix? > > Thanks! > > Nan Yu > > > > > Please note that my email address has changed to [email protected] > > ________________________________ > > *** The information contained in this communication may be confidential, is > intended only for the use of the recipient(s) named above, and may be > legally privileged. If the reader of this message is not the intended > recipient, you are hereby notified that any dissemination, distribution, or > copying of this communication, or any of its contents, is strictly > prohibited. If you have received this communication in error, please return > it to the sender immediately and delete the original message and any copies > of it. If you have any questions concerning this message, please contact the > sender. *** _______________________________________________ > Perldl mailing list > [email protected] > http://mailman.jach.hawaii.edu/mailman/listinfo/perldl > > > > > Please note that my email address has changed to [email protected] > > ________________________________ > *** The information contained in this communication may be confidential, is > intended only for the use of the recipient(s) named above, and may be > legally privileged. If the reader of this message is not the intended > recipient, you are hereby notified that any dissemination, distribution, or > copying of this communication, or any of its contents, is strictly > prohibited. If you have received this communication in error, please return > it to the sender immediately and delete the original message and any copies > of it. If you have any questions concerning this message, please contact the > sender. *** > > _______________________________________________ > Perldl mailing list > [email protected] > http://mailman.jach.hawaii.edu/mailman/listinfo/perldl > _______________________________________________ Perldl mailing list [email protected] http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
