On 11/12/2014 07:43 AM, kmx wrote: > my $dbh = DBI->connect($dsn); > my $pdl = pdl($dbh->selectall_arrayref($sql_query)); > > But it does not scale well for very large data (millions of rows).
Hi KMX If you're using Postgresql you should use the DBD::Pg->pg_getcopydata using the "COPY mytable to STDOUT" functionality for accessing millions of rows. You can do this in async or sync mode. This will get you there faster than using selectall_arrayref(). This allows you to get the rows without having to redesign your DB. SQLite has a stream API but I am not familiar with it. --Vikas
_______________________________________________ Perldl mailing list Perldl@jach.hawaii.edu http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
