Ok, but how come
when I run a similiar call (with less returned rows 1000 vs 25k in the
previous one) it runs through the iterator very quickly? (See Below)
Also, how do I determine the row size? It's just text data, and really not
much.
Finally, is there a way to query for rows that do not have a column? (Ie all
rows without Files:path1)
HBaseTableDataManagerImpl htdmni = new HBaseTableDataManagerImpl(
"GS_Applications");
String[] columns = { "Files:path1" };
log.info("Getting all Rows with Files");
Scanner s = htdmni.getScannerForAllRows(columns);
log.info("Got all Rows with Files");
Iterator<RowResult> iter = s.iterator();
out
.write("Application_Full_Name,Version,Application_installer_name,Operating
System, Application_platform
,Application_sub_category,md5Hash,Sha1Hash,Sha256Hash,filepath,fileName,modified,size,operation\n");
out.write("<BR>");
while (iter.hasNext())
{
Ananth T Sarathy
On Tue, Oct 20, 2009 at 4:44 PM, Jean-Daniel Cryans <[email protected]>wrote:
> If you have a very slow data source (S3), then it fetches 100 row
> before coming back to your client with all of them and that can take a
> lot of time. Also make sure that 100 of your rows can fit in a region
> server's memory. How big is each row?
>
> J-D
>
> On Tue, Oct 20, 2009 at 1:32 PM, Ananth T. Sarathy
> <[email protected]> wrote:
> > I am running this code where
> >
> > getScannerForAllRows(columns) just does return table.getScanner(columns);
> >
> > and the table has setScannerCaching(100);
> >
> > But it spins forever after getting the iterator. Why would that be? How
> can
> > I speed it up?
> >
> > HBaseTableDataManagerImpl htdmni = new HBaseTableDataManagerImpl(
> > "GS_Applications");
> >
> > String[] columns = { "Files:Name" };
> > log.info("Getting all Rows with Files");
> > Scanner s = htdmni.getScannerForAllRows(columns);
> > log.info("Got all Rows with Files");
> > log.info("Getting Iterator");
> >
> > Iterator<RowResult> iter = s.iterator();
> > log.info("Got Iterator");
> >
> > while (iter.hasNext())
> > {
> > log.info("Getting next Row");
> > RowResult rr = iter.next();
> >
> >
> > Ananth T Sarathy
> >
>
