Re: Fetch Taking Longer Than Expected

Thu, 13 Aug 2015 10:30:20 -0700

Looks like we have a SetGroupsCommand (setgroups) which is analogous to the Java API in TableOperations, but we don't have those nice add/remove commands for locality groups (likely as they're harder to implement atomically).
Feel free to log a JIRA if you'd like to see add/remove instead of set. We can at least have the conversation on the matter -- there may be a reason I'm not aware of :) 

Daniel Ruiz wrote:

Thanks for the quick response.  It does make sense, although it would be
nice to have an addgroup and delgroup (not sure if that exists in 1.7).
This was extremely helpful, thanks again!

V/r,
-Daniel

-----Original Message-----
From: Josh Elser [mailto:[email protected]]
Sent: Wednesday, August 12, 2015 9:20 AM
To: [email protected]
Subject: Re: Fetch Taking Longer Than Expected

Yup, that would be expected.

Remember that doing `scan -c ...` is an unbounded search over your
entire table. So, it takes approximately 3 minutes to read your
GUIDIndexTable. Because you have a single locality group, all of the
columns in your table are grouped together.

One exercise that may be interesting for yourself is to create a
locality group that has your specific column family in it, compact your
GUIDIndexTable, and rerun your `scan -c` query. The speed should be
similar to your exact scan. Removing the locality group and
re-compacting the table should return the query time back to the slow 3
minutes.

Does that make sense?

Daniel Ruiz wrote:

Hi All,

I am having an issue where column fetches are taking over a minute on
1.6.3. I don't believe this should be case and my experience in the past
supports the idea that fetches should be very fast.

For example we doing a scan on the table gives results instantly but
doing a scan -c vesselmmsitext=2706758566 takes 2 minutes and 44 seconds
(plus or minus 1 second).

Figure 1.1. Generated Test Data on GUIDIndexTable

Here is the table config



-----------+---------------------------------------------------------------+
----------------------------------------------------------------------------
-----

SCOPE | NAME | VALUE



-----------+---------------------------------------------------------------+
----------------------------------------------------------------------------
-----

default | table.balancer ..............................................
| org.apache.accumulo.server.master.balancer.DefaultLoadBalancer

default | table.bloom.enabled .........................................
| false

default | table.bloom.error.rate ......................................
| 0.5%

default | table.bloom.hash.type .......................................
| murmur

default | table.bloom.key.functor .....................................
| org.apache.accumulo.core.file.keyfunctor.RowFunctor

default | table.bloom.load.threshold .................................. |

1

default | table.bloom.size ............................................
| 1048576

default | table.cache.block.enable ....................................
| false

default | table.cache.index.enable ....................................
| true

default | table.classpath.context ..................................... |

default | table.compaction.major.everything.idle ...................... |

1h

default | table.compaction.major.ratio ................................ |

3

default | table.compaction.minor.idle ................................. |

5m

default | table.compaction.minor.logs.threshold ....................... |

3

table | table.constraint.1 .......................................... |
org.apache.accumulo.core.constraints.DefaultKeySizeConstraint

default | table.failures.ignore .......................................
| false

default | table.file.blocksize ........................................ |

0B

default | table.file.compress.blocksize ...............................
| 100K

default | table.file.compress.blocksize.index .........................
| 128K

default | table.file.compress.type .................................... |

gz

default | table.file.max .............................................. |

15

default | table.file.replication ...................................... |

0

default | table.file.type ............................................. |

rf

default | table.formatter .............................................
| org.apache.accumulo.core.util.format.DefaultFormatter

default | table.groups.enabled ........................................ |

default | table.interepreter ..........................................
| org.apache.accumulo.core.util.interpret.DefaultScanInterpreter

table | table.iterator.majc.AgeOffIterator##GUIDIndexTable .......... |
1,org.apache.accumulo.core.iterators.user.AgeOffFilter

table | table.iterator.majc.AgeOffIterator##GUIDIndexTable.opt.ttl .. |
2592000000

table | table.iterator.majc.vers .................................... |
20,org.apache.accumulo.core.iterators.user.VersioningIterator

table | table.iterator.majc.vers.opt.maxVersions .................... | 1

table | table.iterator.minc.AgeOffIterator##GUIDIndexTable .......... |
1,org.apache.accumulo.core.iterators.user.AgeOffFilter

table | table.iterator.minc.AgeOffIterator##GUIDIndexTable.opt.ttl .. |
2592000000

table | table.iterator.minc.vers .................................... |
20,org.apache.accumulo.core.iterators.user.VersioningIterator

table | table.iterator.minc.vers.opt.maxVersions .................... | 1

table | table.iterator.scan.AgeOffIterator##GUIDIndexTable .......... |
1,org.apache.accumulo.core.iterators.user.AgeOffFilter

table | table.iterator.scan.AgeOffIterator##GUIDIndexTable.opt.ttl .. |
2592000000

---------------------------------------------------------- hit any key
to continue or 'q' to quit
----------------------------------------------------------

table | table.iterator.scan.vers .................................... |
20,org.apache.accumulo.core.iterators.user.VersioningIterator

table | table.iterator.scan.vers.opt.maxVersions .................... | 1

default | table.majc.compaction.strategy ..............................
| org.apache.accumulo.tserver.compaction.DefaultCompactionStrategy

default | table.scan.max.memory .......................................
| 512K

table | @override ................................................ | 1M

default | table.security.scan.visibility.default ...................... |

default | table.split.threshold ....................................... |

1G

default | table.walog.enabled .........................................
| true



-----------+---------------------------------------------------------------+
----------------------------------------------------------------------------
-----

More Table Info:

GUIDIndexTable<http://107.23.12.24:50095/tables?t=f>

        

ONLINE

        

2

        

0

        

82.56M

        

810.00K

        

159

Please let me know if I am doing something wrong to if there is more
information you need.

V/r,

-Daniel

























					Reply via email to