So the only tunable elements in the BatchWriter are memory size and timeout? Otherwise the data is available for the next scanner. Thanks for the help!
From: John Vines [mailto:[email protected]] Sent: Wednesday, August 08, 2012 14:06 To: [email protected] Subject: Re: Time delay question Once BatchWriter's return from flush() or when they automatically do a flush in the background (when their buffers get sufficiently large), the data is availble for scanners. At that point the data is at least in the in memory map, as well as the walog. As the in memory maps for the tablets for the table fill, they will write to disk. And when there are more files of given sizes than desired, major compaction will occur which will compact files to a file. So when you scan, data is merged from the in memory map and the files. John On Wed, Aug 8, 2012 at 2:09 PM, Keith Turner <[email protected]> wrote: Once the batch writer flushes, the data is available for read by the batch scanner. On the server side data is read from memory and files. On Wed, Aug 8, 2012 at 1:53 PM, <[email protected]> wrote: > I have a set of actors that are doing batchWriters and another set doing > batchScanner(s) and I'm getting questions from my users about the time > delay between them. So my question is, what are all the tunable > parameters between writes and reads? > > Do I have the following facts correct? > > BatchWriters have a time value for flushing to memory cache > Minor compaction has its time frequency for flushing to WALogs > Major compaction has its time frequency for flushing to disc > > So where all does a batchScanner get its data from? > > Any/all help is appreciated. > > > Bob Thorman > Engineering Fellow > L-3 Communications, ComCept > 1700 Science Place > Rockwall, TX 75032 > (972) 772-7501 <tel:%28972%29%20772-7501> work > [email protected] > [email protected] > >
