ctubbsii commented on a change in pull request #175: Scan exec blog post URL: https://github.com/apache/fluo-website/pull/175#discussion_r329807368
########## File path: _posts/blog/2019-09-30-scan-executors.md ########## @@ -0,0 +1,67 @@ +--- +title: "How Fluo Leveraged Scan Executors" +date: 2019-09-30 11:50:00 +0000 +author: Keith Turner +--- + +Accumulo 2.0 introduced [Scan Executors][1] giving control over processing of +scans in Accumulo tablet servers. Fluo has a good use case for scan executors, +notification scans. Fluo worker processes continually scan for notifications +that indicate there is work to do. All workers continually scanning for +notifications can place a lot of load on Accumulo tablet servers which could +negatively impact transactions. The new scan executor feature provides a way +to limit this load. + +Fluo utilizes this feature by [setting scan hints][2] for notification scans +indicating `scan_type=fluo-ntfy`. These hints are passed to Accumulo tablet +servers and are ignored by default. For these scan types Accumulo could be +configured to either send them to special thread pool and/or prioritize them +differently within a thread pool. Below is an example of Accumulo shell +commands that setup a special executor for notification scans. + +``` +config -s tserver.scan.executors.fnotify.threads=1 +config -t fluo_table -s table.scan.dispatcher=org.apache.accumulo.core.spi.scan.SimpleScanDispatcher +config -t fluo_table -s table.scan.dispatcher.opts.executor.fluo-ntfy=fnotify +``` + +The system setting `tserver.scan.executors.fnotify.threads=1` creates a single +threaded scan executor in each tablet server named `fnotify`. The two per table +settings configure a scan dispatcher (the SimpleScanDispatcher is built into +Accumulo) on the fluo table. The scan dispatcher is configured such that when Review comment: ```suggestion Accumulo) on the table `fluo_table`. The scan dispatcher is configured such that when ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
