nickva opened a new pull request, #4266:
URL: https://github.com/apache/couchdb/pull/4266
Clean up, optimize and increase test coverage for smoosh.
* Use the new, simpler persistence module. Remove the extra `activated` vs
`non-activated` state. Use `handle_continue` gen_server callback to
avoid blocking smoosh application initialization. Then, let channel
unpersist in their init function which happens outside the general
application
initialization. Persistence module commit has more details about the
implementation of the persistence module itself.
* Add an index cleanup channel and enqueue index cleanup jobs by default.
* Optimize smoosh_priority queue. The separate commit message has more
details.
* Remove gen_server bottlenecks for status and last update calls. Instead
rely on ets table lookups and gen_casts only.
* Add a few more `try ... catch`'s to avoid crashing the channels.
* Use maps to keep track of starting and active jobs in the channel.
* Re-check priority again before starting jobs. This is needed when jobs
are
un-persisted after restart and database may have been compacted or
deleted
already.
* Update periodic channel scheduled checks to have a common scheduling
mechanism.
* Remove testing-only API functions to avoid cluttering the API.
* Store messages off-heap for channels and smoosh_server.
* Instead of a per-channel `last_updated` gen_server:call(...), use a
single
access ets table which is periodically cleaned from stale entries. As an
optimization, before enqueueing the shard, check last access first. This
avoids sending an extra message to smoosh_server.
* As a protection mechanism against overload, cap the access table size at
250k entries. Don't allow enqueueing more than that many entries during
the
configured `[smoosh] staleness = Minutes` period.
* Increase test coverage from 60% to 90%
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
