[
https://issues.apache.org/jira/browse/COUCHDB-1153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089361#comment-13089361
]
Benoit Chesneau commented on COUCHDB-1153:
------------------------------------------
My point hasn't be addressed too. I also put a formal -1 that have been
ignored. I'm not so happy with that but that's not big deal either.
One of the point I made join rnewson & davisp concern. I would prefer all this
this thing more evented/asynchronous and in anycase (except on startup) be
based on polling db lists and such. That open the door to a lot of expected
problem. This daemon is also not the only one to run around. I've the feeling
we could have a generic service in couch handling a pool of workers reacting on
db events with different kind of workers could be used for that and useful for
others purposes too. I will provide such thing asap (developed for refuge)
probably on thursday on the release.
Second is config / db. Having it configured in an ini file is not the best
thing to do. Having to parse n lines / dbs is awkward. I would prefer this
config like readers/admins set on a db level. On that part , it can of course
be added later but I would really prefer we handle it when 1.2 is out.
I will open another ticket for this _meta thing.
> Database and view index compaction daemon
> -----------------------------------------
>
> Key: COUCHDB-1153
> URL: https://issues.apache.org/jira/browse/COUCHDB-1153
> Project: CouchDB
> Issue Type: New Feature
> Environment: trunk
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Priority: Minor
> Labels: compaction
>
> I've recently written an Erlang process to automatically compact databases
> and they're views based on some configurable parameters. These parameters can
> be global or per database and are: minimum database fragmentation, minimum
> view fragmentation, allowed period and "strict_window" (whether an ongoing
> compaction should be canceled if it doesn't finish within the allowed
> period). These fragmentation values are based on the recently added
> "data_size" parameter to the database and view group information URIs
> (COUCHDB-1132).
> I've documented the .ini configuration, as a comment in default.ini, which I
> paste here:
> [compaction_daemon]
> ; The delay, in seconds, between each check for which database and view
> indexes
> ; need to be compacted.
> check_interval = 60
> ; If a database or view index file is smaller then this value (in bytes),
> ; compaction will not happen. Very small files always have a very high
> ; fragmentation therefore it's not worth to compact them.
> min_file_size = 131072
> [compactions]
> ; List of compaction rules for the compaction daemon.
> ; The daemon compacts databases and they're respective view groups when all
> the
> ; condition parameters are satisfied. Configuration can be per database or
> ; global, and it has the following format:
> ;
> ; database_name = parameter=value [, parameter=value]*
> ; _default = parameter=value [, parameter=value]*
> ;
> ; Possible parameters:
> ;
> ; * db_fragmentation - If the ratio (as an integer percentage), of the amount
> ; of old data (and its supporting metadata) over the
> database
> ; file size is equal to or greater then this value, this
> ; database compaction condition is satisfied.
> ; This value is computed as:
> ;
> ; (file_size - data_size) / file_size * 100
> ;
> ; The data_size and file_size values can be obtained when
> ; querying a database's information URI (GET /dbname/).
> ;
> ; * view_fragmentation - If the ratio (as an integer percentage), of the
> amount
> ; of old data (and its supporting metadata) over the
> view
> ; index (view group) file size is equal to or greater
> then
> ; this value, then this view index compaction
> condition is
> ; satisfied. This value is computed as:
> ;
> ; (file_size - data_size) / file_size * 100
> ;
> ; The data_size and file_size values can be obtained
> when
> ; querying a view group's information URI
> ; (GET /dbname/_design/groupname/_info).
> ;
> ; * period - The period for which a database (and its view groups) compaction
> ; is allowed. This value must obey the following format:
> ;
> ; HH:MM - HH:MM (HH in [0..23], MM in [0..59])
> ;
> ; * strict_window - If a compaction is still running after the end of the
> allowed
> ; period, it will be canceled if this parameter is set to
> "yes".
> ; It defaults to "no" and it's meaningful only if the
> *period*
> ; parameter is also specified.
> ;
> ; * parallel_view_compaction - If set to "yes", the database and its views are
> ; compacted in parallel. This is only useful on
> ; certain setups, like for example when the
> database
> ; and view index directories point to different
> ; disks. It defaults to "no".
> ;
> ; Before a compaction is triggered, an estimation of how much free disk space
> is
> ; needed is computed. This estimation corresponds to 2 times the data size of
> ; the database or view index. When there's not enough free disk space to
> compact
> ; a particular database or view index, a warning message is logged.
> ;
> ; Examples:
> ;
> ; 1) foo = db_fragmentation = 70%, view_fragmentation = 60%
> ; The `foo` database is compacted if its fragmentation is 70% or more.
> ; Any view index of this database is compacted only if its fragmentation
> ; is 60% or more.
> ;
> ; 2) foo = db_fragmentation = 70%, view_fragmentation = 60%, period =
> 00:00-04:00
> ; Similar to the preceding example but a compaction (database or view
> index)
> ; is only triggered if the current time is between midnight and 4 AM.
> ;
> ; 3) foo = db_fragmentation = 70%, view_fragmentation = 60%, period =
> 00:00-04:00, strict_window = yes
> ; Similar to the preceding example - a compaction (database or view index)
> ; is only triggered if the current time is between midnight and 4 AM. If at
> ; 4 AM the database or one of its views is still compacting, the compaction
> ; process will be canceled.
> ;
> ;_default = db_fragmentation = 70%, view_fragmentation = 60%, period = 23:00
> - 04:00
> (from https://github.com/fdmanana/couchdb/compare/compaction_daemon#L0R195)
> The full patch is mostly a new module but also does some minimal changes and
> a small refactoring to the view compaction code, not changing the current
> behaviour.
> Patch is at:
> https://github.com/fdmanana/couchdb/compare/compaction_daemon.patch
> By default the daemon is idle, without any configuration enabled. I'm open to
> suggestions on additional parameters and a better configuration system.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
