[
https://issues.apache.org/jira/browse/COUCHDB-1153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089978#comment-13089978
]
Paul Joseph Davis commented on COUCHDB-1153:
--------------------------------------------
Gonna make this in two comments. This one will be the non-ranty version.
@Jan
Calling waving your hands at something and calling it bike-shedding is not
overly productive. I listed specific points on why I prefer 0.6 over "60%". If
you would like to argue something along the lines of "I see your point, but I
believe that the percent format will be more intuitive to users," that's fine.
But to dismiss it as bike-shedding is a bit off putting.
Discarding the point about reducing the redundancy of code as bike-shedding is
equally off putting. If some asks me to review code and I make suggestions, I'd
rather not expect people to walk by and say "that's just bike-shedding."
On the other hand I do admit that the order of functions can come off as
bike-shedding. Then again I also feel obliged to apologize to other Erlangers
when they mention they've looked at our code base.
@Filipe
No worries. couch_server is quite a hard bit of code to test in any sort of
thorough manner at the scale that matters. I'm still mulling over how to test
it so that I can have a way to decide if a fix has fixed anything or not. Its
on my agenda to address in the next couple weeks so I'll make sure and ping you
with anything I find.
> Database and view index compaction daemon
> -----------------------------------------
>
> Key: COUCHDB-1153
> URL: https://issues.apache.org/jira/browse/COUCHDB-1153
> Project: CouchDB
> Issue Type: New Feature
> Environment: trunk
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Priority: Minor
> Labels: compaction
>
> I've recently written an Erlang process to automatically compact databases
> and they're views based on some configurable parameters. These parameters can
> be global or per database and are: minimum database fragmentation, minimum
> view fragmentation, allowed period and "strict_window" (whether an ongoing
> compaction should be canceled if it doesn't finish within the allowed
> period). These fragmentation values are based on the recently added
> "data_size" parameter to the database and view group information URIs
> (COUCHDB-1132).
> I've documented the .ini configuration, as a comment in default.ini, which I
> paste here:
> [compaction_daemon]
> ; The delay, in seconds, between each check for which database and view
> indexes
> ; need to be compacted.
> check_interval = 60
> ; If a database or view index file is smaller then this value (in bytes),
> ; compaction will not happen. Very small files always have a very high
> ; fragmentation therefore it's not worth to compact them.
> min_file_size = 131072
> [compactions]
> ; List of compaction rules for the compaction daemon.
> ; The daemon compacts databases and they're respective view groups when all
> the
> ; condition parameters are satisfied. Configuration can be per database or
> ; global, and it has the following format:
> ;
> ; database_name = parameter=value [, parameter=value]*
> ; _default = parameter=value [, parameter=value]*
> ;
> ; Possible parameters:
> ;
> ; * db_fragmentation - If the ratio (as an integer percentage), of the amount
> ; of old data (and its supporting metadata) over the
> database
> ; file size is equal to or greater then this value, this
> ; database compaction condition is satisfied.
> ; This value is computed as:
> ;
> ; (file_size - data_size) / file_size * 100
> ;
> ; The data_size and file_size values can be obtained when
> ; querying a database's information URI (GET /dbname/).
> ;
> ; * view_fragmentation - If the ratio (as an integer percentage), of the
> amount
> ; of old data (and its supporting metadata) over the
> view
> ; index (view group) file size is equal to or greater
> then
> ; this value, then this view index compaction
> condition is
> ; satisfied. This value is computed as:
> ;
> ; (file_size - data_size) / file_size * 100
> ;
> ; The data_size and file_size values can be obtained
> when
> ; querying a view group's information URI
> ; (GET /dbname/_design/groupname/_info).
> ;
> ; * period - The period for which a database (and its view groups) compaction
> ; is allowed. This value must obey the following format:
> ;
> ; HH:MM - HH:MM (HH in [0..23], MM in [0..59])
> ;
> ; * strict_window - If a compaction is still running after the end of the
> allowed
> ; period, it will be canceled if this parameter is set to
> "yes".
> ; It defaults to "no" and it's meaningful only if the
> *period*
> ; parameter is also specified.
> ;
> ; * parallel_view_compaction - If set to "yes", the database and its views are
> ; compacted in parallel. This is only useful on
> ; certain setups, like for example when the
> database
> ; and view index directories point to different
> ; disks. It defaults to "no".
> ;
> ; Before a compaction is triggered, an estimation of how much free disk space
> is
> ; needed is computed. This estimation corresponds to 2 times the data size of
> ; the database or view index. When there's not enough free disk space to
> compact
> ; a particular database or view index, a warning message is logged.
> ;
> ; Examples:
> ;
> ; 1) foo = db_fragmentation = 70%, view_fragmentation = 60%
> ; The `foo` database is compacted if its fragmentation is 70% or more.
> ; Any view index of this database is compacted only if its fragmentation
> ; is 60% or more.
> ;
> ; 2) foo = db_fragmentation = 70%, view_fragmentation = 60%, period =
> 00:00-04:00
> ; Similar to the preceding example but a compaction (database or view
> index)
> ; is only triggered if the current time is between midnight and 4 AM.
> ;
> ; 3) foo = db_fragmentation = 70%, view_fragmentation = 60%, period =
> 00:00-04:00, strict_window = yes
> ; Similar to the preceding example - a compaction (database or view index)
> ; is only triggered if the current time is between midnight and 4 AM. If at
> ; 4 AM the database or one of its views is still compacting, the compaction
> ; process will be canceled.
> ;
> ;_default = db_fragmentation = 70%, view_fragmentation = 60%, period = 23:00
> - 04:00
> (from https://github.com/fdmanana/couchdb/compare/compaction_daemon#L0R195)
> The full patch is mostly a new module but also does some minimal changes and
> a small refactoring to the view compaction code, not changing the current
> behaviour.
> Patch is at:
> https://github.com/fdmanana/couchdb/compare/compaction_daemon.patch
> By default the daemon is idle, without any configuration enabled. I'm open to
> suggestions on additional parameters and a better configuration system.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
