[
https://issues.apache.org/jira/browse/SAMZA-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13905641#comment-13905641
]
ASF GitHub Bot commented on SAMZA-47:
-------------------------------------
GitHub user ept opened a pull request:
https://github.com/apache/incubator-samza/pull/1
SAMZA-47 Clean up configuration for key-value store
* stores.my-store.cache.size is now split into two different properties,
stores.my-store.container.cache.size.bytes (LevelDB block cache size
in bytes, per container) and stores.my-store.object.cache.size (Samza
store cache size in number of objects, per task).
* stores.my-store.write.buffer.size is now known as
stores.my-store.container.write.buffer.size.bytes (semantics changed: was
measured per-task, now measured per-container)
* stores.my-store.compress is now known as
stores.my-store.leveldb.compression (semantics changed: was a boolean,
is now a string indicating the compression codec)
* stores.my-store.batch.size is now known as
stores.my-store.write.batch.size (unchanged semantics)
* stores.my-store.leveldb.block.size is now known as
stores.my-store.leveldb.block.size.bytes (unchanged semantics)
Also changed default values for the properties that are now
per-container.
Behaviour of the following configuration properties did not change (but
docs were brought up-to-date with the code):
* stores.my-store.factory
* stores.my-store.changelog
* stores.my-store.key.serde
* stores.my-store.msg.serde
* stores.my-store.object.cache.size
Manually tested changes (using hello-samza). Not yet figured out how to
add automated tests for this.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ept/incubator-samza SAMZA-47
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-samza/pull/1.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1
----
commit fa9926a9810d997e5384a2def460faa41cced0fe
Author: Martin Kleppmann <[email protected]>
Date: 2014-02-18T21:55:17Z
SAMZA-47 Clean up configuration for key-value store
* stores.my-store.cache.size is now split into two different properties,
stores.my-store.container.cache.size.bytes (LevelDB block cache size
in bytes, per container) and stores.my-store.object.cache.size (Samza
store cache size in number of objects, per task).
* stores.my-store.write.buffer.size is now known as
stores.my-store.container.write.buffer.size.bytes (semantics changed: was
measured per-task, now measured per-container)
* stores.my-store.compress is now known as
stores.my-store.leveldb.compression (semantics changed: was a boolean,
is now a string indicating the compression codec)
* stores.my-store.batch.size is now known as
stores.my-store.write.batch.size (unchanged semantics)
* stores.my-store.leveldb.block.size is now known as
stores.my-store.leveldb.block.size.bytes (unchanged semantics)
Also changed default values for the properties that are now
per-container.
Behaviour of the following configuration properties did not change (but
docs were brought up-to-date with the code):
* stores.my-store.factory
* stores.my-store.changelog
* stores.my-store.key.serde
* stores.my-store.msg.serde
* stores.my-store.object.cache.size
Manually tested changes (using hello-samza). Not yet figured out how to
add automated tests for this.
----
> LevelDB and L1 cache use the same configuration value in KeyValueStorageEngine
> ------------------------------------------------------------------------------
>
> Key: SAMZA-47
> URL: https://issues.apache.org/jira/browse/SAMZA-47
> Project: Samza
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Jay Kreps
> Assignee: Martin Kleppmann
>
> Both seem to key off of
> cache.size
> This is not right. The L1 cache is caching a number of objects and leveldb is
> allocating a number of bytes. In general the leveldb cache should be big
> (tens of MBs) and the L1 cache small (a few thousand).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
