On 03/04/2014 11:36 AM, Jose A. Lopes wrote:
On Feb 27 18:06, Constantinos Venetsanopoulos wrote:
On 02/27/2014 05:38 PM, Jose A. Lopes wrote:
On Feb 27 17:31, Constantinos Venetsanopoulos wrote:
Hi,
On 02/27/2014 04:47 PM, Jose A. Lopes wrote:
Hi,
riba suggested to share the cache design on the mailing list because
others might have comments. This cache is used for the OS installs in
the following use case:
Suppose you want to install several vms with the same image which is
stored somewhere on a remote repository but you only want to
download the image once (per node).
While reading this keep in mind that the cache is made simple on
purpose. We could add lots of features but I think currently we do
not have other uses cases that justify making it more complicated.
Also, if there is a general interest in the cache we can also make a
design doc.
The cache contains an index that is persisted to disk and operations
that modify the index must first acquire a filesystem lock on the
index file because the node daemon runs jobs that do not share memory.
The index file is read often from disk but it is only written back if
it has been modified (if it is dirty). Currently the cache index and
files reside in '/var/cache/ganeti'. riba suggested using '/srv'
instead because there is more space there. It's also a possibility.
Downloads do not hold a lock on the index but do use an auxiliary lock
in order to prevent and synchronize parallel downloads on the same
file. File checksums (called fingerprints) are also calculated
without holding the index lock. Currently, bad fingerprints simply
issue a warning on the log file and the user needs to check the log
file because this warning is not displayed on the CLI.
Each cached resource has a dtime (like ctime and mtime) but fo death.
The dtime specifies the absolute time when a resource is expunged.
Once the dtime has expired, the next cache flush will delete the
resources.
The cache uses an ensure semantics which means resources are created
or modified using the same function. This function simply ensures
that the specified state is in the cache after it has been executed.
When a resource is cached, a file path pointing to the file that is
being cached is returned. This file path is calculated from the URL
of the resource and therefore the path is the same in all nodes.
Also, by returning a file path we can cache virtually any file.
On the command line, the following commands are used:
gnt-cluster cache-modify ...
gnt-cluster cache-list ...
It is also possible to have gnt-cache, if it becomes necessary.
Finally, riba suggested adding a space limit to the cache. This way
the cache would eliminate resources to free up space if the limit was
reached. This complicates things considerably as it becomes necessary
to have a policy on what to delete, etc.
Comments/feedback is highly appreciated.
Just 2 questions:
1. Will there be the option to disable the cache altogether?
2. If I understand correctly (from the OS design doc) the first
implementation of fetching an Image will include fetching (a)
from an http/ftp location or (b) a local file location. I
guess cache makes sense when fetching from (a). What happens
with (b)? If cache is enabled it will be used for both?
If you don't want the cache we could have a cluster option to
enable/disable the cache.
I think that this would be definitely a good option to have.
Just to clarify, by saying the cache is not
used I just mean that the file is downloaded, the instance is created
and the file is immediately deleted afterwards. Do this sound good?
Oh, so when you are fetching from an http/ftp location you are
always saving the image locally before you copy it to the
instance's disk? Even if the cache is disabled?
What happens if one doesn't want to allocate any space on
the node? Wouldn't it better for the mechanism to directly
pass the image to the target disk, if the admin chooses so?
(e.g.: by doing curl <location> | dd <target_disk)
Is 'dd' used to limit the maximum file size, given that curl cannot do
this reliably?
We do this to buffer curl's output and send it to the disk
in bigger chunks, by tuning dd's bs parameter. Also, in
general we find that using oflag=direct with dd helps with
not polluting the host page cache with streaming image data,
and has better performance.
Thanks,
Constantinos
IMHO, the caching mechanism should be separate and independent
from the fetching mechanism. The admin should have the option
to never store anything on the node if he/she chooses to go
with http fetch + no cache.
What do you think?
Thanks,
Constantinos
Thanks,
Constantinos
Cheers,
Jose
