On 02/04/2016 09:38 AM, Vijay Bellur wrote:
On 02/03/2016 11:34 AM, Venky Shankar wrote:
On Wed, Feb 03, 2016 at 09:24:06AM -0500, Jeff Darcy wrote:
Problem is with workloads which know the files that need to be read
without readdir, like hyperlinks (webserver), swift objects etc. These
are two I know of which will have this problem, which can't be improved
because we don't have metadata, data co-located. I have been trying to
think of a solution for past few days. Nothing good is coming up :-/
In those cases, caching (at the MDS) would certainly help a lot. Some
variation of the compounding infrastructure under development for Samba
etc. might also apply, since this really is a compound operation.
Compounding in this case can help, but still without the cache, the read
has to go to the DS, and on such a compounding, the MDS would reach out
to the DS for the information than the client. Another possibility based
on what we decide as the cache mechanism.
When a client is done modifying a file, MDS would refresh it's size,
mtime
attributes by fetching it from the DS. As part of this refresh, DS could
additionally send back the content if the file size falls in range, with
MDS persisting it, sending it back for subsequent lookup calls as it does
now. The content (on MDS) can be zapped once the file size crosses the
defined limit.
Venky, when you say persisting, I assume on disk, is that right?
If so, then the MDS storage size requirements would increase (based on
amount of file data that need to be stored). As of now it is only
inodes, and as we move to a db a record. In this case we may have
*fatter* MDS partitions. Any comments/thoughts on that?
As with memory I would assume some form of eviction of data from MDS, to
control the space utilization here as a possibility.
I like the idea. However the memory implications of maintaining content
in MDS is something to watch out for. quick-read is interested in files
of size 64k by default and with a reasonable number of files in that
range, we might end up consuming significant memory with this scheme.
Vijay, I think what Venky states is to stash the file on the local
storage and not in memory. If it was in memory then brick process
restarts would nuke the cache, and either we need mechanisms to
rebuild/warm the cache or just start caching afresh.
If we were caching in memory, then yes the concern is valid, and one
possibility is some form of LRU for the same, to keep memory
consumption in check.
Overall I would steer away from memory for this use case, and use the
disk, as we do not know which files to cache (well in either case, but
disk offers us more space to possibly punt on that issue). For files
where the cache is missing and the file is small enough, either perform
async read from the client (gaining some overlap time with the app) or
just let it be, as we would get the open/read anyway, but would slow
things down.
-Vijay
_______________________________________________
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel
_______________________________________________
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel
