Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-04 Thread Ira Cooper
Pranith Kumar Karampuri writes: > On 02/03/2016 07:54 PM, Jeff Darcy wrote: >>> Problem is with workloads which know the files that need to be read >>> without readdir, like hyperlinks (webserver), swift objects etc. These >>> are two I know of which will have this problem, which can't be improve

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-04 Thread Jeff Darcy
> Even with compound fops It will still require two sequential network > operations from dht2. One to MDC and one to DC So I don't think it helps. There are still two hops, but making it a compound op keeps the server-to-server communication in the compounding translator (which should already be a

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-04 Thread Venky Shankar
On Thu, Feb 04, 2016 at 11:34:04AM +0530, Shyam wrote: > On 02/04/2016 09:38 AM, Vijay Bellur wrote: > >On 02/03/2016 11:34 AM, Venky Shankar wrote: > >>On Wed, Feb 03, 2016 at 09:24:06AM -0500, Jeff Darcy wrote: > Problem is with workloads which know the files that need to be read > withou

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-04 Thread Pranith Kumar Karampuri
On 02/03/2016 07:54 PM, Jeff Darcy wrote: Problem is with workloads which know the files that need to be read without readdir, like hyperlinks (webserver), swift objects etc. These are two I know of which will have this problem, which can't be improved because we don't have metadata, data co-lo

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-03 Thread Shyam
On 02/04/2016 09:38 AM, Vijay Bellur wrote: On 02/03/2016 11:34 AM, Venky Shankar wrote: On Wed, Feb 03, 2016 at 09:24:06AM -0500, Jeff Darcy wrote: Problem is with workloads which know the files that need to be read without readdir, like hyperlinks (webserver), swift objects etc. These are two

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-03 Thread Vijay Bellur
On 02/03/2016 11:34 AM, Venky Shankar wrote: On Wed, Feb 03, 2016 at 09:24:06AM -0500, Jeff Darcy wrote: Problem is with workloads which know the files that need to be read without readdir, like hyperlinks (webserver), swift objects etc. These are two I know of which will have this problem, whic

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-03 Thread Venky Shankar
On Wed, Feb 03, 2016 at 09:24:06AM -0500, Jeff Darcy wrote: > > Problem is with workloads which know the files that need to be read > > without readdir, like hyperlinks (webserver), swift objects etc. These > > are two I know of which will have this problem, which can't be improved > > because we d

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-03 Thread Shyam
On 02/03/2016 07:54 PM, Jeff Darcy wrote: Problem is with workloads which know the files that need to be read without readdir, like hyperlinks (webserver), swift objects etc. These are two I know of which will have this problem, which can't be improved because we don't have metadata, data co-loca

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-03 Thread Jeff Darcy
> Problem is with workloads which know the files that need to be read > without readdir, like hyperlinks (webserver), swift objects etc. These > are two I know of which will have this problem, which can't be improved > because we don't have metadata, data co-located. I have been trying to > think o

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-03 Thread Pranith Kumar Karampuri
The file data would be located based on its GFID, so before the *first* lookup/stat for a file, there is no way to know it's GFID. NOTE: Instead of a name hash the GFID hash is used, to get immunity against renames and the like, as a name hash could change the location information for the file (

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-03 Thread Shyam
On 02/03/2016 09:20 AM, Shyam wrote: On 02/02/2016 06:22 PM, Jeff Darcy wrote: Background: Quick-read + open-behind xlators are developed to help in small file workload reads like apache webserver, tar etc to get the data of the file in lookup FOP itself. What happens is, when a lookup FO

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-02 Thread Pranith Kumar Karampuri
On 02/03/2016 11:49 AM, Pranith Kumar Karampuri wrote: On 02/03/2016 09:20 AM, Shyam wrote: On 02/02/2016 06:22 PM, Jeff Darcy wrote: Background: Quick-read + open-behind xlators are developed to help in small file workload reads like apache webserver, tar etc to get the data of the

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-02 Thread Pranith Kumar Karampuri
On 02/03/2016 09:20 AM, Shyam wrote: On 02/02/2016 06:22 PM, Jeff Darcy wrote: Background: Quick-read + open-behind xlators are developed to help in small file workload reads like apache webserver, tar etc to get the data of the file in lookup FOP itself. What happens is, when a lookup

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-02 Thread Raghavendra Gowdappa
- Original Message - > From: "Pranith Kumar Karampuri" > To: "Jeff Darcy" > Cc: "Gluster Devel" > Sent: Tuesday, February 2, 2016 7:52:25 PM > Subject: Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with > small fil

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-02 Thread Shyam
On 02/02/2016 06:22 PM, Jeff Darcy wrote: Background: Quick-read + open-behind xlators are developed to help in small file workload reads like apache webserver, tar etc to get the data of the file in lookup FOP itself. What happens is, when a lookup FOP is executed, GF_CONTENT_KEY is added

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-02 Thread Pranith Kumar Karampuri
On 02/02/2016 06:22 PM, Jeff Darcy wrote: Background: Quick-read + open-behind xlators are developed to help in small file workload reads like apache webserver, tar etc to get the data of the file in lookup FOP itself. What happens is, when a lookup FOP is executed, GF_CONTENT_KEY is add

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-02 Thread Jeff Darcy
> Background: Quick-read + open-behind xlators are developed to help > in small file workload reads like apache webserver, tar etc to get the > data of the file in lookup FOP itself. What happens is, when a lookup > FOP is executed, GF_CONTENT_KEY is added in xdata with max-length and > posix

[Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-01 Thread Pranith Kumar Karampuri
hi, Background: Quick-read + open-behind xlators are developed to help in small file workload reads like apache webserver, tar etc to get the data of the file in lookup FOP itself. What happens is, when a lookup FOP is executed, GF_CONTENT_KEY is added in xdata with max-length and posix x