Re: Hadoop DNS/topology details

2013-02-20 Thread Noah Watkins
On Feb 20, 2013, at 9:57 AM, Noah Watkins wrote: > >vector osds; > } > > ceph_get_file_extents(file, offset, length, vector& extents); > > Then we could re-use the Striper or something? > > -Noah Although, I think your previous suggestion would be much simpler to do for the C api :)--

Re: Hadoop DNS/topology details

2013-02-20 Thread Noah Watkins
On Feb 20, 2013, at 9:31 AM, Sage Weil wrote: >> or something like this that replaces the current extent-to-sockaddr >> interface? The proposed interface about would do the host/ip mapping, as >> well as the topology mapping? > > Yeah. The ceph_offset_to_osds should probably also have an (op

Re: Hadoop DNS/topology details

2013-02-20 Thread Sage Weil
On Wed, 20 Feb 2013, Noah Watkins wrote: > > On Feb 19, 2013, at 4:39 PM, Sage Weil wrote: > > > However, we do have host and rack information in the crush map, at least > > for non-customized installations. How about something like > > > > string ceph_get_osd_crush_location(int osd, string

Re: Hadoop DNS/topology details

2013-02-20 Thread Noah Watkins
On Feb 19, 2013, at 4:39 PM, Sage Weil wrote: > However, we do have host and rack information in the crush map, at least > for non-customized installations. How about something like > > string ceph_get_osd_crush_location(int osd, string type); > > or similar. We could call that with "host"

Re: Hadoop DNS/topology details

2013-02-19 Thread Gregory Farnum
On Tue, Feb 19, 2013 at 4:39 PM, Sage Weil wrote: > On Tue, 19 Feb 2013, Noah Watkins wrote: >> On Feb 19, 2013, at 2:22 PM, Gregory Farnum wrote: >> > On Tue, Feb 19, 2013 at 2:10 PM, Noah Watkins >> > wrote: >> > >> > That is just truly annoying. Is this described anywhere in their docs? >> >>

Re: Hadoop DNS/topology details

2013-02-19 Thread Sage Weil
On Tue, 19 Feb 2013, Noah Watkins wrote: > On Feb 19, 2013, at 2:22 PM, Gregory Farnum wrote: > > On Tue, Feb 19, 2013 at 2:10 PM, Noah Watkins > > wrote: > > > > That is just truly annoying. Is this described anywhere in their docs? > > Not really. It's just there in the code--I can figure ou

Re: Hadoop DNS/topology details

2013-02-19 Thread Noah Watkins
On Feb 19, 2013, at 2:22 PM, Gregory Farnum wrote: > On Tue, Feb 19, 2013 at 2:10 PM, Noah Watkins wrote: > > That is just truly annoying. Is this described anywhere in their docs? Not really. It's just there in the code--I can figure out the metric if you're interested. I suspect it is loca

Re: Hadoop DNS/topology details

2013-02-19 Thread Gregory Farnum
On Tue, Feb 19, 2013 at 2:10 PM, Noah Watkins wrote: > Here is the information that I've found so far regarding the operation of > Hadoop w.r.t. DNS/topology. There are two parts, the file system client > requirements, and other consumers of topology information. > > -- File System Client -- > >

Hadoop DNS/topology details

2013-02-19 Thread Noah Watkins
Here is the information that I've found so far regarding the operation of Hadoop w.r.t. DNS/topology. There are two parts, the file system client requirements, and other consumers of topology information. -- File System Client -- The relevant interface between the Hadoop VFS and its underlying