I think the attachment is missing. Steven On Thu, Sep 10, 2015 at 10:14 AM, Eldon Carman <[email protected]> wrote:
> I updated the diagrams based on the feedback. The Yarn diagram opens the > question about how to return the result from a query. I think this > discussion be started in a new thread. Otherwise, what do you think of the > new diagrams. > > On Tue, Aug 18, 2015 at 9:43 PM, Michael Carey <[email protected]> > wrote: > > > These look nice! Technical Q - is "layout unknown" the truth in terms of > > how assignments of processing to the NC's is being done? There is some > > (future) opportunity to do somewhat better, if desired, if so, as it > would > > be possible for the HDFS name node to provide info that the compiler > could > > use to try and set location constraints for the Hyracks operators - so > that > > the latter two figures behave closer to the first one as well (instead of > > being location-unaware). > > > > Cheers, > > Mike > > > > > > > > On 8/18/15 4:13 PM, Preston Carman wrote: > > > >> The figures have been updated based on Till's feedback. I also noticed I > >> did not include the Yarn figure link. > >> > >> - Full names of processes > >> - Legend added > >> - Added outline to represent cluster > >> - Standardized the process > >> > >> The figures seem to better express the logical and physical layout > better > >> now. Ready for the next round of suggestions. > >> > >> Preston > >> > >> > >> VXQuery Cluster: > >> > >> > https://docs.google.com/drawings/d/1PZbvJk-G0J3hQffd-fFr2n893bXSNg3xfXFexM5c2A8/edit?usp=sharing > >> > >> VXQuery Cluster using HDFS: > >> > >> > https://docs.google.com/drawings/d/1ge-0h8wa0Epio42Wor-SeBoafQdLSZxfKZFFQtcN1w0/edit?usp=sharing > >> > >> VXQuery Yarn Cluster using HDFS: > >> > >> > https://docs.google.com/drawings/d/13_kP4Yt1ze_pgqQcbVLmlBOxE6aX0Pmjg3FT2q4XX2k/edit?usp=sharing > >> > >> On Mon, Aug 17, 2015 at 4:08 PM, Till Westmann <[email protected]> > wrote: > >> > >> Hi Preston, > >>> > >>> Thanks for creating those diagrams! > >>> > >>> A few comments/proposals: > >>> 1) I think that it would be good clarify the meaning of the shapes and > >>> lines. For the first diagram I read regular rectangles as machines, > round > >>> rectangles as processes and the rectangle with the wavy bottom as > files. > >>> On the second one I'm not sure if the rounded rectangle around HDFS is > a > >>> process. Maybe we could add a legend for the diagrams? > >>> 2) When naming the machines I would replace "laptop" with "client" as > >>> that's more generic and potentially fix the spelling of controller. > >>> However, I think that the naming of the "Hyracks machines" doesn't add > a > >>> lot. Maybe we could just expand on the name of the processes to > >>> NodeController and ClusterController and not have names for the > >>> individual > >>> cluster nodes. Having he long process names would also ease the > >>> connection > >>> between the diagrams and the code. > >>> > >>> Does this make sense? > >>> > >>> Cheers, > >>> Till > >>> > >>> > >>> On 17 Aug 2015, at 12:05, Eldon Carman wrote: > >>> > >>> The following diagrams are intended to be used on our documentation > site > >>> > >>>> (as images in the HTML). I think they will be helpful in discussing > the > >>>> actual architecture of the VXQuery cluster, especially in Yarn. > >>>> > >>>> Please post questions or suggestions on how to clarify or improve the > >>>> diagrams or cluster architecture. > >>>> > >>>> > >>>> VXQuery Cluster: > >>>> > >>>> > >>>> > https://docs.google.com/drawings/d/1PZbvJk-G0J3hQffd-fFr2n893bXSNg3xfXFexM5c2A8/edit?usp=sharing > >>>> > >>>> VXQuery Cluster using HDFS: > >>>> > >>>> > >>>> > https://docs.google.com/drawings/d/1ge-0h8wa0Epio42Wor-SeBoafQdLSZxfKZFFQtcN1w0/edit?usp=sharing > >>>> > >>>> VXQuery Yarn Cluster using HDFS: > >>>> > >>>> > >> > https://docs.google.com/drawings/d/13_kP4Yt1ze_pgqQcbVLmlBOxE6aX0Pmjg3FT2q4XX2k/edit?usp=sharing > >> > >> > > >
