In addition to previous mail, how does Optiq decide the data size fit in the memory or when does it use the swap operation?
On Sun, Mar 24, 2013 at 5:09 PM, burakkk <[email protected]> wrote: > Thanks for the answers. I searched a little bit. I have some questions > about it. > > 1. In the high level architecture page there are some pluggable sources > that can be rdbms, mongodb or hadoop etc. But every source has different > data model. For example when we use mongodb(mongoql), does drill pipe to > mongodb to running the query or drill actually read the data which mongodb > store its own way? If the answer is drill send mongodb to get the results, > there's no need to parse query and prepare a logical plan, right? It's also > eligiable to RDBMS(Oracle, mysql, postresql..etc), Hive, Hbase. > > 2. Drill is using optiq to prepare the execution plan(cost based > optimizer) so that it decides to which join method(hash join, merge > join..etc) will use or what execution plan will be, right? From Apache > Drill Plan Syntax I've seen some analytical sql such as window > frame, RunningAggregate. Is it in progress? Can we add these functions > RANK, CUBE, LAG, LEAD, ROW_NUMBER? They're very good functions when you > develop data mining application. Is the source code in the > https://github.com/julianhyde/optiq.git page? > > 3. I can't find the physical plans of code in github repository. For > example where's the merge join impleatition? How can I find it? > > I think I can help you one or more of with these items: DRILL-7, DRILL-10, > DRILL-17, DRILL-34, DRILL-32, DRILL-49. DRILL-7's priority is blocked. Some > of these issues are in progress. Where do I begin? > > Thanks > Best Regards... > > > > > On Sun, Mar 24, 2013 at 7:21 AM, Ted Dunning <[email protected]>wrote: > >> Sounds like an excellent background. >> >> There are three areas of work going on right now with differing levels of >> specialized expertise required: >> >> a) adding to the list of logical and physical operators. The general >> outline of these can be found in the logical syntax document (see the home >> page for a link). There are JIRA's already opened for several of these, >> but I don't think we have a wiki list of all of the operators known to be >> needed yet. The skills here are mostly the ability to go from a fairly >> broad specification to detailed Java code with unit tests. >> >> b) extending/completing the SQL to logical plan translation system. >> Julian >> Hyde has had his hands all over that and has a github work-in-progress >> that >> he has been exposing. The parser is complete (pre-existing) and the >> current work involves building rules for Optiq (cost based optimizer) and >> adapting the framework of Optiq in order to handle nested data. Doing >> this >> work necessarily entails knowledge of Optiq, which is relatively new. >> >> c) building out the distributed execution framework. This part is much >> less well specified at this point so the mailing list is the place to find >> out more about that. The necessary skills are familiarity if protobufs, >> RPC mechanisms and efficient Java programming. >> >> Of course there are always the additional areas of >> >> d) document/test/cleanup the current code >> >> e) do wiki gardening. Writing down what you learn about the system as you >> get going on (a), (b) or (c) is a grand way to help the project. The wiki >> is open to all so you can get started on this immediately. For that >> matter, simply turning this email into a roadmap wiki page would be a >> start >> on this. >> >> The high-level architecture page, for instance could be a good place to >> start extending the high level documentation. See: >> >> https://cwiki.apache.org/confluence/display/DRILL/High-level+Architecture >> >> >> >> On Sun, Mar 24, 2013 at 3:34 AM, burakkk <[email protected]> wrote: >> >> > Hi, >> > I want to contribute to this project. I've worked on database systems, >> > stream processing, data integration. I read the contributing page and >> I've >> > got the source code from >> > https://git-wip-us.apache.org/repos/asf/incubator-drill.git >> > What do I need to do, open a jira request or wait for you? >> > >> > Thanks >> > Best regards... >> > >> > -- >> > >> > *BURAK ISIKLI** *| *http://burakisikli.wordpress.com* >> > >> > > > > -- > > *BURAK ISIKLI** *| *http://burakisikli.wordpress.com* > * > * > -- *BURAK ISIKLI** *| *http://burakisikli.wordpress.com* * *
