Hello, I re-adapted a large part of my previous work to suit the new major changes in Tajo. I documented the changes on the project's website and on Github. I estimate that in a couple of days I will be able to move forward to the final part of the project, i.e. to the physical operators.
Now, I wanted to ask You for some advice about: PART A. QUESTIONS ABOUT RUNNING THE NEW TAJO VERSION 1) I have a problem because TAJO is not executing queries, not even for queries without outer join. The log is the following. It seems that there is a problem with the resource allocation 2013-08-28 11:44:39,965 INFO event.AsyncDispatcher (AsyncDispatcher.java:register(153)) - Registering class org.apache.tajo.master.querymaster.QueryJobEvent$Type for class org.apache.tajo.master.querymaster.QueryInProgress$QueryInProgressEventHandler 2013-08-28 11:44:39,965 INFO service.AbstractService (AbstractService.java:init(81)) - Service:Dispatcher is inited. 2013-08-28 11:44:39,965 INFO service.AbstractService (AbstractService.java:init(81)) - Service:org.apache.tajo.master.querymaster.QueryInProgress is inited. 2013-08-28 11:44:39,966 INFO service.AbstractService (AbstractService.java:start(94)) - Service:Dispatcher is started. 2013-08-28 11:44:39,966 INFO service.AbstractService (AbstractService.java:start(94)) - Service:org.apache.tajo.master.querymaster.QueryInProgress is started. 2013-08-28 11:44:39,966 INFO querymaster.QueryInProgress (QueryInProgress.java:startQueryMaster(140)) - Initializing QueryInProgress for QueryID=q_1377678790404_0001 2013-08-28 11:44:39,966 WARN rm.TajoWorkerResourceManager (TajoWorkerResourceManager.java:allocateQueryMaster(101)) - No available resource for querymaster:q_1377678790404_0001 2013-08-28 11:44:39,983 INFO rm.TajoWorkerResourceManager (TajoWorkerResourceManager.java:run(196)) - ====> allocateWorkerResources:eb_0000000000000_0000_000000, required:1, allocated:0, queryMasterRequest=true, liveWorkers=0 2013-08-28 11:44:40,460 INFO rm.TajoWorkerResourceManager (TajoWorkerResourceManager.java:run(196)) - ====> allocateWorkerResources:eb_0000000000000_0000_000000, required:1, allocated:0, queryMasterRequest=true, liveWorkers=0 2013-08-28 11:44:43,460 INFO rm.TajoWorkerResourceManager (TajoWorkerResourceManager.java:run(196)) - ====> allocateWorkerResources:eb_0000000000000_0000_000000, required:1, allocated:0, queryMasterRequest=true, liveWorkers=0 2013-08-28 11:44:46,460 INFO rm.TajoWorkerResourceManager (TajoWorkerResourceManager.java:run(196)) - ====> allocateWorkerResources:eb_0000000000000_0000_000000, required:1, allocated:0, queryMasterRequest=true, liveWorkers=0 etc...(it goes on like this forever, until forced exit) Meanwhile, in the terminal I can't see any progress , as I could in the old version of Tajo. It doesn't even say Progress 0, nothing. 2) The $TAJO_HOME/bin/tajo-daemons.sh doesn't have execution rights and that is why, when I do $TAJO_HOME/bin/$TAJO_HOME/bin/start-tajo.sh I receive the following error: /home/camelia/tajo_git/incubator-tajo/tajo-dist/target/tajo-0.2.0-SNAPSHOT/bin/start-tajo.sh: line 35: exec: /home/camelia/tajo_git/incubator-tajo/tajo-dist/target/tajo-0.2.0-SNAPSHOT/bin/tajo-daemons.sh Also, when I do $TAJO_HOME/bin/$TAJO_HOME/bin/stop-tajo.sh I receive the following error: /home/camelia/tajo_git/incubator-tajo/tajo-dist/target/tajo-0.2.0-SNAPSHOT/bin/stop-tajo.sh: line 34: /home/camelia/tajo_git/incubator-tajo/tajo-dist/target/tajo-0.2.0-SNAPSHOT/bin/tajo-daemons.sh: Permission denied What is the purpose of this file tajo-daemons.sh ? I tried to chmod it for execution but then, when starting or stopping Tajo it asked for root password each time, which doesn't seem right ... 3) An e-mail received yesterday raised some questions in my mind about the new contents that tajo-site.xml should have. Until now I used : <configuration> <property> <name>tajo.rootdir</name> <value>file:/home/camelia/tajo_git</value> </property> <property> <name>tajo.cluster.distributed</name> <value>true</value> </property> </configuration> And it worked fine like this. Is it still all right or do I need to use a different one in the new version of Tajo? Mr Jihoon Son's e-mail suggested : <property> <name>tajo.master.manager.addr</name> <value>*addr:port*</value> </property> 4) About the message I received a couple of days ago: "[jira] [Work started] (TAJO-135) Bump up hadoop to 2.1.0-beta. Work on TAJO-135 started by Hyunsik Choi". I would like to inform You that my Hadoop version is Hadoop 0.20.2-cdh3u3 and I will not change it untill the end of the GSoC project on September 27. I would like to continue the project with the Hadoop version I have. PART B. QUESTIONS ABOUT CODE 5) The BasicLogicalPlanVisitor can be restricted to a specific query block by blockName? How? I need to be able to do this restriction for the outer join rewrite rule I'm working at. 6) Does the new Tajo version preserve the left deep tree shape of the query plan? What happens to subqueries: are they unnested and merged in the main query? What interests me at this moment regards the types of nodes to be found as left child and right child of a join node. I worked with the assumption that in the initial un-optimized logical plan, for a join node: the right child is always a ScanNode, whereas the left child may be a ScanNode or a JoinNode. Please contradict me if You consider otherwise. 7) I understand that FieldEval is for column of a data source, but what is BinaryEval for? 8) In class FilterPushDownRule , at line 87, the lines are inversed between them: LogicalNode left = joinNode.getRightChild(); LogicalNode right = joinNode.getLeftChild(); Shouldn't it be LogicalNode left = joinNode.getLeftChild(); LogicalNode right = joinNode.getRightChild(); ? Thank You very much in advance and after I finish this part, I'll write to You again for the physical operators part. Yours sincerely, Camelia ________________________________
