Hello Christian, TajoPullServer use the local address. and we doesn’t change the loopback address. if you startup the cluster, each worker must have a unique address I think your /etc/hosts or virtualMachine does have ‘<127.0.1.1 yourhostname>’
-Jinho Best regards 2014-08-19 11:11 GMT+09:00 Christian Schwabe <[email protected]>: > > Hello Hyunsik, > > after some intensive research in the error log of Tajo I think I have found > the error. > First of all: Master and Worker converge on my MacBook. > The error is exactly in the first line of the stack trace: > > 2014-08-19 03:56:17,334 INFO org.apache.tajo.worker.Fetcher: Status: > FETCH_FETCHING, URI: > http://127.0.1.1:56178/?qid=q_1408413350431_0001&sid=1&p=0&type=h&ta=0_0 > 2014-08-19 03:56:17,362 INFO > org.apache.tajo.pullserver.TajoPullServerService: PullServer request param: > shuffleType=h, sid=1, partId=0, taskIds=[0_0] > 2014-08-19 03:56:17,363 INFO > org.apache.tajo.pullserver.TajoPullServerService: PullServer baseDir: > /tmp/tajo-christian/tmpdir/q_1408413350431_0001/output > 2014-08-19 03:56:17,386 ERROR org.apache.tajo.worker.Fetcher: Fetch failed > : > java.lang.IllegalArgumentException: invalid version format: 404 > > at > org.jboss.netty.handler.codec.http.HttpVersion.<init>(HttpVersion.java:102) > at > org.jboss.netty.handler.codec.http.HttpVersion.valueOf(HttpVersion.java:62) > at > org.jboss.netty.handler.codec.http.HttpResponseDecoder.createMessage(HttpResponseDecoder.java:104) > at > org.jboss.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:189) > at > org.jboss.netty.handler.codec.http.HttpClientCodec$Decoder.decode(HttpClientCodec.java:143) > at > org.jboss.netty.handler.codec.http.HttpClientCodec$Decoder.decode(HttpClientCodec.java:127) > at > org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:500) > at > org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435) > at > org.jboss.netty.handler.codec.http.HttpClientCodec.handleUpstream(HttpClientCodec.java:92) > at > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) > at > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) > at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) > at > org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109) > at > org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) > at > org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90) > at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:744) > > > Tajo try to connect to an IP address that is not valid. > The correct IP address on which it should connect is 127.0.0.1 but not > 127.0.1.1 > I hope my conclusions are correct up to this point. > Is there a chance to force Tajo on localhost? > > I want to reiterate that this behavior was never previously occurred. > > Best regards, > Chris > > > > > Am 17.08.2014 16:06:00, schrieb Christian Schwabe: > > Hello Hyunsik, > > thanks for your reply. Unfortunately I have never heard of "Netty" and > therefore do not know what that is exactly. How and where is this supposed > to check exactly? > > > Best regards, > Chris > Am 17.08.2014 um 09:27 schrieb Hyunsik Choi <[email protected]>: > > Hi Chris, > > According to your log message, Http Server and Client implemented in Netty > do not work correctly due to some problems. I'm expecting that your > application includes another Netty version by other dependencies. Current > Tajo uses 3.6.6.Final. Could you check if your dependency includes other > netty version? > > > 2014-08-14 08:46:26,795 INFO > org.apache.tajo.pullserver.TajoPullServerService: PullServer baseDir: > /tmp/tajo-chris/tmpdir/q_1407998756522_0001/output > 2014-08-14 08:46:26,798 ERROR org.apache.tajo.worker.Fetcher: Fetch failed > : > java.lang.IllegalArgumentException: invalid version format: FOUND > at > org.jboss.netty.handler.codec.http.HttpVersion.<init>(HttpVersion.java:102) > at > org.jboss.netty.handler.codec.http.HttpVersion.valueOf(HttpVersion.java:62) > at > org.jboss.netty.handler.codec.http.HttpResponseDecoder.createMessage(HttpResponseDecoder.java:104) > at > org.jboss.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:189) > at > org.jboss.netty.handler.codec.http.HttpClientCodec$Decoder.decode(HttpClientCodec.java:143) > at > org.jboss.netty.handler.codec.http.HttpClientCodec$Decoder.decode(HttpClientCodec.java:127) > at > org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:500) > at > org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435) > > Best regards, > Hyunsik > > > On Thu, Aug 14, 2014 at 4:04 PM, Christian Schwabe < > [email protected]> wrote: > > Hello Hyunsik, > > Thank you for your reply. > I'm already very happy that it is obviously not a mistake on my part and > it is a known issue in which of course I would like help to resolve this. > In addition I have attached the worker log. > Glad to hear that already a committer works on the second problem. > > Best regards, > Chris > > > > Am 14.08.2014 um 00:24 schrieb Hyunsik Choi <[email protected]>: > > There may be two problems: > 1) One is that a simple query causes an error. > 2) Another is endless running query. > > In the first problem, a query is too simple. So, I expect that there is > some trivial problem related data or configuration. In order to help you > figure out the problem, could you share the worker log? The worker log > contains the detailed causes of the error. > > Actually, one committer is already digging into the second problem. > Certain few cases causes a endless running query. I believe that this > problem will be fixed soon. > > Thanks, > Hyunsik > > > On Thu, Aug 14, 2014 at 4:08 AM, Christian Schwabe < > [email protected]> wrote: > > Hello guys, > > and another Mail. Sorry for spamming. > To deal with Apache Tajo's is just so exciting, because now occur once a > lot of questions and problems. > > Now I just pulled the new status from the GitHub repository. Recompiled > all with 'mvn clean package -DskipTests -Pdist -Dtar > The previously saved configuration copied back into place. Up to this > point everything usual no problem and everything. > To test the new status I execute the minimalistic example ( > http://tajo.apache.org/docs/0.8.0/jdbc_driver.html) > I used the the following statement 'select count(*) from table1 on the > mentioned dataset. > > I received the following warnings: > 2014-08-13 20:45:04.856 java[14232:1903] Unable to load realm info from > SCDynamicStore > 2014-08-13 20:45:04,925 WARN: org.apache.hadoop.util.NativeCodeLoader > (<clinit>(62)) - Unable to load native-hadoop library for your platform... > using builtin-java classes where applicable > 2014-08-13 20:45:06,372 WARN: org.apache.tajo.client.TajoClient > (getQueryResultAndWait(528)) - Query (q_1407955121364_0003) failed: > QUERY_ERROR > > For a look in the WebUI I see following execution errors: > > Finished QueryQueryIdStatusStartTime FinishTimeProgressRunTime > q_1407955121364_0003 QUERY_ERROR2014-08-13 20:45:05 2014-08-13 20:45:0650%,0 > sec > > > > > > > > >> Now the details for the Tajo Worker << > > q_1407955121364_0003 [Query Plan] > <http://christians-mbp.fritz.box:28080/queryplan.jsp?queryId=q_1407955121364_0003> > IDStateStartedFinished Running timeProgressTasks > eb_1407955121364_0003_000001 SUCCEEDED2014-08-13 20:45:052014-08-13 > 20:45:05 ,0 sec100,0%1/1eb_1407955121364_0003_000002 ERROR2014-08-13 > 20:45:052014-08-13 20:45:06 ,0 sec,0%0/1 > > Applied Session Variables > ------------------------------ > Logical Plan > > ----------------------------- > Query Block Graph > ----------------------------- > |-#ROOT > ----------------------------- > Optimization Log: > [LogicalPlan] > > ProjectionNode is eliminated. > ----------------------------- > > GROUP_BY(2)() > => exprs: (count()) > => target list: ?count (INT8) > => out schema:{(1) ?count (INT8)} > => in schema:{(0) } > SCAN(0) on default.table1 > => target list: > => out schema: {(0) } > => in schema: {(5) default.table1.id (INT4),default.table1.new_column > (TEXT),default.table1.name (TEXT),default.table1.score > (FLOAT4),default.table1.type (TEXT)} > > ------------------------------ > Distributed Query Plan > > ------------------------------------------------------------------------------- > Execution Block Graph (TERMINAL - eb_1407955121364_0003_000003) > ------------------------------------------------------------------------------- > |-eb_1407955121364_0003_000003 > |-eb_1407955121364_0003_000002 > |-eb_1407955121364_0003_000001 > ------------------------------------------------------------------------------- > Order of Execution > ------------------------------------------------------------------------------- > 1: eb_1407955121364_0003_000001 > 2: eb_1407955121364_0003_000002 > 3: eb_1407955121364_0003_000003 > ------------------------------------------------------------------------------- > > ======================================================= > Block Id: eb_1407955121364_0003_000001 [LEAF] > ======================================================= > > [Outgoing] > [q_1407955121364_0003] 1 => 2 (type=HASH_SHUFFLE, key=, num=1) > > GROUP_BY(5)() > => exprs: (count()) > => target list: ?count_1 (INT8) > => out schema:{(1) ?count_1 (INT8)} > => in schema:{(0) } > SCAN(0) on default.table1 > => target list: > => out schema: {(0) } > => in schema: {(5) default.table1.id (INT4),default.table1.new_column > (TEXT),default.table1.name (TEXT),default.table1.score > (FLOAT4),default.table1.type (TEXT)} > > ======================================================= > Block Id: eb_1407955121364_0003_000002 [ROOT] > ======================================================= > > [Incoming] > [q_1407955121364_0003] 1 => 2 (type=HASH_SHUFFLE, key=, num=1) > > GROUP_BY(2)() > => exprs: (count(?count_1 (INT8))) > => target list: ?count (INT8) > => out schema:{(1) ?count (INT8)} > => in schema:{(1) ?count_1 (INT8)} > SCAN(6) on eb_1407955121364_0003_000001 > => out schema: {(1) ?count_1 (INT8)} > => in schema: {(1) ?count_1 (INT8)} > > ======================================================= > Block Id: eb_1407955121364_0003_000003 [TERMINAL] > ======================================================= > > > > > > eb_1407955121364_0003_000002 > ------------------------------ > > GROUP_BY(2)() > => exprs: (count(?count_1 (INT8))) > => target list: ?count (INT8) > => out schema:{(1) ?count (INT8)} > => in schema:{(1) ?count_1 (INT8)} > SCAN(6) on eb_1407955121364_0003_000001 > => out schema: {(1) ?count_1 (INT8)} > => in schema: {(1) ?count_1 (INT8)} > > > Status:ERROR Started:2014-08-13 20:45:05 ~ 2014-08-13 20:45:06# Tasks: 1 > (Local Tasks: 0, Rack Local Tasks: 0)Progress:,0% # Shuffles:0Input Bytes:0 > B (0 B) Actual Processed Bytes:- Input Rows:0Output Bytes:0 B (0 B) Output > Rows:0 > ------------------------------ > Status: ALL SCHEDULED RUNNING SUCCEEDED > NoIdStatusProgress StartedRunning TimeHost1 > t_1407955121364_0003_000002_000000RUNNING,0%2014-08-13 20:45:05 1054768 ms > christians-mbp.fritz.box > > > > > > eb_1407955121364_0003_000002 > <http://christians-mbp.fritz.box:28080/querytasks.jsp?queryId=q_1407955121364_0003&ebid=eb_1407955121364_0003_000002&sort=id&sortOrder=asc&status=ALL> > ------------------------------ > IDt_1407955121364_0003_000002_000000 Progress,0%StateRUNNINGLaunch > Time2014-08-13 > 20:45:05 Finish Time-Running Time1116702 ms Hostchristians-mbp.fritz.box > Shuffles# Shuffle Outputs: 0, Shuffle Key: -, Shuffle file: -Data > Locations DataLocation{host=unknown, volumeId=-1}Fragment"fragment": > {"id": "eb_1407955121364_0003_000001", "path": > file:/tmp/tajo-chris/warehouse/eb_1407955121364_0003_000001", "start": > 0,"length": 0} Input StatisticsNo input statistics Output StatisticsNo > input statisticsFetches *eb_1407955121364_0003_000001* > http://192.168.178.101:56834/?qid=q_1407955121364_0003&sid=1&p=0&type=h > > > > >> As u can see here the query is still running and running, like an > endless loop. I dont no whats wrong with it. Its a simple query. > But the strange thing is that the same query is running correctly from the > console. > I hope this was not too much information for this moment. But I think > these are the minimum necessary logs you need to understand the described > error. While I describe this error here the query just continue now been 21 > minutes. > > > Best regards, > Chris > > > > > > > > > > >
