Hello Hyunsik, thanks anyway. One question is still unanswered. How To configure Tajo for parallel processing? I already describe in the post before this one.
Best regards, Chris On 18. September 2014 06:42:51 MESZ, Hyunsik Choi <[email protected]> wrote: >iimissed one thing. I'd like to reserve your dinner when I visit German >someday :p > >Anyway, thank you for your interest in Tajo. Please keep going your >trials >with Tajo. > >Thanks! > >2014년 9월 18일 목요일, Christian Schwabe<[email protected]>님이 작성한 >메시지: > >> >> Hello together, >> >> I am speechless. The query has been successfully completed. Running >time: 30 >> minutes. >> >> I've done the following: >> git pull >> git clean -d -f >> git reset --hard >> patch -p0 < TAJO-1049_3.patch >> >> You are great. I'm very thankful. >> If you would live in Germany, I would invite you to dinner, all of >your >> team :-D >> >> It is now important for me to understand yet what is there now >happens in >> the background. In https://issues.apache.org/jira/browse/TAJO-1048 >Fixed >> the bug that the session variables have some parameters are not >> overwritten at run time, right? >> Difficult with the understanding it is at >> https://issues.apache.org/jira/browse/TAJO-1049th What is the >limiting >> factor you have fixed it, which now apparently had this effect? >> >> In the just traversed test I noticed that only one task is processed. >I >> started just another run with the following configuration in >tajo-site.xml >> to use multiple cores: >> >> <configuration> >> <property> >> <name>tajo.resource-tracker.heartbeat.timeout-secs</name> >> <value>240000</value> >> </property> >> <property> >> ><name>tajo.worker.resource.tajo.worker.resource.cpu-cores</name> >> <value>4</value> >> </property> >> </configuration> >> >> But the tasks are apparently still only processed serially by one >thread. >> Is this still an incorrect configuration due? Before the error was >not >> fixed, I had made the same settings and determines the tasks were >> distributed on three cores with four selected cores. >> >> I would like to highlight particularly once: I am very grateful to >you. I >> very much hope that other interested persons in Tajo from our >discussions >> and the experience gained in this and also can learn from it. >> >> Warm regards and a thankful, >> Chris >> >> Am 17.09.2014 20:04:21, schrieb Hyunsik Choi: >> >> You don't need to apply TAJO-983.140916.patch. As I mentioned, this >> patch is still ongoing. You only need TAJO-1049_3.patch. >> >> Thanks, >> Hyunsik >> >> >> On Thu, Sep 18, 2014 at 2:45 AM, Christian Schwabe >> <[email protected] <#14885373167cb354_>> wrote: >> >> Hello Hyunsik, >> >> Done. Same errors. >> No compilation possible. >> >> Warm regards, >> Chris >> >> >> Am 17.09.2014 um 19:05 schrieb Hyunsik Choi <[email protected] >> <#14885373167cb354_>>: >> >> Did you update and clean your git repository? In order to apply the >> submitted patches without conflict, you should use the latest and >> clean revision. >> >> In order to clean and update your revision, please type the following >> commands: >> >> git clean -d -f >> git reset --hard >> patch -p0 < filename.patch >> >> >> Best regards, >> Hyunsik >> >> On Thu, Sep 18, 2014 at 1:57 AM, Christian Schwabe >> <[email protected] <#14885373167cb354_>> wrote: >> >> Hello Hyunsik, >> >> i read the bugs to this patch under the review. >> What i’ve done: >> -pull to the latest review >> -I applied the following patches: >> >> patch -p0 <TAJO-1049_3.patch >> patch -p0 <TAJO-983.140916.patch >> >> get the following output: >> >> >> christians-mbp:tajo chris$ patch -p0 <TAJO-983.140916.patch >> patching file >tajo-core/src/main/java/org/apache/tajo/worker/Fetcher.java >> Hunk #2 succeeded at 52 (offset -1 lines). >> Hunk #3 FAILED at 68. >> 1 out of 6 hunks FAILED -- saving rejects to file >> tajo-core/src/main/java/org/apache/tajo/worker/Fetcher.java.rej >> patching file >tajo-core/src/main/java/org/apache/tajo/worker/Task.java >> Hunk #1 FAILED at 48. >> Hunk #2 succeeded at 97 (offset 1 line). >> Hunk #3 succeeded at 197 (offset 1 line). >> Hunk #4 succeeded at 594 (offset 1 line). >> Hunk #5 succeeded at 639 (offset 1 line). >> Hunk #6 FAILED at 699. >> Hunk #7 succeeded at 754 (offset 2 lines). >> 2 out of 7 hunks FAILED -- saving rejects to file >> tajo-core/src/main/java/org/apache/tajo/worker/Task.java.rej >> patching file >> >> >tajo-core/src/main/java/org/apache/tajo/worker/dataserver/retriever/FileChunk.java >> patching file >> tajo-core/src/test/java/org/apache/tajo/worker/TestFetcher.java >> Hunk #2 FAILED at 94. >> Hunk #3 FAILED at 140. >> Hunk #4 FAILED at 170. >> Hunk #5 FAILED at 204. >> Hunk #6 FAILED at 220. >> 5 out of 6 hunks FAILED -- saving rejects to file >> tajo-core/src/test/java/org/apache/tajo/worker/TestFetcher.java.rej >> patching file >> >> >tajo-yarn-pullserver/src/main/java/org/apache/tajo/pullserver/TajoPullServerService.java >> patching file >> >> >tajo-yarn-pullserver/src/main/java/org/apache/tajo/pullserver/retriever/FileChunk.java >> christians-mbp:tajo chris$ patch -p0 <TAJO-1049_3.patch >> patching file >> >tajo-core/src/main/java/org/apache/tajo/master/querymaster/SubQuery.java >> >> Patchs seems to be applied correctly, despite the error. >> >> recompiled code with mvn clean package -DskipTests -Pdist -Dtar >> -Dhadoop.version=2.4.0 and get the following output — see attached >log from >> maven. Build process crashed. >> >> There seems to be something wrong. I hope i’ve done all right. >> >> >> >> >> >> Warm regards, >> Chris >> >> >> >> >> Am 17.09.2014 um 16:39 schrieb Hyunsik Choi <[email protected] >> <#14885373167cb354_>>: >> >> Hi Chris, >> >> Do you mean that your query is finished well? Good to hear. But, you >> need to know that the patch still has some bugs. Please see the >review >> https://reviews.apache.org/r/25242/. >> >> If your problem is solved by this patch, your problem was related to >> Fetcher. I'm also expecting your problem is related to this issue >> (https://github.com/apache/tajo/pull/141/files), which already >> committed to master branch. You can use this fix from master branch >> too. >> >> Please refer to this document in order to learn how to apply patch. >We >> mostly use '--no-prefix' option in order to generate patch files. So, >> you should use the patch command with -p0 instead of -p1 as follows: >> >> patch -p0 < ${PATCH_FILE}.patch >> >> By default, patch command uses -p1. >> >> Thanks, >> Hyunsik >> >> On Wed, Sep 17, 2014 at 8:07 AM, Christian Schwabe >> <[email protected] <#14885373167cb354_>> wrote: >> >> Hello guys, >> >> after days of efforts to apply the patch from May Hai, I finally be >able to >> announce a success today. I want to a brief description where the >problem >> was. The few guides out there to apply a patch, using the Git clients >often >> use git apply. However, I had no success. See discussion below. >> >> christians-mbp:tajo chris$ git apply --check >> /Users/chris/Desktop/TAJO-983.140916.patch >> error: src/main/java/org/apache/tajo/worker/Fetcher.java: No such >file or >> directory >> error: src/main/java/org/apache/tajo/worker/Task.java: No such file >or >> directory >> error: >> >src/main/java/org/apache/tajo/worker/dataserver/retriever/FileChunk.java: >> No >> such file or directory >> error: src/test/java/org/apache/tajo/worker/TestFetcher.java: No such >file >> or directory >> error: >src/main/java/org/apache/tajo/pullserver/TajoPullServerService.java: >> No such file or directory >> error: >src/main/java/org/apache/tajo/pullserver/retriever/FileChunk.java: >> No >> such file or directory >> >> ########################################### >> >> christians-mbp:tajo chris$ git apply --check >> /Users/chris/Desktop/TAJO-983.140916.patch >> error: src/main/java/org/apache/tajo/worker/Fetcher.java: No such >file or >> directory >> error: src/main/java/org/apache/tajo/worker/Task.java: No such file >or >> directory >> error: >> >src/main/java/org/apache/tajo/worker/dataserver/retriever/FileChunk.java: >> No >> such file or directory >> error: src/test/java/org/apache/tajo/worker/TestFetcher.java: No such >file >> or directory >> error: >src/main/java/org/apache/tajo/pullserver/TajoPullServerService.java: >> No such file or directory >> error: >src/main/java/org/apache/tajo/pullserver/retriever/FileChunk.java: >> No >> such file or directory >> >> ########################################### >> >> christians-mbp:tajo chris$ git apply -v --check >> /Users/chris/Desktop/TAJO-983.140916.patch >> Checking patch src/main/java/org/apache/tajo/worker/Fetcher.java... >> error: src/main/java/org/apache/tajo/worker/Fetcher.java: No such >file or >> directory >> Checking patch src/main/java/org/apache/tajo/worker/Task.java... >> error: src/main/java/org/apache/tajo/worker/Task.java: No such file >or >> directory >> Checking patch >> >src/main/java/org/apache/tajo/worker/dataserver/retriever/FileChunk.java... >> error: >> >src/main/java/org/apache/tajo/worker/dataserver/retriever/FileChunk.java: >> No >> such file or directory >> Checking patch >src/test/java/org/apache/tajo/worker/TestFetcher.java... >> error: src/test/java/org/apache/tajo/worker/TestFetcher.java: No such >file >> or directory >> Checking patch >> >src/main/java/org/apache/tajo/pullserver/TajoPullServerService.java... >> error: >src/main/java/org/apache/tajo/pullserver/TajoPullServerService.java: >> No such file or directory >> Checking patch >> src/main/java/org/apache/tajo/pullserver/retriever/FileChunk.java... >> error: >src/main/java/org/apache/tajo/pullserver/retriever/FileChunk.java: >> No >> such file or directory >> >> ########################################### >> >> christians-mbp:tajo-core chris$ git apply >> /Users/chris/Desktop/TAJO-983.140916.patch >> /Users/chris/Desktop/TAJO-983.140916.patch:58: trailing whitespace. >> >> >> >> /Users/chris/Desktop/TAJO-983.140916.patch:80: trailing whitespace. >> >> >> >> /Users/chris/Desktop/TAJO-983.140916.patch:487: trailing whitespace. >> return " (start=" + startOffset() + ", length=" + length + ", >> fromRemote=" + fromRemote >> warning: 3 lines add whitespace errors. >> >> ########################################### >> >> Removal of whitespaces also brought no success. >> My hope depended least on this patch. As far Tajo not run in my >environment >> and my standard configuration, I will unfortunately have to complete >my >> thesis with the result that it is currently not suitable for >practical use. >> This really is not a reproach to formulate that clear again. I really >do >> appreciate your hard work very much. However, I also think it's >slightly >> doubtful that the advice you have given can't read anywhere and only >> inquired. I think that's still a pity. Surely you can not really >expect, or >> rather the can but not really your goal to be always be the first >point of >> contact. I think at this point there is still a lot of catching up to >do. >> This is still a large deficit. Surely you can not really expect, or >rather >> it is hopefully not really your goal to be always be the first point >of >> contact. I think at this point there is still a lot of catching up to >do. >> >> I am not come within the scope of my thesis within the time to the >desired >> goal, but of course I'm still interested in a close contact with you >and >> and >> a further effort to solve the outstanding problem. >> >> Until I have encountered the following with the patch was >successfully >> carried out. Unfortunately, this patch did not bring the hoped >solution. No >> change in the behavior described herein. Are there any other ideas >that I >> could try to fix the problem? >> >> I really thank you very much for your support thus far. I really >thank you >> very much for your support thus far. I really thank you very much for >your >> support thus far. I really thank you very much for your support thus >far. I >> really thank you very much for your support thus far. I really thank >you >> very much for your support thus far. I really thank you very much for >your >> support thus far. I really thank you very much for your support thus >far. >> ;-) >> >> >> christians-mbp:tajo chris$ cd tajo-yarn-pullserver/ >> christians-mbp:tajo-yarn-pullserver chris$ patch -p1 < >> ../TAJO-983.140916.patch >> can't find file to patch at input line 5 >> Perhaps you used the wrong -p or --strip option? >> The text leading up to this was: >> -------------------------- >> |diff --git >tajo-core/src/main/java/org/apache/tajo/worker/Fetcher.java >> tajo-core/src/main/java/org/apache/tajo/worker/Fetcher.java >> |index 64475fe..7310736 100644 >> |--- tajo-core/src/main/java/org/apache/tajo/worker/Fetcher.java >> |+++ tajo-core/src/main/java/org/apache/tajo/worker/Fetcher.java >> -------------------------- >> File to patch: >> Skip this patch? [y] y >> Skipping patch. >> 6 out of 6 hunks ignored >> can't find file to patch at input line 98 >> Perhaps you used the wrong -p or --strip option? >> The text leading up to this was: >> -------------------------- >> |diff --git tajo-core/src/main/java/org/apache/tajo/worker/Task.java >> tajo-core/src/main/java/org/apache/tajo/worker/Task.java >> |index 7b4cbe1..ecc4078 100644 >> |--- tajo-core/src/main/java/org/apache/tajo/worker/Task.java >> |+++ tajo-core/src/main/java/org/apache/tajo/worker/Task.java >> -------------------------- >> File to patch: y >> y: No such file or directory >> Skip this patch? [y] y >> Skipping patch. >> 7 out of 7 hunks ignored >> can't find file to patch at input line 339 >> Perhaps you used the wrong -p or --strip option? >> The text leading up to this was: >> -------------------------- >> |diff --git >> >> >tajo-core/src/main/java/org/apache/tajo/worker/dataserver/retriever/FileChunk.java >> >> >tajo-core/src/main/java/org/apache/tajo/worker/dataserver/retriever/FileChunk.java >> |index 4f11168..911c0c2 100644 >> |--- >> >> >tajo-core/src/main/java/org/apache/tajo/worker/dataserver/retriever/FileChunk.java >> |+++ >> >> >tajo-core/src/main/java/org/apache/tajo/worker/dataserver/retriever/FileChunk.java >> -------------------------- >> File to patch: y >> y: No such file or directory >> Skip this patch? [y] y >> Skipping patch. >> 1 out of 1 hunk ignored >> can't find file to patch at input line 362 >> Perhaps you used the wrong -p or --strip option? >> The text leading up to this was: >> -------------------------- >> |diff --git >tajo-core/src/test/java/org/apache/tajo/worker/TestFetcher.java >> tajo-core/src/test/java/org/apache/tajo/worker/TestFetcher.java >> |index b755e02..cb19d54 100644 >> |--- tajo-core/src/test/java/org/apache/tajo/worker/TestFetcher.java >> |+++ tajo-core/src/test/java/org/apache/tajo/worker/TestFetcher.java >> -------------------------- >> File to patch: y >> y: No such file or directory >> Skip this patch? [y] y >> Skipping patch. >> 6 out of 6 hunks ignored >> patching file >> src/main/java/org/apache/tajo/pullserver/TajoPullServerService.java >> patching file >> src/main/java/org/apache/tajo/pullserver/retriever/FileChunk.java >> christians-mbp:tajo-yarn-pullserver chris$ pwd >> /Users/chris/6_Semester/Thesis/07_git/tajo/tajo-yarn-pullserver >> >> >> Warm regards, >> Chris >> >> >> >> >> >>
