Hello guys, I am so relieved to announce you that finally the functions sum () and count (*) function work now. So it must either be due to the more recent version of Hadoop, or compile it into a 64x Platform finally. As I have implemented your tips and what I've done thus far?
I have cloned the current branch from hadoop and recompiled the new binaries for 64x Platform. So I have made a version jump from 2.4.0 to 3.0.0-SNAPSHOT. The compile was still peppered with a few small pitfalls, because initially maven needs more memory. So i had to be assigned the variable MAVEN_OPTS = Xmx4069m After 30 minutes to compile it was successfully build. Thank you so much for your purposeful tips and your help. Kind regards and a very happy Chris Am 27.07.2014 um 21:59 schrieb Christian Schwabe <[email protected]>: > Hello Hyunsik, > > Please excuse my incomplete answer. I forgot to mention that also the > recompile and leads to the already mentioned > java.lang.ArrayIndexOutOfBoundsException. > > Kind regards, > Christian > > Am 27.07.2014 um 21:46 schrieb Christian Schwabe <[email protected]>: > >> Hello Hyunsik, today I have started the trial to recompile with following >> 'mvn clean package -DskipTests -Pdist -Dtar -Dhadoop.version=2.4.0' >> statement. I regret to inform you that the recompile not brought the desired >> success. >> >> I use Hadoop in version 2.4.0. >> >> This is my current JVM version: >> java version "1.8.0_05" >> Java(TM) SE Runtime Environment (build 1.8.0_05-b13) >> Java HotSpot(TM) 64-Bit Server VM (build 25.5-b02, mixed mode) >> >> My Tajo environment (tajo-env.sh) has following content: >> >> # Licensed to the Apache Software Foundation (ASF) under one >> # or more contributor license agreements. See the NOTICE file >> # distributed with this work for additional information >> # regarding copyright ownership. The ASF licenses this file >> # to you under the Apache License, Version 2.0 (the >> # "License"); you may not use this file except in compliance >> # with the License. You may obtain a copy of the License at >> # >> # http://www.apache.org/licenses/LICENSE-2.0 >> # >> # Unless required by applicable law or agreed to in writing, software >> # distributed under the License is distributed on an "AS IS" BASIS, >> # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. >> # See the License for the specific language governing permissions and >> # limitations under the License. >> >> # Set Tajo-specific environment variables here. >> >> # The only required environment variable is JAVA_HOME. All others are >> # optional. When running a distributed configuration it is best to >> # set JAVA_HOME in this file, so that it is correctly defined on >> # remote nodes. >> >> # ————————————————————————————— >> # Hadoop home. Required >> #export HADOOP_HOME=/home/christian/6_Semester/Thesis/02_Hadoop/hadoop-2.4.0 >> >> #Mac OS X Konfiguration >> export HADOOP_HOME=/Users/chris/6_Semester/Thesis/02_Hadoop/hadoop-2.4.0 >> # ————————————————————————————— >> >> # The java implementation to use. Required. >> # export JAVA_HOME=/usr/java/default >> #export JAVA_HOME=/usr >> #Mac OS X Konfiguration >> export >> JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home >> >> # Extra Java CLASSPATH elements. Optional. >> # export TAJO_CLASSPATH=/xxx/extlib/*:/xxx/xxx.jar >> >> # The maximum amount of heap to use, in MB. Default is 1000. >> # export TAJO_MASTER_HEAPSIZE=1000 >> >> # The maximum amount of heap to use, in MB. Default is 1000. >> # export TAJO_WORKER_HEAPSIZE=1000 >> >> # The maximum amount of heap to use, in MB. Default is 1000. >> # export TAJO_QUERYMASTER_HEAPSIZE=1000 >> >> # Extra Java runtime options. Empty by default. >> # export TAJO_OPTS=-server >> >> # Extra TajoMaster's java runtime options for TajoMaster. Empty by default >> # export TAJO_MASTER_OPTS= >> >> # Extra TajoWorker's java runtime options for TajoMaster. Empty by default >> # export TAJO_WORKER_OPTS= >> >> # Extra QueryMaster mode TajoWorker's java runtime options for TajoMaster. >> Empty by default >> # export TAJO_QUERYMASTER_OPTS= >> >> # Where log files are stored. $TAJO_HOME/logs by default. >> # export TAJO_LOG_DIR=${TAJO_HOME}/logs >> >> # The directory where pid files are stored. /tmp by default. >> # export TAJO_PID_DIR=/var/tajo/pids >> >> # A string representing this instance of tajo. $USER by default. >> # export TAJO_IDENT_STRING=$USER >> >> # The scheduling priority for daemon processes. See 'man nice'. >> # export TAJO_NICENESS=10 >> >> # Tajo cluster mode. the default mode is standby mode. >> export TAJO_WORKER_STANDBY_MODE=true >> >> # It must be required to use HCatalogStore >> # export HIVE_HOME= >> # export HIVE_JDBC_DRIVER_DIR= >> >> Findest du da einen Fehler in der Konfiguration oder hast du noch einen Tip >> was genau falsch sein könnte? Ich kann in der nächsten Woche leider erst >> einmal nicht antworten, werde mich sobald ich wieder da bin wieder melden. >> >> Am 27.07.2014 um 06:06 schrieb Hyunsik Choi <[email protected]>: >> >>> Hi Christian, >>> >>> In order to clarify your problem, could you share your environments, >>> such as Hadoop version and JVM version? For some hadoop version, you >>> need to give '-Dhadoop.version=2.4.0' to maven build command. >>> Otherwise, incorrect class sources may be built for your binary dist. >>> >>> Best regards, >>> Hyunsik >>> >>> On Sat, Jul 26, 2014 at 6:11 AM, Christian Schwabe >>> <[email protected]> wrote: >>>> Hello Hyunsik, >>>> >>>> thanks for your quick response. >>>> I clone the actual github repository and compile the Version by my own. >>>> So actual I use the version 0.9.0-Snapshot. >>>> After your answer it seems als the bugs exist in the current version? Can >>>> you confirm this? >>>> >>>> Kind regards, >>>> Chris >>>> >>>> On 25. Juli 2014 18:39:37 MESZ, Hyunsik Choi <[email protected]> wrote: >>>>> >>>>> Hi Christian, >>>>> >>>>> Above all, thank you for your interest in Tajo. >>>>> >>>>> The problem seems to be an old bug. >>>>> Are you using Tajo 0.9.0? Did you get Tajo via git? >>>>> >>>>> Best regards, >>>>> Hyunsik >>>>> >>>>> On Fri, Jul 25, 2014 at 9:23 PM, Christian Schwabe >>>>> <[email protected]> wrote: >>>>>> >>>>>> Hello guys, >>>>>> >>>>>> this is the first time for me to write to an email list. Sorry for any >>>>>> wrong >>>>>> formatting. >>>>>> I am a student of computer science and use the framework Apache Tajo as >>>>>> part >>>>>> of my thesis. However, I have initial teething problems and >>>>>> unfortunately do >>>>>> not know how to remedy them. >>>>>> >>>>>> First, I want to describe what steps do I do to start Tajo: >>>>>> 1) Start Tajo with tajo-start.sh >>>>>> 2) Create a new Table >>>>>> 3) Start bash tsql >>>>>> >>>>>> Here details to my table contents which is composed from a csv file: >>>>>> >>>>>> default> SELECT * from table1; >>>>>> id, name, score, type >>>>>> ________________________________ >>>>>> >>>>>> 1, abc, 1.1, a >>>>>> 2, def, 2.3, b >>>>>> 3, ghi, 3.4, c >>>>>> 4, jkl, 4.5, d >>>>>> 5, mno, 5.6, e >>>>>> (5 rows, 0.275 sec, 59 B selected) >>>>>> >>>>>> Everything seems to work up here. Also substr () or concat () function. >>>>>> Do I >>>>>> use now but the count (*) or sum () I get the following exception: >>>>>> >>>>>> default> SELECT count(*) FROM table1; >>>>>> ERROR: 1 >>>>>> java.lang.ArrayIndexOutOfBoundsException: 1 >>>>>> at org.apache.tajo.util.BitArray.fromByteBuffer(BitArray.java:84) >>>>>> at org.apache.tajo.storage.RawFile$RawFileScanner.next(RawFile.java:243) >>>>>> at >>>>>> >>>>>> org.apache.tajo.engine.planner.physical.SeqScanExec.next(SeqScanExec.java:254) >>>>>> at >>>>>> >>>>>> org.apache.tajo.engine.planner.physical.HashAggregateExec.compute(HashAggregateExec.java:51) >>>>>> at >>>>>> >>>>>> org.apache.tajo.engine.planner.physical.HashAggregateExec.next(HashAggregateExec.java:87) >>>>>> at >>>>>> >>>>>> org.apache.tajo.engine.planner.physical.StoreTableExec.next(StoreTableExec.java:77) >>>>>> at org.apache.tajo.worker.Task.run(Task.java:425) >>>>>> at org.apache.tajo.worker.TaskRunner$1.run(TaskRunner.java:406) >>>>>> at java.lang.Thread.run(Thread.java:744) >>>>>> >>>>>> >>>>>> Can you tell me what I'm doing wrong at this point that I have neither >>>>>> the >>>>>> count (*) still can use sum ()? >>>>>> Is this a known problem? Is it fixed? When will it be fixed? How can I >>>>>> fix >>>>>> it myself? I suppose that I have a wrong configuration, although I have >>>>>> not >>>>>> had much konfikuriert. Because I can hardly imagine that such trivial >>>>>> functionalities not in my used version 0.9.0 are available. >>>>>> >>>>>> Kind regards, >>>>>> Christian Schwabe >> >
