Hi Amos, You need to increase your limits (/etc/security/limits.conf) for max number of open files (nofile). Use a pretty big number (e.g. 500K) for both soft and hard.
Hope that helps. Dimitris On Tue, Nov 1, 2016 at 8:57 PM, Amos Bird <[email protected]> wrote: > > Hi there, > > After days of efforts to make impala's local tests work on my Centos > machine, I finally gave up and turns to Ubuntu. I followed this simple > guide > https://cwiki.apache.org/confluence/display/IMPALA/Bootstrapping+an+Impala+Development+Environment+From+Scratch > on a fresh installed Ubuntu 14.04. Unfortunately there are still errors > in loading data phase. Here is the error log, > > --------------------------------------------------------------------------------------------- > Loading Kudu TPCH (logging to > /home/amos/impala/logs/data_loading/load-kudu-tpch.log)... FAILED > 'load-data tpch core kudu/none/none force' failed. Tail of log: > distribute by hash (c_custkey) into 9 buckets stored as kudu > > (load-tpch-core-impala-generated-kudu-none-none.sql): > > > Executing HBase Command: hbase shell load-tpch-core-hbase-generated.create > 16/11/02 01:07:58 INFO Configuration.deprecation: hadoop.native.lib is > deprecated. Instead, use io.native.lib.available > SLF4J: Class path contains multiple SLF4J bindings. > SLF4J: Found binding in > [jar:file:/home/amos/impala/toolchain/cdh_components/hbase-1.2.0-cdh5.10.0-SNAPSHOT/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: Found binding in > [jar:file:/home/amos/impala/toolchain/cdh_components/hadoop-2.6.0-cdh5.10.0-SNAPSHOT/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation. > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] > Executing HBase Command: hbase shell post-load-tpch-core-hbase-generated.sql > 16/11/02 01:08:03 INFO Configuration.deprecation: hadoop.native.lib is > deprecated. Instead, use io.native.lib.available > SLF4J: Class path contains multiple SLF4J bindings. > SLF4J: Found binding in > [jar:file:/home/amos/impala/toolchain/cdh_components/hbase-1.2.0-cdh5.10.0-SNAPSHOT/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: Found binding in > [jar:file:/home/amos/impala/toolchain/cdh_components/hadoop-2.6.0-cdh5.10.0-SNAPSHOT/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation. > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] > Invalidating Metadata > (load-tpch-core-impala-load-generated-kudu-none-none.sql): > INSERT INTO TABLE tpch_kudu.lineitem SELECT * FROM tpch.lineitem > > Data Loading from Impala failed with error: ImpalaBeeswaxException: > Query aborted: > Kudu error(s) reported, first error: Timed out: Failed to write batch of 2708 > ops to tablet 84aa134fb6c24916aa16cf50f48ec557 after 329 attempt(s): Failed > to write to server: (no server available): Write(tablet: > 84aa134fb6c24916aa16cf50f48ec557, num_ops: 2708, num_attempts: 329) passed > its deadline: Network error: recv error: Connection reset by peer (error 104) > > > > Kudu error(s) reported, first error: Timed out: Failed to write batch of 2708 > ops to tablet 84aa134fb6c24916aa16cf50f48ec557 after 329 attempt(s): Failed > to write to server: (no server available): Write(tablet: > 84aa134fb6c24916aa16cf50f48ec557, num_ops: 2708, num_attempts: 329) passed > its deadline: Network error: recv error: Connection reset by peer (error 104) > Error in Kudu table 'impala::tpch_kudu.lineitem': Timed out: Failed to write > batch of 2708 ops to tablet 84aa134fb6c24916aa16cf50f48ec557 after 329 > attempt(s): Failed to write to server: (no server available): Write(tablet: > 84aa134fb6c24916aa16cf50f48ec557, num_ops: 2708, num_attempts: 329) passed > its deadline: Network error: recv error: Connection reset by peer (error 104) > (1 of 2708 similar) > > Traceback (most recent call last): > File "/home/amos/impala/bin/load-data.py", line 158, in > exec_impala_query_from_file > result = impala_client.execute(query) > File "/home/amos/impala/tests/beeswax/impala_beeswax.py", line 173, in > execute > handle = self.__execute_query(query_string.strip(), user=user) > File "/home/amos/impala/tests/beeswax/impala_beeswax.py", line 339, in > __execute_query > self.wait_for_completion(handle) > File "/home/amos/impala/tests/beeswax/impala_beeswax.py", line 359, in > wait_for_completion > raise ImpalaBeeswaxException("Query aborted:" + error_log, None) > ImpalaBeeswaxException: ImpalaBeeswaxException: > Query aborted: > Kudu error(s) reported, first error: Timed out: Failed to write batch of 2708 > ops to tablet 84aa134fb6c24916aa16cf50f48ec557 after 329 attempt(s): Failed > to write to server: (no server available): Write(tablet: > 84aa134fb6c24916aa16cf50f48ec557, num_ops: 2708, num_attempts: 329) passed > its deadline: Network error: recv error: Connection reset by peer (error 104) > > > > Kudu error(s) reported, first error: Timed out: Failed to write batch of 2708 > ops to tablet 84aa134fb6c24916aa16cf50f48ec557 after 329 attempt(s): Failed > to write to server: (no server available): Write(tablet: > 84aa134fb6c24916aa16cf50f48ec557, num_ops: 2708, num_attempts: 329) passed > its deadline: Network error: recv error: Connection reset by peer (error 104) > Error in Kudu table 'impala::tpch_kudu.lineitem': Timed out: Failed to write > batch of 2708 ops to tablet 84aa134fb6c24916aa16cf50f48ec557 after 329 > attempt(s): Failed to write to server: (no server available): Write(tablet: > 84aa134fb6c24916aa16cf50f48ec557, num_ops: 2708, num_attempts: 329) passed > its deadline: Network error: recv error: Connection reset by peer (error 104) > (1 of 2708 similar) > > Error in /home/amos/impala/testdata/bin/create-load-data.sh at line 45: while > [ -n "$*" ] > + cleanup > + rm -rf /tmp/tmp.hMzGwIcUo3 > --------------------------------------------------------------------------------------------- > > This kinda blocks my patch's rebasing. Any help is much appreciated! > > regards, > Amos
