I was unable to reproduce this in a fresh ec2 instance. I ran:

sudo docker run --interactive --privileged --tty ubuntu:16.04
# inside the container
apt-get update
apt-get install -y sudo
adduser --disabled-password --gecos '' impdev
echo 'impdev ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
sudo su - impdev
# as impdev
sudo apt install -y git
git clone https://git-wip-us.apache.org/repos/asf/impala.git ~/Impala


time ~/Impala/bin/bootstrap_development.sh

On Sat, Apr 7, 2018 at 6:44 PM, Jim Apple <jbap...@cloudera.com> wrote:

> The attachment didn't attach. This is a common issue on our mailing lists.
> I've started another thread about it.
>
> I'll try to reproduce this. My first suspicions are disk space, networking
> restrictions, and https://issues.apache.org/jira/browse/IMPALA-6737
>
> Inside of the container, can you run df -h before exiting?
>
> On Sat, Apr 7, 2018 at 5:28 PM, Joseph Koshakow <kosh...@gmail.com> wrote:
>
>> Hi,
>>
>> Tried this again recently and still experiencing the same error. I've
>> attached a copy of all my shell commands and output.
>>
>> On Sat, Feb 24, 2018 at 6:34 PM, Joseph Koshakow <kosh...@gmail.com>
>> wrote:
>>
>>> Though I just upgraded to the most recent version. It's still failing
>>> but seems to be making it one step farther. I see from the emails that
>>> there's some issues with the current build so I'll just wait until that's
>>> all resolved. The output from my most recent attempt is below if you have a
>>> use for it.
>>>
>>> Loading custom schemas OK (Took: 0 min 40 sec)
>>> Started Loading functional-query data in background; pid 22229.
>>> Loading functional-query data (logging to /home/impdev/Impala/logs/data_
>>> loading/load-functional-query.log)...
>>> Started Loading TPC-H data in background; pid 22230.
>>> Started Loading TPC-DS data in background; pid 22231.
>>> Loading TPC-DS data (logging to /home/impdev/Impala/logs/data_
>>> loading/load-tpcds.log)...
>>> Loading TPC-H data (logging to /home/impdev/Impala/logs/data_
>>> loading/load-tpch.log)...
>>>     FAILED (Took: 25 min 51 sec)
>>>     'load-data tpch core' failed. Tail of log:
>>> INVALIDATE METADATA tpch_kudu.region
>>>
>>> (invalidate-tpch-core-impala-generated.sql):
>>> INVALIDATE METADATA tpch_kudu.orders
>>>
>>> (invalidate-tpch-core-impala-generated.sql):
>>> INVALIDATE METADATA tpch_kudu.customer
>>>
>>> (invalidate-tpch-core-impala-generated.sql):
>>>
>>>
>>> (load-tpch-core-impala-generated-kudu-none-none.sql):
>>> INSERT INTO TABLE tpch_kudu.lineitem SELECT * FROM tpch.lineitem
>>>
>>> (load-tpch-core-impala-generated-parquet-none-none.sql):
>>> INSERT OVERWRITE TABLE tpch_parquet.lineitem SELECT * FROM tpch.lineitem
>>>
>>> (load-tpch-core-impala-generated-parquet-none-none.sql):
>>> INSERT OVERWRITE TABLE tpch_parquet.part SELECT * FROM tpch.part
>>>
>>> (load-tpch-core-impala-generated-parquet-none-none.sql):
>>> INSERT OVERWRITE TABLE tpch_parquet.partsupp SELECT * FROM tpch.partsupp
>>>
>>> (load-tpch-core-impala-generated-parquet-none-none.sql):
>>> INSERT OVERWRITE TABLE tpch_parquet.supplier SELECT * FROM tpch.supplier
>>>
>>> Data Loading from Impala failed with error: ImpalaBeeswaxException:
>>>  Query aborted:Kudu error(s) reported, first error: Timed out: Failed to
>>> write batch of 21493 ops to tablet 5c5c917295d642edb9cdbe152f3283d9
>>> after 4 attempt(s): GetTableLocations { table:
>>> 'impala::tpch_kudu.lineitem', partition-key: (<start>), attempt: 1 }
>>> failed: Client connection negotiation failed: client connection to
>>> 127.0.0.1:7051
>>>
>>> Error in Kudu table 'impala::tpch_kudu.lineitem': Timed out: Failed to
>>> write batch of 21537 ops to tablet e9f68f0ee8e945ebb7630947169a6c34
>>> after 4 attempt(s): GetTableLocations { table:
>>> 'impala::tpch_kudu.lineitem', partition-key: (HASH (l_orderkey): 2),
>>> attempt: 1 } failed: Client connection negotiation failed: client
>>> connection to 127.0.0.1:7051 (1 of 21494 similar)
>>>
>>>
>>> Traceback (most recent call last):
>>>   File "/home/impdev/Impala/bin/load-data.py", line 179, in
>>> exec_impala_query_from_file
>>> (load-tpch-core-impala-generated-parquet-none-none.sql):
>>> INSERT OVERWRITE TABLE tpch_parquet.nation SELECT * FROM tpch.nation
>>>
>>>     result = impala_client.execute(query)
>>>   File "/home/impdev/Impala/tests/beeswax/impala_beeswax.py", line 173,
>>> in execute
>>>     handle = self.__execute_query(query_string.strip(), user=user)
>>>   File "/home/impdev/Impala/tests/beeswax/impala_beeswax.py", line 341,
>>> in __execute_query
>>>     self.wait_for_completion(handle)
>>>   File "/home/impdev/Impala/tests/beeswax/impala_beeswax.py", line 361,
>>> in wait_for_completion
>>>     raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
>>> ImpalaBeeswaxException: ImpalaBeeswaxException:
>>>  Query aborted:Kudu error(s) reported, first error: Timed out: Failed to
>>> write batch of 21493 ops to tablet 5c5c917295d642edb9cdbe152f3283d9
>>> after 4 attempt(s): GetTableLocations { table:
>>> 'impala::tpch_kudu.lineitem', partition-key: (<start>), attempt: 1 }
>>> failed: Client connection negotiation failed: client connection to
>>> 127.0.0.1:7051
>>>
>>> Error in Kudu table 'impala::tpch_kudu.lineitem': Timed out: Failed to
>>> write batch of 21537 ops to tablet e9f68f0ee8e945ebb7630947169a6c34
>>> after 4 attempt(s): GetTableLocations { table:
>>> 'impala::tpch_kudu.lineitem', partition-key: (HASH (l_orderkey): 2),
>>> attempt: 1 } failed: Client connection negotiation failed: client
>>> connection to 127.0.0.1:7051 (1 of 21494 similar)
>>>
>>>
>>> On Fri, Feb 23, 2018 at 6:34 PM, Joseph Koshakow <kosh...@gmail.com>
>>> wrote:
>>>
>>>> 8302608bdff15d6e8af324fc885dd0a8c2b44a5d
>>>>
>>>> On Fri, Feb 23, 2018 at 12:41 AM, Jim Apple <jbap...@cloudera.com>
>>>> wrote:
>>>>
>>>>> What commit hash is checked out?
>>>>>
>>>>> On Thu, Feb 22, 2018 at 8:23 PM Joseph Koshakow <kosh...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> > Host OS is Ubunto 16.04.3 LTS. Docker OS is also Ubuntu 16.04 as per
>>>>> > instructions. I have 202.61Gb of free space in the host environment
>>>>> > (304.58Gb size total). I'm not sure how to check how much space is
>>>>> left in
>>>>> > docker containers but the size of the one created for this is 97.81
>>>>> Gb.
>>>>> >
>>>>> > -Joe
>>>>> >
>>>>> > On Thu, Feb 22, 2018 at 11:15 PM, Jim Apple <jbap...@cloudera.com>
>>>>> wrote:
>>>>> >
>>>>> > > What is the host OS? What is the docker OS? How much space is left
>>>>> on
>>>>> > disk
>>>>> > > in each environment?
>>>>> > >
>>>>> > > On Thu, Feb 22, 2018 at 8:05 PM, Joseph Koshakow <
>>>>> kosh...@gmail.com>
>>>>> > > wrote:
>>>>> > >
>>>>> > > > Hi,
>>>>> > > >
>>>>> > > > I've been trying to set up my development environment using
>>>>> docker by
>>>>> > > > following the following instructions: https://cwiki.ap
>>>>> > > > ache.org/confluence/display/IMPALA/Docker+for+Impala+Developers
>>>>> > > > I'm able to get to the step where you run `source
>>>>> > > > ~/Impala/bin/bootsrap_development.sh`. Everything runs fine
>>>>> until I get
>>>>> > > > to this part:
>>>>> > > > Loading TPC-DS data (logging to /home/impdev/Impala/logs/data_
>>>>> > > > loading/load-tpcds.log)...
>>>>> > > >
>>>>> > > > That hangs for about 30 minutes and then prints the following
>>>>> (Also
>>>>> > > > attached) before hanging again indefinitely until I quit out.
>>>>> I've
>>>>> > tried
>>>>> > > > multiple times and even deleted the docker image and started
>>>>> over. Any
>>>>> > > > ideas?
>>>>> > > >
>>>>> > > > FAILED (Took: 26 min 21 sec)
>>>>> > > >     'load-data tpch core' failed. Tail of log:
>>>>> > > > INVALIDATE METADATA tpch_kudu.nation
>>>>> > > >
>>>>> > > > (invalidate-tpch-core-impala-generated.sql):
>>>>> > > > INVALIDATE METADATA tpch_kudu.region
>>>>> > > >
>>>>> > > > (invalidate-tpch-core-impala-generated.sql):
>>>>> > > > INVALIDATE METADATA tpch_kudu.orders
>>>>> > > >
>>>>> > > > (invalidate-tpch-core-impala-generated.sql):
>>>>> > > > INVALIDATE METADATA tpch_kudu.customer
>>>>> > > >
>>>>> > > > (invalidate-tpch-core-impala-generated.sql):
>>>>> > > >
>>>>> > > >
>>>>> > > > (load-tpch-core-impala-generated-parquet-none-none.sql):
>>>>> > > > INSERT OVERWRITE TABLE tpch_parquet.lineitem SELECT * FROM
>>>>> > tpch.lineitem
>>>>> > > >
>>>>> > > > (load-tpch-core-impala-generated-kudu-none-none.sql):
>>>>> > > > INSERT INTO TABLE tpch_kudu.lineitem SELECT * FROM tpch.lineitem
>>>>> > > >
>>>>> > > > (load-tpch-core-impala-generated-parquet-none-none.sql):
>>>>> > > > INSERT OVERWRITE TABLE tpch_parquet.part SELECT * FROM tpch.part
>>>>> > > >
>>>>> > > > (load-tpch-core-impala-generated-parquet-none-none.sql):
>>>>> > > > INSERT OVERWRITE TABLE tpch_parquet.partsupp SELECT * FROM
>>>>> > tpch.partsupp
>>>>> > > >
>>>>> > > > Data Loading from Impala failed with error:
>>>>> ImpalaBeeswaxException:
>>>>> > > >  Query aborted:Kudu error(s) reported, first error: Timed out:
>>>>> Failed
>>>>> > to
>>>>> > > > write batch of 23532 ops to tablet 3b2c6a27489d43028912a69ca7be8b
>>>>> 71
>>>>> > > after
>>>>> > > > 4 attempt(s): GetTableLocations { table:
>>>>> 'impala::tpch_kudu.lineitem',
>>>>> > > > partition-key: (HASH (l_orderkey): 1), attempt: 1 } failed:
>>>>> > > > GetTableLocations RPC to 127.0.0.1:7051 timed out after 10.000s
>>>>> (SENT)
>>>>> > > >
>>>>> > > > Error in Kudu table 'impala::tpch_kudu.lineitem': Timed out:
>>>>> Failed to
>>>>> > > > write batch of 23532 ops to tablet 3b2c6a27489d43028912a69ca7be8b
>>>>> 71
>>>>> > > after
>>>>> > > > 4 attempt(s): GetTableLocations { table:
>>>>> 'impala::tpch_kudu.lineitem',
>>>>> > > > partition-key: (HASH (l_orderkey): 1), attempt: 1 } failed:
>>>>> > > > GetTableLocations RPC to 127.0.0.1:7051 timed out after 10.000s
>>>>> (SENT)
>>>>> > > (1
>>>>> > > > of 23532 similar)
>>>>> > > >
>>>>> > > >
>>>>> > > > Traceback (most recent call last):
>>>>> > > > (load-tpch-core-impala-generated-parquet-none-none.sql):
>>>>> > > > INSERT OVERWRITE TABLE tpch_parquet.supplier SELECT * FROM
>>>>> > tpch.supplier
>>>>> > > >   File "/home/impdev/Impala/bin/load-data.py", line 179, in
>>>>> > > > exec_impala_query_from_file
>>>>> > > >
>>>>> > > >     result = impala_client.execute(query)
>>>>> > > >   File "/home/impdev/Impala/tests/beeswax/impala_beeswax.py",
>>>>> line 173,
>>>>> > > > in execute
>>>>> > > >     handle = self.__execute_query(query_string.strip(),
>>>>> user=user)
>>>>> > > >   File "/home/impdev/Impala/tests/beeswax/impala_beeswax.py",
>>>>> line 341,
>>>>> > > > in __execute_query
>>>>> > > >     self.wait_for_completion(handle)
>>>>> > > >   File "/home/impdev/Impala/tests/beeswax/impala_beeswax.py",
>>>>> line 361,
>>>>> > > > in wait_for_completion
>>>>> > > >     raise ImpalaBeeswaxException("Query aborted:" + error_log,
>>>>> None)
>>>>> > > > ImpalaBeeswaxException: ImpalaBeeswaxException:
>>>>> > > >  Query aborted:Kudu error(s) reported, first error: Timed out:
>>>>> Failed
>>>>> > to
>>>>> > > > write batch of 23532 ops to tablet 3b2c6a27489d43028912a69ca7be8b
>>>>> 71
>>>>> > > after
>>>>> > > > 4 attempt(s): GetTableLocations { table:
>>>>> 'impala::tpch_kudu.lineitem',
>>>>> > > > partition-key: (HASH (l_orderkey): 1), attempt: 1 } failed:
>>>>> > > > GetTableLocations RPC to 127.0.0.1:7051 timed out after 10.000s
>>>>> (SENT)
>>>>> > > >
>>>>> > > > Error in Kudu table 'impala::tpch_kudu.lineitem': Timed out:
>>>>> Failed to
>>>>> > > > write batch of 23532 ops to tablet 3b2c6a27489d43028912a69ca7be8b
>>>>> 71
>>>>> > > after
>>>>> > > > 4 attempt(s): GetTableLocations { table:
>>>>> 'impala::tpch_kudu.lineitem',
>>>>> > > > partition-key: (HASH (l_orderkey): 1), attempt: 1 } failed:
>>>>> > > > GetTableLocations RPC to 127.0.0.1:7051 timed out after 10.000s
>>>>> (SENT)
>>>>> > > (1
>>>>> > > > of 23532 similar)
>>>>> > > >
>>>>> > > > Thanks,
>>>>> > > > Joe Koshakow
>>>>> > > >
>>>>> > > >
>>>>> > >
>>>>> >
>>>>>
>>>>
>>>>
>>>
>>
>

Reply via email to