Re: Impala - 5314
Hi Aayush, you might want to check logs/cluster/impalad.INFO for an error related to the impala service. Keep in mind though from the output you had in your gist, everything compiles and the local impala came up, what's failing is loading the initial test data which can be quite large at times. For example see the info here https://cwiki.apache.org/confluence/display/IMPALA/Building+Impala [image: build_impala.png] At this point you can run that last command to build and spin up the system for testing. -Shant On Tue, Jul 21, 2020 at 12:31 PM Aayush Bhan wrote: > Hi Quanlong, > I went through the 3 log files and it gave me 2 different errors. > > The 1st one, i.e. > > /home/impdev/Impala/logs/data_loading/sql/tpch/load-tpch-core-impala-generated-kudu-none-none.sql.log > > INNER EXCEPTION: > MESSAGE: [Errno 104] Connection reset by peer > > This error came up while - INSERT INTO TABLE tpch_kudu.lineitem SELECT * > FROM tpch.lineitem > > In the second and third log files, i.e. > > > /home/impdev/Impala/logs/data_loading/sql/tpcds/invalidate-tpcds-core-impala-generated.sql.log > and > > /home/impdev/Impala/logs/data_loading/sql/functional/invalidate-functional-query-exhaustive-impala-generated.sql.log > > INNER EXCEPTION: > MESSAGE: Could not connect to localhost:21000 > > Thanks, > Aayush > > On Mon, Jul 20, 2020 at 7:30 PM Quanlong Huang > wrote: > > > Hi Aayush, > > > > You need to look into some log files for more details. In the console > > output you shared, there are three log files mentioned about the > failures. > > > > One is the failure on loading tpch dataset: > > > > > Loading TPC-H data (logging to > > > /home/impdev/Impala/logs/data_loading/load-tpch.log)... > > > Loading TPC-DS data (logging to > > > /home/impdev/Impala/logs/data_loading/load-tpcds.log)... > > > FAILED (Took: 8 min 8 sec) > > > 'load-data tpch core' failed. Tail of log: > > > 'load-data tpch core' failed. Tail of log: > > > .. > > > 11:51:28 Error executing impala SQL: > > > > > > /home/impdev/Impala/logs/data_loading/sql/tpch/load-tpch-core-impala-generated-kudu-none-none.sql > > > See: > > > > > > /home/impdev/Impala/logs/data_loading/sql/tpch/load-tpch-core-impala-generated-kudu-none-none.sql.log > > > > You should look > > into > > > /home/impdev/Impala/logs/data_loading/sql/tpch/load-tpch-core-impala-generated-kudu-none-none.sql.log > > for more details. > > > > The other one is the failure on loading tpcds dataset: > > > > > FAILED (Took: 15 min 28 sec) > > > 'load-data tpcds core' failed. Tail of log: > > > .. > > > 11:58:47 Error executing impala SQL: > > > > > > /home/impdev/Impala/logs/data_loading/sql/tpcds/invalidate-tpcds-core-impala-generated.sql > > > See: > > > > > > /home/impdev/Impala/logs/data_loading/sql/tpcds/invalidate-tpcds-core-impala-generated.sql.log > > > > You should look > > into > > > /home/impdev/Impala/logs/data_loading/sql/tpcds/invalidate-tpcds-core-impala-generated.sql.log > > for more details. > > > > The last one is the failure on loading functional-query dataset: > > > > > FAILED (Took: 33 min 22 sec) > > > 'load-data functional-query exhaustive' failed. Tail of log: > > > .. > > > 12:16:42 Error executing impala SQL: > > > > > > /home/impdev/Impala/logs/data_loading/sql/functional/invalidate-functional-query-exhaustive-impala-generated.sql > > > See: > > > > > > /home/impdev/Impala/logs/data_loading/sql/functional/invalidate-functional-query-exhaustive-impala-generated.sql.log > > > > You should look > > into > > > /home/impdev/Impala/logs/data_loading/sql/functional/invalidate-functional-query-exhaustive-impala-generated.sql.log > > for more details. > > > > Some common reasons are not enough disk spaces or memory. If these log > > files containing failures on executing hive queries. You may need to look > > into hive logs for more details. They are in > > $IMPALA_HOME/logs/cluster/hive* > > > > Quanlong > > > > On Mon, Jul 20, 2020 at 6:22 PM Aayush Bhan > wrote: > > > > > Hey Jim, > > > I looked into what you said and there was just enough disk space on my > > > system. > > > So I figured I should clean it up a bit. I tried building again after > > > having sufficient > > > disk space. However, the log file seems to be giving me the same > problem > > as > > > before. > > > Is it possible there could be some other reason for encountering this? > > > > > > Thanks, > > > Aayush > > > > > > On Fri, Jul 17, 2020 at 8:10 PM Jim Apple wrote: > > > > > > > In my experience, a failure in the master branch during data load > > > indicated > > > > being out of disk space. This sometimes has manifested even when df > -h > > > > shows just enough space, so this was successful for me most often > when > > I > > > > had more than enough space. > > > > > > > > On Fri, Jul 17, 2020 at 7:06 AM Aayush Bhan > > > wrote: > > > > > > > > > Hi Impala Dev Team, > > > > > > > > > > I was assigned the ticket - 5314 on jira. As a first time > > contributor,
Re: Impala - 5314
Hi Quanlong, I went through the 3 log files and it gave me 2 different errors. The 1st one, i.e. /home/impdev/Impala/logs/data_loading/sql/tpch/load-tpch-core-impala-generated-kudu-none-none.sql.log INNER EXCEPTION: MESSAGE: [Errno 104] Connection reset by peer This error came up while - INSERT INTO TABLE tpch_kudu.lineitem SELECT * FROM tpch.lineitem In the second and third log files, i.e. /home/impdev/Impala/logs/data_loading/sql/tpcds/invalidate-tpcds-core-impala-generated.sql.log and /home/impdev/Impala/logs/data_loading/sql/functional/invalidate-functional-query-exhaustive-impala-generated.sql.log INNER EXCEPTION: MESSAGE: Could not connect to localhost:21000 Thanks, Aayush On Mon, Jul 20, 2020 at 7:30 PM Quanlong Huang wrote: > Hi Aayush, > > You need to look into some log files for more details. In the console > output you shared, there are three log files mentioned about the failures. > > One is the failure on loading tpch dataset: > > > Loading TPC-H data (logging to > > /home/impdev/Impala/logs/data_loading/load-tpch.log)... > > Loading TPC-DS data (logging to > > /home/impdev/Impala/logs/data_loading/load-tpcds.log)... > > FAILED (Took: 8 min 8 sec) > > 'load-data tpch core' failed. Tail of log: > > 'load-data tpch core' failed. Tail of log: > > .. > > 11:51:28 Error executing impala SQL: > > > /home/impdev/Impala/logs/data_loading/sql/tpch/load-tpch-core-impala-generated-kudu-none-none.sql > > See: > > > /home/impdev/Impala/logs/data_loading/sql/tpch/load-tpch-core-impala-generated-kudu-none-none.sql.log > > You should look > into > /home/impdev/Impala/logs/data_loading/sql/tpch/load-tpch-core-impala-generated-kudu-none-none.sql.log > for more details. > > The other one is the failure on loading tpcds dataset: > > > FAILED (Took: 15 min 28 sec) > > 'load-data tpcds core' failed. Tail of log: > > .. > > 11:58:47 Error executing impala SQL: > > > /home/impdev/Impala/logs/data_loading/sql/tpcds/invalidate-tpcds-core-impala-generated.sql > > See: > > > /home/impdev/Impala/logs/data_loading/sql/tpcds/invalidate-tpcds-core-impala-generated.sql.log > > You should look > into > /home/impdev/Impala/logs/data_loading/sql/tpcds/invalidate-tpcds-core-impala-generated.sql.log > for more details. > > The last one is the failure on loading functional-query dataset: > > > FAILED (Took: 33 min 22 sec) > > 'load-data functional-query exhaustive' failed. Tail of log: > > .. > > 12:16:42 Error executing impala SQL: > > > /home/impdev/Impala/logs/data_loading/sql/functional/invalidate-functional-query-exhaustive-impala-generated.sql > > See: > > > /home/impdev/Impala/logs/data_loading/sql/functional/invalidate-functional-query-exhaustive-impala-generated.sql.log > > You should look > into > /home/impdev/Impala/logs/data_loading/sql/functional/invalidate-functional-query-exhaustive-impala-generated.sql.log > for more details. > > Some common reasons are not enough disk spaces or memory. If these log > files containing failures on executing hive queries. You may need to look > into hive logs for more details. They are in > $IMPALA_HOME/logs/cluster/hive* > > Quanlong > > On Mon, Jul 20, 2020 at 6:22 PM Aayush Bhan wrote: > > > Hey Jim, > > I looked into what you said and there was just enough disk space on my > > system. > > So I figured I should clean it up a bit. I tried building again after > > having sufficient > > disk space. However, the log file seems to be giving me the same problem > as > > before. > > Is it possible there could be some other reason for encountering this? > > > > Thanks, > > Aayush > > > > On Fri, Jul 17, 2020 at 8:10 PM Jim Apple wrote: > > > > > In my experience, a failure in the master branch during data load > > indicated > > > being out of disk space. This sometimes has manifested even when df -h > > > shows just enough space, so this was successful for me most often when > I > > > had more than enough space. > > > > > > On Fri, Jul 17, 2020 at 7:06 AM Aayush Bhan > > wrote: > > > > > > > Hi Impala Dev Team, > > > > > > > > I was assigned the ticket - 5314 on jira. As a first time > contributor, > > I > > > > was going through the cwiki links on how to get started. So I was > > setting > > > > up my development environment in a docker container. I went ahead > with > > > all > > > > the steps that were given. So every time I try to run the > > > > bootstrap_development script, it leads to some build errors which i'm > > not > > > > able to ascertain how to fix. > > > > > > > > The log i'm being directed to after running the script tells me the > > > > following - > > > > Error in /home/impdev/Impala/testdata/bin/create-load-data.sh at line > > 85: > > > > fi > > > > > > > > I got the following output while running the bootstrap_development > > > script - > > > > https://gist.github.com/aayushbhan/dee15ba212225e780196e2964d279681 > > > > > > > > I would really appreciate it if anyone could tell me any resources or > > > >
Re: Impala - 5314
Hi Aayush, You need to look into some log files for more details. In the console output you shared, there are three log files mentioned about the failures. One is the failure on loading tpch dataset: > Loading TPC-H data (logging to > /home/impdev/Impala/logs/data_loading/load-tpch.log)... > Loading TPC-DS data (logging to > /home/impdev/Impala/logs/data_loading/load-tpcds.log)... > FAILED (Took: 8 min 8 sec) > 'load-data tpch core' failed. Tail of log: > 'load-data tpch core' failed. Tail of log: > .. > 11:51:28 Error executing impala SQL: > /home/impdev/Impala/logs/data_loading/sql/tpch/load-tpch-core-impala-generated-kudu-none-none.sql > See: > /home/impdev/Impala/logs/data_loading/sql/tpch/load-tpch-core-impala-generated-kudu-none-none.sql.log You should look into /home/impdev/Impala/logs/data_loading/sql/tpch/load-tpch-core-impala-generated-kudu-none-none.sql.log for more details. The other one is the failure on loading tpcds dataset: > FAILED (Took: 15 min 28 sec) > 'load-data tpcds core' failed. Tail of log: > .. > 11:58:47 Error executing impala SQL: > /home/impdev/Impala/logs/data_loading/sql/tpcds/invalidate-tpcds-core-impala-generated.sql > See: > /home/impdev/Impala/logs/data_loading/sql/tpcds/invalidate-tpcds-core-impala-generated.sql.log You should look into /home/impdev/Impala/logs/data_loading/sql/tpcds/invalidate-tpcds-core-impala-generated.sql.log for more details. The last one is the failure on loading functional-query dataset: > FAILED (Took: 33 min 22 sec) > 'load-data functional-query exhaustive' failed. Tail of log: > .. > 12:16:42 Error executing impala SQL: > /home/impdev/Impala/logs/data_loading/sql/functional/invalidate-functional-query-exhaustive-impala-generated.sql > See: > /home/impdev/Impala/logs/data_loading/sql/functional/invalidate-functional-query-exhaustive-impala-generated.sql.log You should look into /home/impdev/Impala/logs/data_loading/sql/functional/invalidate-functional-query-exhaustive-impala-generated.sql.log for more details. Some common reasons are not enough disk spaces or memory. If these log files containing failures on executing hive queries. You may need to look into hive logs for more details. They are in $IMPALA_HOME/logs/cluster/hive* Quanlong On Mon, Jul 20, 2020 at 6:22 PM Aayush Bhan wrote: > Hey Jim, > I looked into what you said and there was just enough disk space on my > system. > So I figured I should clean it up a bit. I tried building again after > having sufficient > disk space. However, the log file seems to be giving me the same problem as > before. > Is it possible there could be some other reason for encountering this? > > Thanks, > Aayush > > On Fri, Jul 17, 2020 at 8:10 PM Jim Apple wrote: > > > In my experience, a failure in the master branch during data load > indicated > > being out of disk space. This sometimes has manifested even when df -h > > shows just enough space, so this was successful for me most often when I > > had more than enough space. > > > > On Fri, Jul 17, 2020 at 7:06 AM Aayush Bhan > wrote: > > > > > Hi Impala Dev Team, > > > > > > I was assigned the ticket - 5314 on jira. As a first time contributor, > I > > > was going through the cwiki links on how to get started. So I was > setting > > > up my development environment in a docker container. I went ahead with > > all > > > the steps that were given. So every time I try to run the > > > bootstrap_development script, it leads to some build errors which i'm > not > > > able to ascertain how to fix. > > > > > > The log i'm being directed to after running the script tells me the > > > following - > > > Error in /home/impdev/Impala/testdata/bin/create-load-data.sh at line > 85: > > > fi > > > > > > I got the following output while running the bootstrap_development > > script - > > > https://gist.github.com/aayushbhan/dee15ba212225e780196e2964d279681 > > > > > > I would really appreciate it if anyone could tell me any resources or > > > anything of the sort to help me fix this issue and get my development > > > environment set up. > > > > > > Thanks, > > > Aayush > > > > > > > > -- > aayushbhan >
Re: Impala - 5314
Hey Jim, I looked into what you said and there was just enough disk space on my system. So I figured I should clean it up a bit. I tried building again after having sufficient disk space. However, the log file seems to be giving me the same problem as before. Is it possible there could be some other reason for encountering this? Thanks, Aayush On Fri, Jul 17, 2020 at 8:10 PM Jim Apple wrote: > In my experience, a failure in the master branch during data load indicated > being out of disk space. This sometimes has manifested even when df -h > shows just enough space, so this was successful for me most often when I > had more than enough space. > > On Fri, Jul 17, 2020 at 7:06 AM Aayush Bhan wrote: > > > Hi Impala Dev Team, > > > > I was assigned the ticket - 5314 on jira. As a first time contributor, I > > was going through the cwiki links on how to get started. So I was setting > > up my development environment in a docker container. I went ahead with > all > > the steps that were given. So every time I try to run the > > bootstrap_development script, it leads to some build errors which i'm not > > able to ascertain how to fix. > > > > The log i'm being directed to after running the script tells me the > > following - > > Error in /home/impdev/Impala/testdata/bin/create-load-data.sh at line 85: > > fi > > > > I got the following output while running the bootstrap_development > script - > > https://gist.github.com/aayushbhan/dee15ba212225e780196e2964d279681 > > > > I would really appreciate it if anyone could tell me any resources or > > anything of the sort to help me fix this issue and get my development > > environment set up. > > > > Thanks, > > Aayush > > > -- aayushbhan
Re: Impala - 5314
In my experience, a failure in the master branch during data load indicated being out of disk space. This sometimes has manifested even when df -h shows just enough space, so this was successful for me most often when I had more than enough space. On Fri, Jul 17, 2020 at 7:06 AM Aayush Bhan wrote: > Hi Impala Dev Team, > > I was assigned the ticket - 5314 on jira. As a first time contributor, I > was going through the cwiki links on how to get started. So I was setting > up my development environment in a docker container. I went ahead with all > the steps that were given. So every time I try to run the > bootstrap_development script, it leads to some build errors which i'm not > able to ascertain how to fix. > > The log i'm being directed to after running the script tells me the > following - > Error in /home/impdev/Impala/testdata/bin/create-load-data.sh at line 85: > fi > > I got the following output while running the bootstrap_development script - > https://gist.github.com/aayushbhan/dee15ba212225e780196e2964d279681 > > I would really appreciate it if anyone could tell me any resources or > anything of the sort to help me fix this issue and get my development > environment set up. > > Thanks, > Aayush >