RE: [kudu] import from hdfs

2017-08-17 Thread Andrey Kuznetsov
and destroy any printed copies. From: Jean-Daniel Cryans [mailto:jdcry...@apache.org] Sent: Wednesday, August 16, 2017 9:39 PM To: user@kudu.apache.org Cc: Special SBER-BPOC Team Subject: Re: [kudu] import from hdfs Huh this is confusing, how much memory did you say you have per node? You mention

Re: [kudu] import from hdfs

2017-08-16 Thread Jean-Daniel Cryans
*From:* Jean-Daniel Cryans [mailto:jdcry...@apache.org] > *Sent:* Thursday, August 10, 2017 6:55 PM > > *To:* user@kudu.apache.org > *Cc:* Special SBER-BPOC Team > *Subject:* Re: [kudu] import from hdfs > > > > Hi Andrey, > > > > Can you double check how muc

Re: [kudu] import from hdfs

2017-08-10 Thread Jean-Daniel Cryans
reby notified that any dissemination, distribution or > copying of this communication is strictly prohibited. All unintended > recipients are obliged to delete this message and destroy any printed > copies. > > > > *From:* Jean-Daniel Cryans [mailto:jdcry...@apache.org] > *

RE: [kudu] import from hdfs

2017-08-10 Thread Andrey Kuznetsov
Subject: Re: [kudu] import from hdfs Hi Andrey, Which version of Kudu and Impala are you using? Just that can make a huge difference. Apart from that, make sure Kudu has enough memory (no memory back pressure), you have enough maintenance manager threads (1/3 or 1/4 the number of disks), a

Re: [kudu] import from hdfs

2017-08-09 Thread Jean-Daniel Cryans
Hi Andrey, Which version of Kudu and Impala are you using? Just that can make a huge difference. Apart from that, make sure Kudu has enough memory (no memory back pressure), you have enough maintenance manager threads (1/3 or 1/4 the number of disks), and that your partitioning favors good load d

[kudu] import from hdfs

2017-08-09 Thread Andrey Kuznetsov
Hi folk, I have a problem with hdfs to kudu performance, I have created external table with CSV data and ran "insert as select" from it to kudu-table and to parquet-table: Importing to parquet-table is 3x faster than to kudu - do you know some tips/tricks to increase performance of import? actu