Install kudu-1.6 in Ubuntu-14.04 via apt-get

2018-09-03 Thread Quanlong Huang
Hi all, We have a kudu cluster in version 1.6.0-cdh5.14.2. It's not managed by Cloudera Manager. When we want to scale out the cluster (following the doc: https://kudu.apache.org/releases/1.6.0/docs/installation.html, install by apt-get), we found that the newly installed nodes are in version

Re:Re: Re: Recommended maximum amount of stored data per tablet server

2018-08-03 Thread Quanlong Huang
Thank you, Todd! I'm really appreciated to your help! At 2018-08-03 08:36:39,"Todd Lipcon" wrote: On Thu, Aug 2, 2018 at 4:54 PM, Quanlong Huang wrote: Thank Adar and Todd! We'd like to contribute when we could. Are there any concerns if we share the machines with HDFS DataNode

Re:Re: Recommended maximum amount of stored data per tablet server

2018-08-02 Thread Quanlong Huang
this goes back to what I led with: beyond the recommended limit we aren't quite sure how Kudu's performance and stability are affected. All that said, you're welcome to try it out and report back with your findings. On Thu, Aug 2, 2018 at 7:23 AM Quanlong Huang wrote: > > Hi all, > >

Recommended maximum amount of stored data per tablet server

2018-08-02 Thread Quanlong Huang
Hi all, In the document of "Known Issues and Limitations", it's recommended that "maximum amount of stored data, post-replication and post-compression, per tablet server is 8TB". How is the 8TB calculated? We have some machines each with 15 * 4TB spinning disk drives and 256GB RAM, 48 cpu

Re:Re: Re: Re: Why RowSet size is much smaller than flush_threshold_mb

2018-08-02 Thread Quanlong Huang
they can find themselves. However, it's not a problem since you guys are quite helpful :) Thanks, Quanlong At 2018-08-02 10:18:00,"Todd Lipcon" wrote: On Wed, Aug 1, 2018 at 4:52 PM, Quanlong Huang wrote: In my experience, when I found the performance is below my expectation, I'd li

Re:Re: Re: Why RowSet size is much smaller than flush_threshold_mb

2018-08-01 Thread Quanlong Huang
ipcon" wrote: On Wed, Aug 1, 2018 at 6:28 AM, Quanlong Huang wrote: Hi Todd and William, I'm really appreciated for your help and sorry for my late reply. I was going to reply with some follow-up questions but was assigned to focus some other works... Now I'm back to this work. The d

Re:Re: Why RowSet size is much smaller than flush_threshold_mb

2018-08-01 Thread Quanlong Huang
original rowsets). The overall compaction time is kinda long but not crazy long. What's the performance you are seeing and what is the performance you would like to see? -Will On Fri, Jun 15, 2018 at 7:52 AM, Quanlong Huang wrote: Hi all, I'm running kudu 1.6.0-cdh5.14.2. When looking into the logs

Why RowSet size is much smaller than flush_threshold_mb

2018-06-15 Thread Quanlong Huang
Hi all, I'm running kudu 1.6.0-cdh5.14.2. When looking into the logs of tablet server, I find most of the compactions are compacting small files (~40MB for each). For example: I0615 07:22:42.63735130614tablet.cc:1661] T 6bdefb8c27764a0597dcf98ee1b450ba P 70f3e54fe0f3490cbf0371a6830a33a7:

How to install the latest kudu release and find the compatible Impala versions?

2018-05-21 Thread Quanlong Huang
Hi friends, We're trying to benchmark Impala+kudu to compare with other lambda architectures like Druid. So we hope we can install the latest release version of Impala (2.12.0) and kudu (1.7.0). However, when following the installation guide in https://kudu.apache.org/docs/installation.html,