+1, I have suffered from gc problem. As I understand, the BatchResult will
be cached and continue to be kept in memory for a little long term, which
cause a lot of data be moved from Young to Old. It is better to move it to
off-heap.
2016-12-20 11:57 GMT+08:00 ZhuWilliam :
> +1 Heap should not
Hi Kumar Vishal,
The accessory for the last email is on the bottom, Pasted as url. The log
is too big to send. I put it on good driver and paste the link.
2016-11-17 16:59 GMT+08:00 An Lan :
> Hi Kumar Vishal,
>
>
>
> I redo some experiment with a detail driver log. The
alancing the data is important.
So I will change the blocket size to 2k rows if the min/max filter could
work on driver side. I have not changed the int type to double type for
measure, I will did it later.
2016-11-17 16:34 GMT+08:00 An Lan :
> Hi Kumar Vishal,
>
>
>
> I redo some e
Hi Kumar Vishal,
Thanks for your suggestion.
The driver log not contain block distribution log by default. How could I
open it?
And how does the order of the dimensions be decided?
2016-11-16 15:14 GMT+08:00 Kumar Vishal :
> Hi An Lan,
> Data is already distributed, in thi
?
2016-11-14 11:29 GMT+08:00 An Lan :
> Hi Kumar Vishal,
>
>
> Driver and some executor logs are in the accessory. The same query run for
> five times.
>
>
>
> Time consume for every query:
>
> 67068ms, 45758ms, 26497ms, 22619ms, 21504ms
>
>
> The first stage
3.log
<https://drive.google.com/file/d/0B1XM6KeI1nB7UHpBQzREX3N5aEk/view?usp=drive_web>
4.log
<https://drive.google.com/file/d/0B1XM6KeI1nB7NVctMmYwNldCVEk/view?usp=drive_web>
5.log
<https://drive.google.com/file/d/0B1XM6KeI1nB7ODhHNE5sSGNfOVE/view?usp=drive_web>
201
reate table..."
2016-11-11 15:08 GMT+08:00 Kumar Vishal :
> Hi An Lan,
>
> Please confirm below things.
>
> 1. Is dynamic executor is enabled?? If it is enabled can u disabled and
> try(this is to check is there any impact with dynamic executor)
> for disabling dynamic
find the problem.
> 3. The TableBlockInfo for task. I log it by myself when
> debugging. It tell me how many blocklets is locality. The spark web monitor
> just give a locality level, but may be only one blocklet is locality.
>
>
> -Regards
> Kumar Vishal
>
>
Hi,
We are using carbondata to build our table and running query in
CarbonContext. We have some performance problem during refining the system.
*Background*：
*cluster*： 100 executor，5 task/executor, 10G
memory/executor
*data*: 60+GB(per one repl
