Thanks Andrei,
For user case, please see my email ("Data modeling for segmenting a huge
data set: precomputing vs real time computations").
I think our main confusion right now is trying to understand how exactly
SQL queries work (when memory is moved to heap, when/how is H2 used, how
the reduce step is performed, etc.), because of that we don't really
understand when data is moved between heap, off-heap and disk, and hence
have hard time sizing it properly (I have crushed Ignite many times during
testing). There is already a simialr email thread ("How much heap to
allocate").
We also cannot get some of our OLAP queries to execute in parallel (see
"Slow SQL query uses only a single CPU"), which again makes it harder to
size the HW (no point using huge instances if only a single CPU is going to
be used per query).
Cheers,
Eugene
Furher
On Fri, Aug 24, 2018 at 6:16 AM, aealexsandrov <[email protected]>
wrote:
> Hi,
>
> Ignite doesn't have such kind of benchmarks because they are very specific
> for every case and setup.
>
> However, exists several common tips:
>
> 1)In case if you will use EBS then try to avoid the NVMe. It is fast but
> looks like doesn't provide the guarantees for saving your data. We face the
> corruption of the working directory on this type of devices.
> 2)To get the best performance you should have enough of RAM to store your
> data in Ignite off-heap
> 3)Volume Type - EBS Provisioned IOPS SSD (io1)
>
> I suggest using x1 or x1e instances from
> https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/
> memory-optimized-instances.html
> list.
>
> Your choice will depend on your case and expectations. But for example:
>
> x1e.32xlarge
> ESB = io1
> 2 disks with 2 TB each
>
> It will provide to the capability to store your data in the memory in one
> node and the disk speed will be around 14000MB/SEC.
>
> Is it possible to describe your case in more detail?
>
> BR,
> Andrei
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
