Re: compute SQL returned data.

[shawn.du]

Hi,

I think my case is a typical Map-Reduce problem.  I go through Ignite Compute API, but I don't see how ignite collocate compute and data when doing map-reduce.

I describe my logic here:

#1 client issues SQL query to server to get the cache keys. It is a very light task. 

#2 client sends the returned keys in step #1 to ignite sever to do map-reduce jobs. 

     The map-reduce job input data are in ignite caches which can be get by the key. 

     we can split the jobs by the key.

#3 client get the map-reduce result. 

Also i don't know to set mappers. please help and thanks in advance.

Thanks

Shawn

On 01/6/2017 01:57， Denis Magda <dma...@apache.org> wrote：

Keep in mind that if you’re going to execute local SQL queries inside of distributed computations then the computations have to be sent to the cluster with affinity based methods

https://apacheignite.readme.io/docs/collocate-compute-and-data#section-affinity-call-and-run-methods

—

Denis

On Jan 5, 2017, at 12:30 AM, Nikolai Tikhonov <ntikho...@apache.org> wrote:

We can submite Compute Task which will be mapped jobs on nodes. Jobs can execute local SQL query (see Query#setLocal). Examples queries and compute task can be found here: https://github.com/apache/ignite/tree/master/examples/src/main/java/org/apache/ignite/examples

On Thu, Jan 5, 2017 at 11:09 AM, Shawn Du <shawn...@neulion.com.cn> wrote:

Thanks!

 

My case looks like this:

 

We store big cache object to overcome the defects of ignite overhead of each cache entry.

For these big cache object, we can’t use SQL to query/aggregate them.  So we need compute by ourselves.

 

We want to compute on the SQL returned data in ignite server side for several reasons:

#1, for the SQL returned data is huge, we don’t want to transfer them from sever to client.

#2, compute on the client server, we need write code to compute parallel. I assume ignite will do it for us.

#3, in future, If we add new server nodes, we will benefit immediately.

 

Thanks

Shawn

 

 

发件人: Nikolai Tikhonov [mailto:ntikho...@apache.org]
发送时间: 2017年1月5日 15:45
收件人: user@ignite.apache.org
主题: Re: compute SQL returned data.

 

Hi,

 

I'm not sure that undestrand what do you want. Could you describe your case more details?

Also you can find examples on here: https://github.com/apache/ignite/tree/master/examples/src/main/java/org/apache/ignite/examples

 

On Thu, Jan 5, 2017 at 10:20 AM, Shawn Du <shawn...@neulion.com.cn> wrote:

Hi experts,

 

I want to compute the SQL returned data on the server side. Is there any examples?

 

 

Thanks

Shawn