Re: Continuous Query

2020-10-03 Thread narges saleh
The latter; the server needs to perform some calculations on the data
without sending any notification to the app.

On Fri, Oct 2, 2020 at 4:25 PM Denis Magda  wrote:

> And after you detect a record that satisfies the condition, do you need to
> send any notification to the application? Or is it more like a server
> detects and does some calculation logically without updating the app.
>
> -
> Denis
>
>
> On Fri, Oct 2, 2020 at 11:22 AM narges saleh  wrote:
>
>> The detection should happen at most a couple of minutes after a record is
>> inserted in the cache but all the detections are local to the node. But
>> some records with the current timestamp might show up in the system with
>> big delays.
>>
>> On Fri, Oct 2, 2020 at 12:23 PM Denis Magda  wrote:
>>
>>> What are your requirements? Do you need to process the records as soon
>>> as they are put into the cluster?
>>>
>>>
>>>
>>> On Friday, October 2, 2020, narges saleh  wrote:
>>>
 Thank you Dennis for the reply.
 From the perspective of performance/resource overhead and reliability,
 which approach is preferable? Does a continuous query based approach impose
 a lot more overhead?

 On Fri, Oct 2, 2020 at 9:52 AM Denis Magda  wrote:

> Hi Narges,
>
> Use continuous queries if you need to be notified in real-time, i.e.
> 1) a record is inserted, 2) the continuous filter confirms the record's
> time satisfies your condition, 3) the continuous queries notifies your
> application that does require processing.
>
> The jobs are better for a batching use case when it's ok to process
> records together with some delay.
>
>
> -
> Denis
>
>
> On Fri, Oct 2, 2020 at 3:50 AM narges saleh 
> wrote:
>
>> Hi All,
>>  If I want to watch for a rolling timestamp pattern in all the
>> records that get inserted to all my caches, is it more efficient to use
>> timer based jobs (that checks all the records in some interval) or
>> continuous queries that locally filter on the pattern? These records can
>> get inserted in any order  and some can arrive with delays.
>> An example is to watch for all the records whose timestamp ends in
>> 50, if the timestamp is in the format -mm-dd hh:mi.
>>
>> thanks
>>
>>
>>>
>>> --
>>> -
>>> Denis
>>>
>>>


Re: Ignite thread pool configuration

2020-10-03 Thread Denis Magda
You can certainly reduce its size (
https://ignite.apache.org/docs/latest/perf-and-troubleshooting/thread-pools-tuning#data-streamer-pool).
But at the same time created but unused threads are just some objects that
consume a bit of memory but don't use any CPU resources (until you start
using those).

-
Denis


On Sat, Oct 3, 2020 at 6:12 AM VeenaMithare 
wrote:

> I noticed that the DataStreamer thread pool size seems to be around 32 in
> my
> local machine. In our cluster setup we dont stream data from any external
> db
> . We use native persistence. Do you think it makes sense to reduce this
> pool
> size for both server and client configuraiton. I am not sure if this pool
> will ever be used in our project.
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>


Re: Ignite thread pool configuration

2020-10-03 Thread VeenaMithare
I noticed that the DataStreamer thread pool size seems to be around 32 in my
local machine. In our cluster setup we dont stream data from any external db
. We use native persistence. Do you think it makes sense to reduce this pool
size for both server and client configuraiton. I am not sure if this pool
will ever be used in our project.





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/