RE: geode and open JDK / newer version of java

[Roi Apelker]

Thanks Jens,

We are planning to have separate efforts - 
1. Open JDK
2. Java 9 (any java)

Has anyone used GEODE with OpenJDK 8? Any issues encountered?

Thanks,

Roi

-Original Message-
From: Jens Deppe [mailto:jde...@pivotal.io] 
Sent: Wednesday, October 18, 2017 5:45 PM
To: dev@geode.apache.org
Subject: Re: geode and open JDK / newer version of java

Hi Roi,

We've spent a little time doing some cursory testing with Java 9 and there are 
definitely some issues:

- Compilation requires additional JVM args because of the new modularization in 
Java 9.
- A few lambda syntaxes in tests need to be adjusted.
- Some tests fail due to powermock issues.
- Using a distro built with Java 1.8, but running on Java 9, I was not able to 
start locators or servers.

I don't remember the exact version used, but I believe it was an OpenJDK 
downloaded just a few days before the official release.

--Jens

On Wed, Oct 18, 2017 at 2:51 AM, Roi Apelker <roi.apel...@amdocs.com> wrote:

> Hi,
>
> Has anyone tried to operate GEODE with open JDK - any version - rather 
> than Oracle's JDK?
>
> Has anyone tried to operate GEODE with open JDK - specifically version 
> 9 (released Sep 21 2017)?
>
> What about Oracle's version 9?
>
> Any migration issues, problems etc. ?
>
> Thanks,
>
> Roi
> This message and the information contained herein is proprietary and 
> confidential and subject to the Amdocs policy statement,
>
> you may review at https://www.amdocs.com/about/email-disclaimer < 
> https://www.amdocs.com/about/email-disclaimer>
>
This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 
<https://www.amdocs.com/about/email-disclaimer>

geode and open JDK / newer version of java

[Roi Apelker]

Hi,

Has anyone tried to operate GEODE with open JDK - any version - rather than 
Oracle's JDK?

Has anyone tried to operate GEODE with open JDK - specifically version 9 
(released Sep 21 2017)?

What about Oracle's version 9?

Any migration issues, problems etc. ?

Thanks,

Roi
This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 

Query mechanism

[Roi Apelker]

Hi,

Is there a design document which explains the query mechanism? Main classes, 
modules, main algorithm?

I have been looking into this area (since I have a performance issue, where it 
seems that data is loaded from disk prematurely in eviction state, which causes 
the system to become really slow),

I have found a few scenarios where things do not work as it seems they should, 
for example using 2 hints doesn't work, using 1 hint may give slower results, 
indexes that are not used (at least according to the trace) - it's something 
larger than a specific issue, and while going through the code I sometimes fail 
to understand what was meant to be and why in several locations.

Any help here would be gladly appreciated,

Thank you,

Roi


This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 

GEODE 1.2 and java 8

[Roi Apelker]

Hi,

As we have moved to Java 8, and compiled GEODE 1.2 with it,

Are there any recommendations regarding java parameters (JVM or any other) that 
can be suggested?

Thanks

Roi
This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 

RE: continuous query internal mechanism questions

[Roi Apelker]

Hi,

After quite some time, I was getting back to this issue where something seems 
wrong with the registration of CQ.

In the java client I see that the code goes to a method called " 
executeCqOnRedundantsAndPrimary " which seems indeed to go to all the relevant 
servers and register the CQ.

Why would there be a need to register the CQ in more than one server, if that 
first server also notifies all the others about the registration?

(the initial registration is with an actual CQ Message, then the server 
notifies the others with  DistributionMessage)

Thanks

Roi

-Original Message-
From: Jason Huynh [mailto:jasonhu...@apache.org] 
Sent: Tuesday, August 15, 2017 9:25 PM
To: dev@geode.apache.org
Subject: Re: continuous query internal mechanism questions

I am not quite sure how native client registers cqs. From my understanding:
with the java api, I believe there is only one message (ExecuteCQ message) that 
is executed on the server side and then replicated to the other nodes through 
the profile (OperationMessage).

It seems the extra ExecuteCQ message failing and then closing the cq might be 
putting the system in a weird state...

On Tue, Aug 15, 2017 at 7:56 AM Roi Apelker <roi.apel...@amdocs.com> wrote:

> Hi,
>
> I have been examining the continuous query registration mechanism for 
> quite some time This is related to an issue that I have, where 
> sometimes a node crashes (1 node out of 2), and the other one does not 
> send CQ events. The CQ is registered on a partitioned region which 
> resides on these 2 nodes.
>
> I noticed the following behavior, and I wonder if anyone can comment 
> regarding it, if it is justified or not and what is the reason:
>
> 1. When the software using the client (native client) registers for 
> the CQ, a CQ command (ExecuteCQ61) is received on both servers.
>  -- is this normal behaviour? Does the client actually send this 
> command to both servers?
>
> 2. When this command is received by a server, and the CQ is 
> registered, another registration message is sent to the other node via 
> an OperationMessage (REGISTER_CQ)
>  -- it seems that regularly, the server can handle this situation as 
> the second registration identifies the previous one and does not 
> affect it. but the question, why do we need this 2nd registration, if 
> there is a command sent to each server?
>
> 3. For some reason, sometimes there is a failure to complete the first 
> registration (executed by ExecuteCQ61) and then this failure causes a 
> closure to the CQ, which is accompanied with a close request to the 
> other node.
>  -- I assume by now, since 2 registrations and one closure have 
> occurred on node 2, the CQ is still active and the client receives 
> notifications.
>
> 4. Sometimes, 1 out of 5, once node 1 crashes, I get a cleanup 
> operation, caused by the crash (via MemberCrashedEvent), and this also 
> closes the existing CQ, and in this case the CQ in node 2 does not 
> operate anymore and the client receives no notifications.
>  -- fact is, that 4 out of 4 times, I do not get this cleanup by 
> MemberCrashedEvent (maybe due to some other error), and that the CQ 
> notifications are received normally.
>
> Can anyone clear things up for me? Any comment on any of the 
> statements above will be greatly appreciated.
>
> Thanks,
>
> Roi
>
>
> -Original Message-
> From: Roi Apelker
> Sent: Wednesday, August 09, 2017 3:21 PM
> To: dev@geode.apache.org
> Subject: RE: continuous query internal mechanism
>
> Dhanyavad
>
> -Original Message-
> From: Anilkumar Gingade [mailto:aging...@pivotal.io]
> Sent: Tuesday, August 08, 2017 9:55 PM
> To: dev@geode.apache.org
> Subject: Re: continuous query internal mechanism
>
> Registered events, i meant, are events generated for interest 
> registration "region.registerInterest(*)". And CqEvents are for CQs 
> registered.
>
> -Anil.
>
>
> On Tue, Aug 8, 2017 at 12:27 AM, Roi Apelker <roi.apel...@amdocs.com>
> wrote:
>
> > Shukriya
> >
> > What is the difference between registered events and CQ events?
> >
> > -Original Message-
> > From: Anilkumar Gingade [mailto:aging...@pivotal.io]
> > Sent: Monday, August 07, 2017 10:12 PM
> > To: dev@geode.apache.org
> > Subject: Re: continuous query internal mechanism
> >
> > CQ Processing on server side is same for all clients (Java, C++)...
> >
> > The subscription events are sent to client as ClientUpdateMessage, 
> > which holds information about registered events and CQ events. The 
> > client process this and updates/invokes the client side 
> > cache/listeners with respective event. Look into 
> > ClientUpdateMess

RE: Indxes and hints

[Roi Apelker]

Thank you,

Can you explain why " The query engine was modified to try to only use one 
index if it can."?

I also noticed that even if I query on A and B, and ORDER BY B - it seemed to 
perform the query on B only, at least in a separate stage. Why is that?




-Roi

-Original Message-
From: Jason Huynh [mailto:jhu...@pivotal.io] 
Sent: Wednesday, August 30, 2017 10:15 PM
To: dev@geode.apache.org
Subject: Re: Indxes and hints

You will probably have to step through debugger for this one.. it really 
depends on the query.  For this query, I expect the query engine to pick one 
index and run the rest of the criteria on the results of the first index used.  
My guess is you have created a CompactRangeIndex, and if so, you can see in 
CompactRangeIndex.java around line 811:

if (ok && runtimeItr != null && iterOps != null) {

  ok = QueryUtils.applyCondition(iterOps, context);

}
This is where it would apply the older conditions (B and C or A and C depending 
on which index was selected) The query engine was modified to try to only use 
one index if it can.

The load from disk (again assuming CompactRangeIndex) is probably occurring in 
MemoryIndexStore.getTargetObject.

On Wed, Aug 30, 2017 at 9:21 AM Roi Apelker <roi.apel...@amdocs.com> wrote:

> One more question:
>
> As I am trying to create a situation where the disk is accessed as 
> least as possible (with a select distinct from X where a=1 and b>10 
> and c=true; In which a and b are indexes and c is not, and c is in the 
> value which is evicted to disk)
>
> Did I get it right - that if I use a hint on a, or a hint on b, or a 
> hint on both, it will first do a select on the hinted, and ONLY THEN the 
> others?
>
> Can anyone refer me to the code (where the 2 phase search occurs)?
>
> Where is the value finally loaded from disk?
>
> Thank you
>
> Roi
> -Original Message-
> From: Roi Apelker
> Sent: Tuesday, August 29, 2017 4:02 PM
> To: dev@geode.apache.org
> Subject: RE: Indxes and hints
>
> Thank you Jason :-)
>
> -Original Message-
> From: Jason Huynh [mailto:jhu...@pivotal.io]
> Sent: Monday, August 28, 2017 7:24 PM
> To: dev@geode.apache.org
> Subject: Re: Indxes and hints
>
> Hi Roi,
>
> Answers are below the questions...
>
> Question 1. Is it true to say, that the query as it is will load all 
> the data values from the file, since the field C is part of the value, 
> which is already persisted to file?
>
> Depending on if an index is used or not, if an index is used, the 
> values that are part of the results will need to be loaded to actually 
> return a result.  If an index is not used, then the all the values 
> would need to be loaded to actually have something to evaluate the filter 
> criteria on.
>
> Question 2. If I add a hint on A and B, will it mean that there will 
> be a
> "2 phase search", first the select on A and B, and then, only on the 
> results, on the field C? (this way, not all records will be loaded 
> from file, only those that suit the A and B condition)
>
> Depending on the  query, it could use one, or more.  If it's a query 
> with only AND clauses, it should just choose one and then evaluate the 
> other filters on the subset that is returned from the index.
>
> Question 3. Is it possible to define an index on a value field? (i.e. 
> not from the key) - will it work exactly like defining one form the 
> key or are three any limitations? (again, I am looking to overcome the 
> situation, where as it seems, the records are loaded unnecessarily 
> from disk)
>
> Yes, indexes can be defined on fields in the value.  It will work the same.
>
>
> If you are sure you are already using an index in the query and still 
> loading every value for every execution of that query, there may be 
> something weird going on...
>
> On Sun, Aug 27, 2017 at 2:55 AM Roi Apelker <roi.apel...@amdocs.com>
> wrote:
> This message and the information contained herein is proprietary and 
> confidential and subject to the Amdocs policy statement,
>
> you may review at https://www.amdocs.com/about/email-disclaimer < 
> https://www.amdocs.com/about/email-disclaimer>
> This message and the information contained herein is proprietary and 
> confidential and subject to the Amdocs policy statement,
>
> you may review at https://www.amdocs.com/about/email-disclaimer < 
> https://www.amdocs.com/about/email-disclaimer>
>
This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 
<https://www.amdocs.com/about/email-disclaimer>

RE: Indxes and hints

[Roi Apelker]

One more question:

As I am trying to create a situation where the disk is accessed as least as 
possible 
(with a select distinct from X where a=1 and b>10 and c=true; 
In which a and b are indexes and c is not, and c is in the value which is 
evicted to disk)

Did I get it right - that if I use a hint on a, or a hint on b, or a hint on 
both, it will first do a select on the hinted, and ONLY THEN the others?

Can anyone refer me to the code (where the 2 phase search occurs)?

Where is the value finally loaded from disk?

Thank you

Roi
-Original Message-
From: Roi Apelker 
Sent: Tuesday, August 29, 2017 4:02 PM
To: dev@geode.apache.org
Subject: RE: Indxes and hints

Thank you Jason :-)

-Original Message-
From: Jason Huynh [mailto:jhu...@pivotal.io] 
Sent: Monday, August 28, 2017 7:24 PM
To: dev@geode.apache.org
Subject: Re: Indxes and hints

Hi Roi,

Answers are below the questions...

Question 1. Is it true to say, that the query as it is will load all the data 
values from the file, since the field C is part of the value, which is already 
persisted to file?

Depending on if an index is used or not, if an index is used, the values that 
are part of the results will need to be loaded to actually return a result.  If 
an index is not used, then the all the values would need to be loaded to 
actually have something to evaluate the filter criteria on.

Question 2. If I add a hint on A and B, will it mean that there will be a
"2 phase search", first the select on A and B, and then, only on the results, 
on the field C? (this way, not all records will be loaded from file, only those 
that suit the A and B condition)

Depending on the  query, it could use one, or more.  If it's a query with only 
AND clauses, it should just choose one and then evaluate the other filters on 
the subset that is returned from the index.

Question 3. Is it possible to define an index on a value field? (i.e. not from 
the key) - will it work exactly like defining one form the key or are three any 
limitations? (again, I am looking to overcome the situation, where as it seems, 
the records are loaded unnecessarily from disk)

Yes, indexes can be defined on fields in the value.  It will work the same.


If you are sure you are already using an index in the query and still loading 
every value for every execution of that query, there may be something weird 
going on...

On Sun, Aug 27, 2017 at 2:55 AM Roi Apelker <roi.apel...@amdocs.com> wrote:
This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 
<https://www.amdocs.com/about/email-disclaimer>
This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 
<https://www.amdocs.com/about/email-disclaimer>

RE: Indxes and hints

[Roi Apelker]

Thank you Jason :-)

-Original Message-
From: Jason Huynh [mailto:jhu...@pivotal.io] 
Sent: Monday, August 28, 2017 7:24 PM
To: dev@geode.apache.org
Subject: Re: Indxes and hints

Hi Roi,

Answers are below the questions...

Question 1. Is it true to say, that the query as it is will load all the data 
values from the file, since the field C is part of the value, which is already 
persisted to file?

Depending on if an index is used or not, if an index is used, the values that 
are part of the results will need to be loaded to actually return a result.  If 
an index is not used, then the all the values would need to be loaded to 
actually have something to evaluate the filter criteria on.

Question 2. If I add a hint on A and B, will it mean that there will be a
"2 phase search", first the select on A and B, and then, only on the results, 
on the field C? (this way, not all records will be loaded from file, only those 
that suit the A and B condition)

Depending on the  query, it could use one, or more.  If it's a query with only 
AND clauses, it should just choose one and then evaluate the other filters on 
the subset that is returned from the index.

Question 3. Is it possible to define an index on a value field? (i.e. not from 
the key) - will it work exactly like defining one form the key or are three any 
limitations? (again, I am looking to overcome the situation, where as it seems, 
the records are loaded unnecessarily from disk)

Yes, indexes can be defined on fields in the value.  It will work the same.


If you are sure you are already using an index in the query and still loading 
every value for every execution of that query, there may be something weird 
going on...

On Sun, Aug 27, 2017 at 2:55 AM Roi Apelker <roi.apel...@amdocs.com> wrote:
This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 
<https://www.amdocs.com/about/email-disclaimer>

Query mechanism

[Roi Apelker]

Hi,

I am looking into the internals of how the query process works, and how 
indexes/hints affect it,

Can anyone point me to the most important classes, the main mechanism "wheels" 
etc.?

Thanks,

Roi


-Original Message-----
From: Roi Apelker 
Sent: Sunday, August 27, 2017 12:55 PM
To: dev@geode.apache.org
Subject: Indxes and hints

Hi,

I have a few questions regarding indexes and hints, if someone could confirm 
the below it would be great:

- I have a situation where I use 3 field values in a select (something like 
select where A>1, B>1, C=true)
- A and B are fields on the key, and C is a field on the value.
- A and B are indexes
- I am looking for the most efficient way to execute the query above, in the 
situation where there is overflow to eviction files, meaning some of the data 
has already been evicted to a file, which slows down the select considerably 
(this is not persistence, but overflow).


1. Is it true to say, that the query as it is will load all the data values 
from the file, since the field C is part of the value, which is already 
persisted to file?
2. If I add a hint on A and B, will it mean that there will be a "2 phase 
search", first the select on A and B, and then, only on the results, on the 
field C? (this way, not all records will be loaded from file, only those that 
suit the A and B condition) 3. Is it possible to define an index on a value 
field? (i.e. not from the key) - will it work exactly like defining one form 
the key or are three any limitations? (again, I am looking to overcome the 
situation, where as it seems, the records are loaded unnecessarily from disk)

Thank you,

Roi


-Original Message-
From: Roi Apelker
Sent: Thursday, August 24, 2017 7:03 PM
To: dev@geode.apache.org
Subject: eviction files

Hi,

I am looking into the internals of the eviction process,

Can anyone point me to the most important classes, the main mechanism "wheels" 
etc.?

Thanks,

Roi

-Original Message-
From: Roi Apelker
Sent: Wednesday, August 16, 2017 8:38 PM
To: dev@geode.apache.org
Subject: RE: continuous query internal mechanism questions

It seems like the code in the native client (in the version I have, which may 
be old) send the message to all servers:

CqResultsPtr CqQueryImpl::executeWithInitialResults(uint32_t timeout) {
  ...

  TcrMessage msg(TcrMessage::EXECUTECQ_WITH_IR_MSG_TYPE, m_cqName, 
m_queryString, CqState::RUNNING, isDurable(), m_tccdm);
  TcrMessage reply(true, m_tccdm);
  ChunkedQueryResponse* resultCollector = (new ChunkedQueryResponse(reply));
  reply.setChunkedResultHandler(static_cast(resultCollector));
  reply.setTimeout(timeout);

  GfErrType err = GF_NOERR;
  err = m_tccdm->sendSyncRequest(msg, reply); ..

And sendSyncRequest:
...

for (std::vector<TcrEndpoint*>::iterator ep = m_endpoints.begin(); ep != 
m_endpoints.end(); ++ep) {
if ((*ep)->connected()) {
  (*ep)->setDM(this);
  opErr = sendRequestToEP(request, reply, *ep);//this will go to 
ThinClientDistributionManager

...


Can this be causing the issue?



-Roi





-Original Message-
From: Jason Huynh [mailto:jasonhu...@apache.org]
Sent: Tuesday, August 15, 2017 9:25 PM
To: dev@geode.apache.org
Subject: Re: continuous query internal mechanism questions

I am not quite sure how native client registers cqs. From my understanding:
with the java api, I believe there is only one message (ExecuteCQ message) that 
is executed on the server side and then replicated to the other nodes through 
the profile (OperationMessage).

It seems the extra ExecuteCQ message failing and then closing the cq might be 
putting the system in a weird state...

On Tue, Aug 15, 2017 at 7:56 AM Roi Apelker <roi.apel...@amdocs.com> wrote:

> Hi,
>
> I have been examining the continuous query registration mechanism for 
> quite some time This is related to an issue that I have, where 
> sometimes a node crashes (1 node out of 2), and the other one does not 
> send CQ events. The CQ is registered on a partitioned region which 
> resides on these 2 nodes.
>
> I noticed the following behavior, and I wonder if anyone can comment 
> regarding it, if it is justified or not and what is the reason:
>
> 1. When the software using the client (native client) registers for 
> the CQ, a CQ command (ExecuteCQ61) is received on both servers.
>  -- is this normal behaviour? Does the client actually send this 
> command to both servers?
>
> 2. When this command is received by a server, and the CQ is 
> registered, another registration message is sent to the other node via 
> an OperationMessage (REGISTER_CQ)
>  -- it seems that regularly, the server can handle this situation as 
> the second registration identifies the previous one and does not 
> affect it. but the question, why do we need this 2nd registration, if 
> there is a command sent to ea

Indxes and hints

[Roi Apelker]

Hi,

I have a few questions regarding indexes and hints, if someone could confirm 
the below it would be great:

- I have a situation where I use 3 field values in a select (something like 
select where A>1, B>1, C=true)
- A and B are fields on the key, and C is a field on the value.
- A and B are indexes
- I am looking for the most efficient way to execute the query above, in the 
situation where there is overflow to eviction files, meaning some of the data 
has already been evicted to a file, which slows down the select considerably 
(this is not persistence, but overflow).


1. Is it true to say, that the query as it is will load all the data values 
from the file, since the field C is part of the value, which is already 
persisted to file?
2. If I add a hint on A and B, will it mean that there will be a "2 phase 
search", first the select on A and B, and then, only on the results, on the 
field C? (this way, not all records will be loaded from file, only those that 
suit the A and B condition)
3. Is it possible to define an index on a value field? (i.e. not from the key) 
- will it work exactly like defining one form the key or are three any 
limitations? (again, I am looking to overcome the situation, where as it seems, 
the records are loaded unnecessarily from disk)

Thank you,

Roi


-Original Message-
From: Roi Apelker 
Sent: Thursday, August 24, 2017 7:03 PM
To: dev@geode.apache.org
Subject: eviction files

Hi,

I am looking into the internals of the eviction process,

Can anyone point me to the most important classes, the main mechanism "wheels" 
etc.?

Thanks,

Roi

-Original Message-
From: Roi Apelker
Sent: Wednesday, August 16, 2017 8:38 PM
To: dev@geode.apache.org
Subject: RE: continuous query internal mechanism questions

It seems like the code in the native client (in the version I have, which may 
be old) send the message to all servers:

CqResultsPtr CqQueryImpl::executeWithInitialResults(uint32_t timeout) {
  ...

  TcrMessage msg(TcrMessage::EXECUTECQ_WITH_IR_MSG_TYPE, m_cqName, 
m_queryString, CqState::RUNNING, isDurable(), m_tccdm);
  TcrMessage reply(true, m_tccdm);
  ChunkedQueryResponse* resultCollector = (new ChunkedQueryResponse(reply));
  reply.setChunkedResultHandler(static_cast(resultCollector));
  reply.setTimeout(timeout);

  GfErrType err = GF_NOERR;
  err = m_tccdm->sendSyncRequest(msg, reply); ..

And sendSyncRequest:
...

for (std::vector<TcrEndpoint*>::iterator ep = m_endpoints.begin(); ep != 
m_endpoints.end(); ++ep) {
if ((*ep)->connected()) {
  (*ep)->setDM(this);
  opErr = sendRequestToEP(request, reply, *ep);//this will go to 
ThinClientDistributionManager

...


Can this be causing the issue?



-Roi





-Original Message-
From: Jason Huynh [mailto:jasonhu...@apache.org]
Sent: Tuesday, August 15, 2017 9:25 PM
To: dev@geode.apache.org
Subject: Re: continuous query internal mechanism questions

I am not quite sure how native client registers cqs. From my understanding:
with the java api, I believe there is only one message (ExecuteCQ message) that 
is executed on the server side and then replicated to the other nodes through 
the profile (OperationMessage).

It seems the extra ExecuteCQ message failing and then closing the cq might be 
putting the system in a weird state...

On Tue, Aug 15, 2017 at 7:56 AM Roi Apelker <roi.apel...@amdocs.com> wrote:

> Hi,
>
> I have been examining the continuous query registration mechanism for 
> quite some time This is related to an issue that I have, where 
> sometimes a node crashes (1 node out of 2), and the other one does not 
> send CQ events. The CQ is registered on a partitioned region which 
> resides on these 2 nodes.
>
> I noticed the following behavior, and I wonder if anyone can comment 
> regarding it, if it is justified or not and what is the reason:
>
> 1. When the software using the client (native client) registers for 
> the CQ, a CQ command (ExecuteCQ61) is received on both servers.
>  -- is this normal behaviour? Does the client actually send this 
> command to both servers?
>
> 2. When this command is received by a server, and the CQ is 
> registered, another registration message is sent to the other node via 
> an OperationMessage (REGISTER_CQ)
>  -- it seems that regularly, the server can handle this situation as 
> the second registration identifies the previous one and does not 
> affect it. but the question, why do we need this 2nd registration, if 
> there is a command sent to each server?
>
> 3. For some reason, sometimes there is a failure to complete the first 
> registration (executed by ExecuteCQ61) and then this failure causes a 
> closure to the CQ, which is accompanied with a close request to the 
> other node.
>  -- I assume by now, since 2 registrations and one closure have 
> occurred on node 2, the CQ 

eviction files

[Roi Apelker]

Hi,

I am looking into the internals of the eviction process,

Can anyone point me to the most important classes, the main mechanism "wheels" 
etc.?

Thanks,

Roi

-Original Message-----
From: Roi Apelker 
Sent: Wednesday, August 16, 2017 8:38 PM
To: dev@geode.apache.org
Subject: RE: continuous query internal mechanism questions

It seems like the code in the native client (in the version I have, which may 
be old) send the message to all servers:

CqResultsPtr CqQueryImpl::executeWithInitialResults(uint32_t timeout) {
  ...

  TcrMessage msg(TcrMessage::EXECUTECQ_WITH_IR_MSG_TYPE, m_cqName, 
m_queryString, CqState::RUNNING, isDurable(), m_tccdm);
  TcrMessage reply(true, m_tccdm);
  ChunkedQueryResponse* resultCollector = (new ChunkedQueryResponse(reply));
  reply.setChunkedResultHandler(static_cast(resultCollector));
  reply.setTimeout(timeout);

  GfErrType err = GF_NOERR;
  err = m_tccdm->sendSyncRequest(msg, reply); ..

And sendSyncRequest:
...

for (std::vector<TcrEndpoint*>::iterator ep = m_endpoints.begin(); ep != 
m_endpoints.end(); ++ep) {
if ((*ep)->connected()) {
  (*ep)->setDM(this);
  opErr = sendRequestToEP(request, reply, *ep);//this will go to 
ThinClientDistributionManager

...


Can this be causing the issue?



-Roi





-Original Message-
From: Jason Huynh [mailto:jasonhu...@apache.org]
Sent: Tuesday, August 15, 2017 9:25 PM
To: dev@geode.apache.org
Subject: Re: continuous query internal mechanism questions

I am not quite sure how native client registers cqs. From my understanding:
with the java api, I believe there is only one message (ExecuteCQ message) that 
is executed on the server side and then replicated to the other nodes through 
the profile (OperationMessage).

It seems the extra ExecuteCQ message failing and then closing the cq might be 
putting the system in a weird state...

On Tue, Aug 15, 2017 at 7:56 AM Roi Apelker <roi.apel...@amdocs.com> wrote:

> Hi,
>
> I have been examining the continuous query registration mechanism for 
> quite some time This is related to an issue that I have, where 
> sometimes a node crashes (1 node out of 2), and the other one does not 
> send CQ events. The CQ is registered on a partitioned region which 
> resides on these 2 nodes.
>
> I noticed the following behavior, and I wonder if anyone can comment 
> regarding it, if it is justified or not and what is the reason:
>
> 1. When the software using the client (native client) registers for 
> the CQ, a CQ command (ExecuteCQ61) is received on both servers.
>  -- is this normal behaviour? Does the client actually send this 
> command to both servers?
>
> 2. When this command is received by a server, and the CQ is 
> registered, another registration message is sent to the other node via 
> an OperationMessage (REGISTER_CQ)
>  -- it seems that regularly, the server can handle this situation as 
> the second registration identifies the previous one and does not 
> affect it. but the question, why do we need this 2nd registration, if 
> there is a command sent to each server?
>
> 3. For some reason, sometimes there is a failure to complete the first 
> registration (executed by ExecuteCQ61) and then this failure causes a 
> closure to the CQ, which is accompanied with a close request to the 
> other node.
>  -- I assume by now, since 2 registrations and one closure have 
> occurred on node 2, the CQ is still active and the client receives 
> notifications.
>
> 4. Sometimes, 1 out of 5, once node 1 crashes, I get a cleanup 
> operation, caused by the crash (via MemberCrashedEvent), and this also 
> closes the existing CQ, and in this case the CQ in node 2 does not 
> operate anymore and the client receives no notifications.
>  -- fact is, that 4 out of 4 times, I do not get this cleanup by 
> MemberCrashedEvent (maybe due to some other error), and that the CQ 
> notifications are received normally.
>
> Can anyone clear things up for me? Any comment on any of the 
> statements above will be greatly appreciated.
>
> Thanks,
>
> Roi
>
>
> -Original Message-
> From: Roi Apelker
> Sent: Wednesday, August 09, 2017 3:21 PM
> To: dev@geode.apache.org
> Subject: RE: continuous query internal mechanism
>
> Dhanyavad
>
> -Original Message-
> From: Anilkumar Gingade [mailto:aging...@pivotal.io]
> Sent: Tuesday, August 08, 2017 9:55 PM
> To: dev@geode.apache.org
> Subject: Re: continuous query internal mechanism
>
> Registered events, i meant, are events generated for interest 
> registration "region.registerInterest(*)". And CqEvents are for CQs 
> registered.
>
> -Anil.
>
>
> On Tue, Aug 8, 2017 at 12:27 AM, Roi Apelker <roi.apel...@amdocs.com>
> wrote:
>
> > Shukriya
> >
> >

RE: continuous query internal mechanism questions

[Roi Apelker]

It seems like the code in the native client (in the version I have, which may 
be old) send the message to all servers:

CqResultsPtr CqQueryImpl::executeWithInitialResults(uint32_t timeout)
{
  ...

  TcrMessage msg(TcrMessage::EXECUTECQ_WITH_IR_MSG_TYPE, m_cqName, 
m_queryString, CqState::RUNNING, isDurable(), m_tccdm);
  TcrMessage reply(true, m_tccdm);
  ChunkedQueryResponse* resultCollector = (new ChunkedQueryResponse(reply));
  reply.setChunkedResultHandler(static_cast(resultCollector));
  reply.setTimeout(timeout);

  GfErrType err = GF_NOERR;
  err = m_tccdm->sendSyncRequest(msg, reply);
..

And sendSyncRequest:
...

for (std::vector<TcrEndpoint*>::iterator ep = m_endpoints.begin(); ep != 
m_endpoints.end(); ++ep) {
if ((*ep)->connected()) {
  (*ep)->setDM(this);
  opErr = sendRequestToEP(request, reply, *ep);//this will go to 
ThinClientDistributionManager

...


Can this be causing the issue?



-Roi





-Original Message-
From: Jason Huynh [mailto:jasonhu...@apache.org] 
Sent: Tuesday, August 15, 2017 9:25 PM
To: dev@geode.apache.org
Subject: Re: continuous query internal mechanism questions

I am not quite sure how native client registers cqs. From my understanding:
with the java api, I believe there is only one message (ExecuteCQ message) that 
is executed on the server side and then replicated to the other nodes through 
the profile (OperationMessage).

It seems the extra ExecuteCQ message failing and then closing the cq might be 
putting the system in a weird state...

On Tue, Aug 15, 2017 at 7:56 AM Roi Apelker <roi.apel...@amdocs.com> wrote:

> Hi,
>
> I have been examining the continuous query registration mechanism for 
> quite some time This is related to an issue that I have, where 
> sometimes a node crashes (1 node out of 2), and the other one does not 
> send CQ events. The CQ is registered on a partitioned region which 
> resides on these 2 nodes.
>
> I noticed the following behavior, and I wonder if anyone can comment 
> regarding it, if it is justified or not and what is the reason:
>
> 1. When the software using the client (native client) registers for 
> the CQ, a CQ command (ExecuteCQ61) is received on both servers.
>  -- is this normal behaviour? Does the client actually send this 
> command to both servers?
>
> 2. When this command is received by a server, and the CQ is 
> registered, another registration message is sent to the other node via 
> an OperationMessage (REGISTER_CQ)
>  -- it seems that regularly, the server can handle this situation as 
> the second registration identifies the previous one and does not 
> affect it. but the question, why do we need this 2nd registration, if 
> there is a command sent to each server?
>
> 3. For some reason, sometimes there is a failure to complete the first 
> registration (executed by ExecuteCQ61) and then this failure causes a 
> closure to the CQ, which is accompanied with a close request to the 
> other node.
>  -- I assume by now, since 2 registrations and one closure have 
> occurred on node 2, the CQ is still active and the client receives 
> notifications.
>
> 4. Sometimes, 1 out of 5, once node 1 crashes, I get a cleanup 
> operation, caused by the crash (via MemberCrashedEvent), and this also 
> closes the existing CQ, and in this case the CQ in node 2 does not 
> operate anymore and the client receives no notifications.
>  -- fact is, that 4 out of 4 times, I do not get this cleanup by 
> MemberCrashedEvent (maybe due to some other error), and that the CQ 
> notifications are received normally.
>
> Can anyone clear things up for me? Any comment on any of the 
> statements above will be greatly appreciated.
>
> Thanks,
>
> Roi
>
>
> -Original Message-
> From: Roi Apelker
> Sent: Wednesday, August 09, 2017 3:21 PM
> To: dev@geode.apache.org
> Subject: RE: continuous query internal mechanism
>
> Dhanyavad
>
> -Original Message-
> From: Anilkumar Gingade [mailto:aging...@pivotal.io]
> Sent: Tuesday, August 08, 2017 9:55 PM
> To: dev@geode.apache.org
> Subject: Re: continuous query internal mechanism
>
> Registered events, i meant, are events generated for interest 
> registration "region.registerInterest(*)". And CqEvents are for CQs 
> registered.
>
> -Anil.
>
>
> On Tue, Aug 8, 2017 at 12:27 AM, Roi Apelker <roi.apel...@amdocs.com>
> wrote:
>
> > Shukriya
> >
> > What is the difference between registered events and CQ events?
> >
> > -Original Message-
> > From: Anilkumar Gingade [mailto:aging...@pivotal.io]
> > Sent: Monday, August 07, 2017 10:12 PM
> > To: dev@geode.apache.org
> > Subject: Re: continuous query internal mechanism
> >
> > CQ Pr

RE: continuous query internal mechanism questions

[Roi Apelker]

Thanks,
The subscription-redundancy is set to "1" and the region is used in 2 nodes 
(there are more nodes which re not related to it).

Yes there is an exception, which I am yet to understand: (and this exception 
causes the closure of the CQ in this node as well as sending operation message 
to the other node to close!)

caught exception while running: 
java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
at sun.nio.ch.IOUtil.write(IOUtil.java:51)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
at 
org.apache.geode.internal.cache.tier.sockets.Message.flushBuffer(Message.java:651)
at 
org.apache.geode.internal.cache.tier.sockets.Message.sendBytes(Message.java:632)
at 
org.apache.geode.internal.cache.tier.sockets.ChunkedMessage.sendChunk(ChunkedMessage.java:314)
at 
org.apache.geode.internal.cache.tier.sockets.ChunkedMessage.sendChunk(ChunkedMessage.java:322)
at 
org.apache.geode.internal.cache.tier.sockets.BaseCommand.writeQueryResponseChunk(BaseCommand.java:756)
at 
org.apache.geode.internal.cache.tier.sockets.BaseCommandQuery.processQueryUsingParams(BaseCommandQuery.java:225)
at 
org.apache.geode.internal.cache.tier.sockets.BaseCommandQuery.processQuery(BaseCommandQuery.java:70)
at 
org.apache.geode.internal.cache.tier.sockets.command.ExecuteCQ61.cmdExecute(ExecuteCQ61.java:179)
at 
org.apache.geode.internal.cache.tier.sockets.BaseCommand.execute(BaseCommand.java:147)
at 
org.apache.geode.internal.cache.tier.sockets.ServerConnection.doNormalMsg(ServerConnection.java:783)
at 
org.apache.geode.internal.cache.tier.sockets.ServerConnection.doOneMessage(ServerConnection.java:913)
at 
org.apache.geode.internal.cache.tier.sockets.ServerConnection.run(ServerConnection.java:1143)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at 
org.apache.geode.internal.cache.tier.sockets.AcceptorImpl$1$1.run(AcceptorImpl.java:546)
at java.lang.Thread.run(Thread.java:745)

could it be the client disconnected from the node right after sending this 
message? (The client itself continues to run normally...)
The scenario is that after all nodes are initialized, I am stopping one server 
out of 2. Sometimes, 1 out of 5 - the CQ stops notifying the client after this 
stop. Most if the time the CQ continues to run fine.
I am certain this is related to some timing issue, some registration which 
fails, something also related to the filter profiles which is held in the 
region...

Thanks

Roi

-Original Message-
From: Anilkumar Gingade [mailto:aging...@pivotal.io] 
Sent: Wednesday, August 16, 2017 1:41 AM
To: dev@geode.apache.org
Subject: Re: continuous query internal mechanism questions

In Geode, high availability for subscription events are achieved by having 
redundant event-queues (HAQueues) on multiple severs; this is configured using 
redundancy-level with client connection. Based on the redundancy level, the 
client register CQs on multiple servers. During the subscription
(CQ) registration, it elects/assigns one of the server to host primary HAQueue.

The client keeps monitoring the redundancy level during node join or failure; 
to satisfy the redundancy level.

You can find more about HAQueues at
https://cwiki.apache.org/confluence/display/GEODE/HA+Client+Event+Queues

I assume, you have 2 node cluster. What is your subscription redundancy level?

>> For some reason, sometimes there is a failure to complete the first
registration
Is there any log message, stack trace, reporting reason for failure? If its dev 
environment, you can run client/server with debug/fine level log to see 
additional info.

Are you trying to stop your server, while registering the CQs? Can you give 
more detail about your test scenario...

-Anil.


On Tue, Aug 15, 2017 at 11:25 AM, Jason Huynh <jasonhu...@apache.org> wrote:

> I am not quite sure how native client registers cqs. From my understanding:
> with the java api, I believe there is only one message (ExecuteCQ 
> message) that is executed on the server side and then replicated to 
> the other nodes through the profile (OperationMessage).
>
> It seems the extra ExecuteCQ message failing and then closing the cq 
> might be putting the system in a weird state...
>
> On Tue, Aug 15, 2017 at 7:56 AM Roi Apelker <roi.apel...@amdocs.com>
> wrote:
>
> > Hi,
> >
> > I have been examining the continuous query registration mechanism 
> > for quite some time This is related to an issue that I have, where 
> > sometimes a node crashes
&g

continuous query - high availability and assurance

[Roi Apelker]

Hi,

I have been working on an issue related to continuous query mechanism, and I 
would like to know if this mechanism, with High availability, assures 100% data 
integrity?

For example, if I have 2 nodes where a client registered to CQ, and one node 
fails, is there any scenario, in which a notification might be lost, either not 
sent at all, or hasn’t reached its destination and will not be sent, or any 
other? 

Is there a guarantee that all information pertinent for the CQ will reach the 
subscriber?

Thanks,

Roi
This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 

continuous query internal mechanism questions

[Roi Apelker]

Hi,

I have been examining the continuous query registration mechanism for quite 
some time 
This is related to an issue that I have, where sometimes a node crashes (1 node 
out of 2), and the other one does not send CQ events. The CQ is registered on a 
partitioned region which resides on these 2 nodes.

I noticed the following behavior, and I wonder if anyone can comment regarding 
it, if it is justified or not and what is the reason:

1. When the software using the client (native client) registers for the CQ, a 
CQ command (ExecuteCQ61) is received on both servers.
 -- is this normal behaviour? Does the client actually send this command to 
both servers?

2. When this command is received by a server, and the CQ is registered, another 
registration message is sent to the other node via an OperationMessage 
(REGISTER_CQ)
 -- it seems that regularly, the server can handle this situation as the second 
registration identifies the previous one and does not affect it. but the 
question, why do we need this 2nd registration, if there is a command sent to 
each server?

3. For some reason, sometimes there is a failure to complete the first 
registration (executed by ExecuteCQ61) and then this failure causes a closure 
to the CQ, which is accompanied with a close request to the other node.
 -- I assume by now, since 2 registrations and one closure have occurred on 
node 2, the CQ is still active and the client receives notifications.

4. Sometimes, 1 out of 5, once node 1 crashes, I get a cleanup operation, 
caused by the crash (via MemberCrashedEvent), and this also closes the existing 
CQ, and in this case the CQ in node 2 does not operate anymore and the client 
receives no notifications.
 -- fact is, that 4 out of 4 times, I do not get this cleanup by 
MemberCrashedEvent (maybe due to some other error), and that the CQ 
notifications are received normally.

Can anyone clear things up for me? Any comment on any of the statements above 
will be greatly appreciated. 
 
Thanks,

Roi


-Original Message-
From: Roi Apelker 
Sent: Wednesday, August 09, 2017 3:21 PM
To: dev@geode.apache.org
Subject: RE: continuous query internal mechanism

Dhanyavad

-Original Message-
From: Anilkumar Gingade [mailto:aging...@pivotal.io]
Sent: Tuesday, August 08, 2017 9:55 PM
To: dev@geode.apache.org
Subject: Re: continuous query internal mechanism

Registered events, i meant, are events generated for interest registration 
"region.registerInterest(*)". And CqEvents are for CQs registered.

-Anil.


On Tue, Aug 8, 2017 at 12:27 AM, Roi Apelker <roi.apel...@amdocs.com> wrote:

> Shukriya
>
> What is the difference between registered events and CQ events?
>
> -Original Message-
> From: Anilkumar Gingade [mailto:aging...@pivotal.io]
> Sent: Monday, August 07, 2017 10:12 PM
> To: dev@geode.apache.org
> Subject: Re: continuous query internal mechanism
>
> CQ Processing on server side is same for all clients (Java, C++)...
>
> The subscription events are sent to client as ClientUpdateMessage, 
> which holds information about registered events and CQ events. The 
> client process this and updates/invokes the client side 
> cache/listeners with respective event. Look into 
> ClientUpdateMessageImpl and CacheClientUpdater (for client side processing).
>
> -Anil.
>
>
>
>
> On Mon, Aug 7, 2017 at 11:01 AM, Roi Apelker <roi.apel...@amdocs.com>
> wrote:
>
> > Thanks,
> >
> > By the way, is there any difference in the behaviour of the server, 
> > if the client that registered the CQ is a native (C++) client?
> >
> > I have been going over the classes and code for some time and can't 
> > seem to find the actual location where a CQ update/notification is
> sent...
> >
> > It's like CqEventImpl class is never even generated in this scenario.
> >
> > If anyone can help here I would be most grateful :-)
> >
> > Thanks
> >
> > Roi
> >
> >
> >
> > -Original Message-
> > From: Anilkumar Gingade [mailto:aging...@pivotal.io]
> > Sent: Monday, August 07, 2017 8:23 PM
> > To: dev@geode.apache.org
> > Subject: Re: continuous query internal mechanism
> >
> > You can find those in CqServiceImpl.process*()...
> >
> > -Anil.
> >
> >
> > On Mon, Aug 7, 2017 at 9:14 AM, Roi Apelker <roi.apel...@amdocs.com>
> > wrote:
> >
> > > Hello,
> > >
> > > I am trying to look into the code of the continuous query 
> > > mechanism
> > > - where the GEODE server sends the notification back to the client.
> > >
> > > Can anyone point me to the central classes of continuous query, 
> > > especially to the one that is responsible for the calculation of 
> 

RE: continuous query internal mechanism

[Roi Apelker]

Dhanyavad

-Original Message-
From: Anilkumar Gingade [mailto:aging...@pivotal.io] 
Sent: Tuesday, August 08, 2017 9:55 PM
To: dev@geode.apache.org
Subject: Re: continuous query internal mechanism

Registered events, i meant, are events generated for interest registration 
"region.registerInterest(*)". And CqEvents are for CQs registered.

-Anil.


On Tue, Aug 8, 2017 at 12:27 AM, Roi Apelker <roi.apel...@amdocs.com> wrote:

> Shukriya
>
> What is the difference between registered events and CQ events?
>
> -Original Message-
> From: Anilkumar Gingade [mailto:aging...@pivotal.io]
> Sent: Monday, August 07, 2017 10:12 PM
> To: dev@geode.apache.org
> Subject: Re: continuous query internal mechanism
>
> CQ Processing on server side is same for all clients (Java, C++)...
>
> The subscription events are sent to client as ClientUpdateMessage, 
> which holds information about registered events and CQ events. The 
> client process this and updates/invokes the client side 
> cache/listeners with respective event. Look into 
> ClientUpdateMessageImpl and CacheClientUpdater (for client side processing).
>
> -Anil.
>
>
>
>
> On Mon, Aug 7, 2017 at 11:01 AM, Roi Apelker <roi.apel...@amdocs.com>
> wrote:
>
> > Thanks,
> >
> > By the way, is there any difference in the behaviour of the server, 
> > if the client that registered the CQ is a native (C++) client?
> >
> > I have been going over the classes and code for some time and can't 
> > seem to find the actual location where a CQ update/notification is
> sent...
> >
> > It's like CqEventImpl class is never even generated in this scenario.
> >
> > If anyone can help here I would be most grateful :-)
> >
> > Thanks
> >
> > Roi
> >
> >
> >
> > -Original Message-
> > From: Anilkumar Gingade [mailto:aging...@pivotal.io]
> > Sent: Monday, August 07, 2017 8:23 PM
> > To: dev@geode.apache.org
> > Subject: Re: continuous query internal mechanism
> >
> > You can find those in CqServiceImpl.process*()...
> >
> > -Anil.
> >
> >
> > On Mon, Aug 7, 2017 at 9:14 AM, Roi Apelker <roi.apel...@amdocs.com>
> > wrote:
> >
> > > Hello,
> > >
> > > I am trying to look into the code of the continuous query 
> > > mechanism
> > > - where the GEODE server sends the notification back to the client.
> > >
> > > Can anyone point me to the central classes of continuous query, 
> > > especially to the one that is responsible for the calculation of 
> > > the new data and packing it as a message back to the client?
> > >
> > > Thanks,
> > >
> > > Roi
> > >
> > > This message and the information contained herein is proprietary 
> > > and confidential and subject to the Amdocs policy statement,
> > >
> > > you may review at https://www.amdocs.com/about/email-disclaimer < 
> > > https://www.amdocs.com/about/email-disclaimer>
> > >
> > This message and the information contained herein is proprietary and 
> > confidential and subject to the Amdocs policy statement,
> >
> > you may review at https://www.amdocs.com/about/email-disclaimer < 
> > https://www.amdocs.com/about/email-disclaimer>
> >
> This message and the information contained herein is proprietary and 
> confidential and subject to the Amdocs policy statement,
>
> you may review at https://www.amdocs.com/about/email-disclaimer < 
> https://www.amdocs.com/about/email-disclaimer>
>
This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 
<https://www.amdocs.com/about/email-disclaimer>

RE: continuous query internal mechanism

[Roi Apelker]

Shukriya

What is the difference between registered events and CQ events?

-Original Message-
From: Anilkumar Gingade [mailto:aging...@pivotal.io] 
Sent: Monday, August 07, 2017 10:12 PM
To: dev@geode.apache.org
Subject: Re: continuous query internal mechanism

CQ Processing on server side is same for all clients (Java, C++)...

The subscription events are sent to client as ClientUpdateMessage, which holds 
information about registered events and CQ events. The client process this and 
updates/invokes the client side cache/listeners with respective event. Look 
into ClientUpdateMessageImpl and CacheClientUpdater (for client side 
processing).

-Anil.




On Mon, Aug 7, 2017 at 11:01 AM, Roi Apelker <roi.apel...@amdocs.com> wrote:

> Thanks,
>
> By the way, is there any difference in the behaviour of the server, if 
> the client that registered the CQ is a native (C++) client?
>
> I have been going over the classes and code for some time and can't 
> seem to find the actual location where a CQ update/notification is sent...
>
> It's like CqEventImpl class is never even generated in this scenario.
>
> If anyone can help here I would be most grateful :-)
>
> Thanks
>
> Roi
>
>
>
> -Original Message-
> From: Anilkumar Gingade [mailto:aging...@pivotal.io]
> Sent: Monday, August 07, 2017 8:23 PM
> To: dev@geode.apache.org
> Subject: Re: continuous query internal mechanism
>
> You can find those in CqServiceImpl.process*()...
>
> -Anil.
>
>
> On Mon, Aug 7, 2017 at 9:14 AM, Roi Apelker <roi.apel...@amdocs.com>
> wrote:
>
> > Hello,
> >
> > I am trying to look into the code of the continuous query mechanism 
> > - where the GEODE server sends the notification back to the client.
> >
> > Can anyone point me to the central classes of continuous query, 
> > especially to the one that is responsible for the calculation of the 
> > new data and packing it as a message back to the client?
> >
> > Thanks,
> >
> > Roi
> >
> > This message and the information contained herein is proprietary and 
> > confidential and subject to the Amdocs policy statement,
> >
> > you may review at https://www.amdocs.com/about/email-disclaimer < 
> > https://www.amdocs.com/about/email-disclaimer>
> >
> This message and the information contained herein is proprietary and 
> confidential and subject to the Amdocs policy statement,
>
> you may review at https://www.amdocs.com/about/email-disclaimer < 
> https://www.amdocs.com/about/email-disclaimer>
>
This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 
<https://www.amdocs.com/about/email-disclaimer>

RE: continuous query internal mechanism

[Roi Apelker]

Thanks,

By the way, is there any difference in the behaviour of the server, if the 
client that registered the CQ is a native (C++) client?

I have been going over the classes and code for some time and can't seem to 
find the actual location where a CQ update/notification is sent...

It's like CqEventImpl class is never even generated in this scenario.

If anyone can help here I would be most grateful :-)

Thanks

Roi



-Original Message-
From: Anilkumar Gingade [mailto:aging...@pivotal.io] 
Sent: Monday, August 07, 2017 8:23 PM
To: dev@geode.apache.org
Subject: Re: continuous query internal mechanism

You can find those in CqServiceImpl.process*()...

-Anil.


On Mon, Aug 7, 2017 at 9:14 AM, Roi Apelker <roi.apel...@amdocs.com> wrote:

> Hello,
>
> I am trying to look into the code of the continuous query mechanism - 
> where the GEODE server sends the notification back to the client.
>
> Can anyone point me to the central classes of continuous query, 
> especially to the one that is responsible for the calculation of the 
> new data and packing it as a message back to the client?
>
> Thanks,
>
> Roi
>
> This message and the information contained herein is proprietary and 
> confidential and subject to the Amdocs policy statement,
>
> you may review at https://www.amdocs.com/about/email-disclaimer < 
> https://www.amdocs.com/about/email-disclaimer>
>
This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 
<https://www.amdocs.com/about/email-disclaimer>

continuous query internal mechanism

[Roi Apelker]

Hello,

I am trying to look into the code of the continuous query mechanism - where the 
GEODE server sends the notification back to the client.

Can anyone point me to the central classes of continuous query, especially to 
the one that is responsible for the calculation of the new data and packing it 
as a message back to the client?

Thanks,

Roi

This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 

RE: Question regarding node failure scenario

[Roi Apelker]

6 3736 
> Support.Pivotal.io <http://support.pivotal.io/>  |  Mon-Fri  9:00am to 
> 5:30pm JST  |  1-877-477-2269
> [image: support] <https://support.pivotal.io/> [image: twitter] 
> <https://twitter.com/pivotal> [image: linkedin] 
> <https://www.linkedin.com/company/3048967> [image: facebook] 
> <https://www.facebook.com/pivotalsoftware> [image: google plus] 
> <https://plus.google.com/+Pivotal> [image: youtube] 
> <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv
> 3kl>
>
>
> 2017-07-23 22:04 GMT+09:00 Roi Apelker <roi.apel...@amdocs.com>:
>
> > Hi,  (Bear with me I am a bit new here :))
> >
> > I have the following scenario, I wonder if anyone can comment on it 
> > - is it a known issue, maybe it was solved already in later version, 
> > etc. (I
> am
> > using version 1.0.0)
> > Or maybe you can point me to somewhere in the code.
> > I have posted this question once before (thanks Dan S. for relating 
> > to it), however I was away for 3 weeks therefore posting again.
> >
> > - I have a replicated Node, say N1 and its replicated N2 (N2 gets 
> > activated when N1 is down) and they are configured to send updates 
> > via continuous query to my process which then reports on these updates.
> >
> > - N1 is working all the time and serves as a server, and data is 
> > written to it continuously from external clients. If 1000 events are 
> > written to
> N1,
> > the final report I am referring to will indicate 1000.
> >
> > - When N1 gets killed for any reason, the client connects to N2 
> > which continues to send the continuous query results.
> >
> > - But sometimes, the report is inaccurate, e.g. after running 1000
> events,
> > my report says 950, as if some events of the continuous query do not
> reach
> > the client (the actual data does arrive its destination, just the 
> > report
> is
> > qrong)
> >
> > The fact is, that the report is not accurate, and I only assume that 
> > something is wrong in the update mechanism, somewhere in the area of 
> > continuous query, or somewhere in the replication between the nodes.
> >
> > Right now, the parameter of subscription-redundancy is not configured.
> But
> > subscription-enabled="true" .
> >
> > What is the significance of not configuring the subscription-redundancy ?
> > does it mean that client disconnection may cause continuous query 
> > events
> to
> > be discarded?
> > And if so, is it "may be discarded" or "will be discarded", meaning, 
> > will the result be always the same or not?
> >
> >
> > Thanks,
> >
> > Roi
> > This message and the information contained herein is proprietary and 
> > confidential and subject to the Amdocs policy statement,
> >
> > you may review at https://www.amdocs.com/about/email-disclaimer < 
> > https://www.amdocs.com/about/email-disclaimer>
> >
> >
>
This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 
<https://www.amdocs.com/about/email-disclaimer>

Question regarding node failure scenario

[Roi Apelker]

Hi,  (Bear with me I am a bit new here :))

I have the following scenario, I wonder if anyone can comment on it - is it a 
known issue, maybe it was solved already in later version, etc. (I am using 
version 1.0.0)
Or maybe you can point me to somewhere in the code.
I have posted this question once before (thanks Dan S. for relating to it), 
however I was away for 3 weeks therefore posting again.

- I have a replicated Node, say N1 and its replicated N2 (N2 gets activated 
when N1 is down) and they are configured to send updates via continuous query 
to my process which then reports on these updates.

- N1 is working all the time and serves as a server, and data is written to it 
continuously from external clients. If 1000 events are written to N1, the final 
report I am referring to will indicate 1000.

- When N1 gets killed for any reason, the client connects to N2 which continues 
to send the continuous query results. 

- But sometimes, the report is inaccurate, e.g. after running 1000 events, my 
report says 950, as if some events of the continuous query do not reach the 
client (the actual data does arrive its destination, just the report is qrong)

The fact is, that the report is not accurate, and I only assume that something 
is wrong in the update mechanism, somewhere in the area of continuous query, or 
somewhere in the replication between the nodes.

Right now, the parameter of subscription-redundancy is not configured. But 
subscription-enabled="true" .

What is the significance of not configuring the subscription-redundancy ? does 
it mean that client disconnection may cause continuous query events to be 
discarded?
And if so, is it "may be discarded" or "will be discarded", meaning, will the 
result be always the same or not?


Thanks,

Roi
This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 

RE: Geode "minor minor" versions

[Roi Apelker]

I am sorry if I have been misunderstood (and yes I am new :-) )

I did not mean that GEODE is going to stop, I was referring to the subject of 
minor versions.

Is there any plan to release a 1.1.2, or a 1.2.1, or will this be simply an "ad 
hoc" situation, such that only an important bug fix will trigger it... I 
understand now that this is the case.

-Original Message-
From: Gregory Chase [mailto:gch...@pivotal.io] 
Sent: Wednesday, July 05, 2017 5:25 PM
To: dev@geode.apache.org
Subject: Re: Geode "minor minor" versions

You make it sound like Geode is going to stop. That's definitely not true.

What affects Geode's future are the requests the community makes for new 
features. Check out our Jira to know what's in our future.

Since this is an open source Apache community, all the planning occurs here in 
the open.  I'm trusting you are new to our community?

Updates get triggered as needed whenever there is an important bug to fix 
and/or foundational changes that need to be brought to production, even if they 
don't create new features right away.


On Tue, Jul 4, 2017 at 4:56 AM, Roi Apelker <roi.apel...@amdocs.com> wrote:

> Hi,
>
> I understood from previous discussions, that apart from version 1.3 
> there are no other foreseeable versions in the future.
>
>
> * What about "minor minor" versions - 1.1.2? 1.2.1? - what is the
> "trigger" for such versions?
>
> * Are fixes being released as well? Or is it just that their code
> is in github?
>
> Thanks,
> Roi
> This message and the information contained herein is proprietary and 
> confidential and subject to the Amdocs policy statement,
>
> you may review at https://www.amdocs.com/about/email-disclaimer < 
> https://www.amdocs.com/about/email-disclaimer>
>



--
Greg Chase

Product team
http://www.pivotal.io/big-data

Pivotal Software
http://www.pivotal.io/

650-215-0477
@GregChase
Blog: http://geekmarketing.biz/
This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 
<https://www.amdocs.com/about/email-disclaimer>

Geode "minor minor" versions

[Roi Apelker]

Hi,

I understood from previous discussions, that apart from version 1.3 there are 
no other foreseeable versions in the future.


* What about "minor minor" versions - 1.1.2? 1.2.1? - what is the 
"trigger" for such versions?

* Are fixes being released as well? Or is it just that their code is in 
github?

Thanks,
Roi
This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 

Question regarding node failure scenario

[Roi Apelker]

Hi,

I have the following scenario, I wonder if anyone can comment on it - is it a 
known issue, maybe it was solved already in later version, etc. (I am using 
version 1.0.0)

Bear with me I am a bit new here :)

I have a replicated Node, say N1 and its replicated N2 (N2 gets activated when 
N1 is down) and they are configured to send updates via continuous query to my 
process which then reports on these updates.

N1 is working all the time and serves as a server, and data is written to it 
continuously. If 1000 events are written to N1, the final report I am referring 
to will indicate 1000.

When N1 gets killed for any reason, sometimes the updates do get to my 
reporting process and the report is 100% accurate.

But sometimes, the report is inaccurate, e.g. after running 1000 events, my 
report says 950.

The fact is, that the report is not accurate, and I only assume that something 
is wrong in the update mechanism, somewhere in the area of continuous query, or 
somewhere in the replication between the nodes.

Has anyone encountered something similar and can share a thought? Maybe it was 
even solved in a later version?

Thanks

Roi
This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 

Question regarding the next release

[Roi Apelker]

Hi,

Is there an expected date for release of 1.2?

Are there any backward compatibility issues in this version? If so, are they 
documented separately from the other issues?

Thanks,

Roi
This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 

RE: GEODE and GEODE-native compatibility

[Roi Apelker]

Thanks Jake :-)

-Original Message-
From: Jacob Barrett [mailto:jbarr...@pivotal.io] 
Sent: Monday, June 05, 2017 4:07 PM
To: dev@geode.apache.org
Subject: Re: GEODE and GEODE-native compatibility

You asked the same question on the user list on 5/21. In case you didn't get my 
reply I'll send the same answer.

There have been no official Geode release of Geode Native but the current 
development branch is known to work with Geode server 1.0 and 1.1.

-Jake


Sent from my iPhone

> On Jun 5, 2017, at 2:49 AM, Roi Apelker <roi.apel...@amdocs.com> wrote:
> 
> Hi,
> 
> I am using both the GEODE and GEODE native libraries, and since the native 
> library has no numbered version I am wondering if there are any compatibility 
> issues.
> 
> Will the native library work with versions 1.0.0 and 1.1.1 seamlessly?
> 
> Thanks,
> 
> Roi
> 
> This message and the information contained herein is proprietary and 
> confidential and subject to the Amdocs policy statement,
> 
> you may review at https://www.amdocs.com/about/email-disclaimer 
> <https://www.amdocs.com/about/email-disclaimer>
This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 
<https://www.amdocs.com/about/email-disclaimer>

RE: Geode versions

[Roi Apelker]

Thanks Anthony,

Can you estimate, whether this will be a week/month/3 months for the release of 
1.2?

What about 1.3?

-Roi

-Original Message-
From: Anthony Baker [mailto:aba...@pivotal.io] 
Sent: Monday, June 05, 2017 7:34 PM
To: dev@geode.apache.org
Subject: Re: Geode versions

Hi Roi,

We have cut a release branch for 1.2.0.  Once we have fixed the remaining bugs 
we will vote on releasing.  We don’t published expected release dates but I 
expect that it won’t be too much longer.

Thanks,
Anthony

> On Jun 5, 2017, at 2:49 AM, Roi Apelker <roi.apel...@amdocs.com> wrote:
> 
> Hello,
> 
> Where can I find information on up-coming GEODE versions, release dates etc.?
> The last released version I know if is 1.1.1,
> 
> There is already a reference for 1.2 in the release notes here:
> https://cwiki.apache.org/confluence/display/GEODE/Release+Notes
> 
> Also, reference for 1.3 here:
> https://issues.apache.org/jira/browse/GEODE/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel
> 
> 
> But no dates... :(
> 
> Anyone knows?
> 
> Thanks,
> Roi
> 
> This message and the information contained herein is proprietary and 
> confidential and subject to the Amdocs policy statement,
> 
> you may review at https://www.amdocs.com/about/email-disclaimer 
> <https://www.amdocs.com/about/email-disclaimer>

This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 
<https://www.amdocs.com/about/email-disclaimer>

GEODE and GEODE-native compatibility

[Roi Apelker]

Hi,

I am using both the GEODE and GEODE native libraries, and since the native 
library has no numbered version I am wondering if there are any compatibility 
issues.

Will the native library work with versions 1.0.0 and 1.1.1 seamlessly?

Thanks,

Roi

This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 

Geode versions

[Roi Apelker]

Hello,

Where can I find information on up-coming GEODE versions, release dates etc.?
The last released version I know if is 1.1.1,

There is already a reference for 1.2 in the release notes here:
https://cwiki.apache.org/confluence/display/GEODE/Release+Notes

Also, reference for 1.3 here:
https://issues.apache.org/jira/browse/GEODE/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel


But no dates... :(

Anyone knows?

Thanks,
Roi

This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer