[jira] [Comment Edited] (ZOOKEEPER-1485) client xid overflow is not handled

[Michael Han (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-1485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15328927#comment-15328927
 ] 

Michael Han edited comment on ZOOKEEPER-1485 at 6/14/16 4:55 AM:
-

bq. I didn't see an existing pattern to add test-only code either. Do you have 
any suggestions?
I think zoo_get_current_server could be one existing example of a 'largely for 
testing purposes' code. 

bq. which could cause performance issues with multiple threads making requests 
with different zhandles.
Yeah this was what I thought about. Might not be an issue if the number of 
threads is capped. 

bq. Did you have a specific implementation in mind?
How about this (in C++, could be updated to use gcc intrinsics as well in C):
{code}
#include 
#include 

std::atomic count;

uint32_t get_count() {
uint32_t new_count;
uint32_t cur_count;
do {
uint32_t cur_count = count;
if (cur_count == std::numeric_limits::max()) {
new_count = 1;
} else {
new_count = cur_count + 1;
}
} while(!std::atomic_compare_exchange_weak(, _count, new_count));
return new_count;
}
{code}


was (Author: hanm):
bq. I didn't see an existing pattern to add test-only code either. Do you have 
any suggestions?
I think zoo_get_current_server could be one existing example of a 'largely for 
testing purposes' code. 

bq. which could cause performance issues with multiple threads making requests 
with different zhandles.
Yeah this was what I thought about. Might not be an issue if the number of 
threads is capped. 

bq. Did you have a specific implementation in mind?
How about this (in C++, could be updated to use gcc intrinsics as well in C):
{code}
#include 
#include 
using namespace std;

std::atomic count;

using namespace std;

uint32_t get_count() {
uint32_t new_count;
uint32_t cur_count;
do {
uint32_t cur_count = count;
if (cur_count == std::numeric_limits::max()) {
new_count = 1;
} else {
new_count = cur_count + 1;
}
} while(!std::atomic_compare_exchange_weak(, _count, new_count));
return new_count;
}
{code}

> client xid overflow is not handled
> --
>
> Key: ZOOKEEPER-1485
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1485
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: c client, java client
>Affects Versions: 3.4.3, 3.3.5
>Reporter: Michi Mutsuzaki
>Assignee: Martin Kuchta
> Attachments: ZOOKEEPER-1485.patch
>
>
> Both Java and C clients use signed 32-bit int as XIDs. XIDs are assumed to be 
> non-negative, and zookeeper uses some negative values as special XIDs (e.g. 
> -2 for ping, -4 for auth). However, neither Java nor C client ensures the 
> XIDs it generates are non-negative, and the server doesn't reject negative 
> XIDs.
> Pat had some suggestions on how to fix this:
> - (bin-compat) Expire the session when the client sends a negative XID.
> - (bin-incompat) In addition to expiring the session, use 64-bit int for XID 
> so that overflow will practically never happen.
> --Michi



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (ZOOKEEPER-1485) client xid overflow is not handled

[Michael Han (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-1485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15328927#comment-15328927
 ] 

Michael Han edited comment on ZOOKEEPER-1485 at 6/14/16 4:53 AM:
-

bq. I didn't see an existing pattern to add test-only code either. Do you have 
any suggestions?
I think zoo_get_current_server could be one existing example of a 'largely for 
testing purposes' code. 

bq. which could cause performance issues with multiple threads making requests 
with different zhandles.
Yeah this was what I thought about. Might not be an issue if the number of 
threads is capped. 

bq. Did you have a specific implementation in mind?
How about this (in C++, could be updated to use gcc intrinsics as well in C):
{code}
#include 
#include 
using namespace std;

std::atomic count;

using namespace std;

uint32_t get_count() {
uint32_t new_count;
uint32_t cur_count;
do {
uint32_t cur_count = count;
if (cur_count == std::numeric_limits::max()) {
new_count = 1;
} else {
new_count = cur_count + 1;
}
} while(!std::atomic_compare_exchange_weak(, _count, new_count));
return new_count;
}
{code}


was (Author: hanm):
bq. I didn't see an existing pattern to add test-only code either. Do you have 
any suggestions?
I think zoo_get_current_server could be one existing example of a 'largely for 
testing purposes' code. 

bq. which could cause performance issues with multiple threads making requests 
with different zhandles.
Yeah this was what I thought about. Might not be an issue if the number of 
threads is capped. 

bq. Did you have a specific implementation in mind?
How about this (in C++, could be updated to use gcc intrinsics as well in C):
{code}
#include 
#include 
using namespace std;

std::atomic count;

using namespace std;

uint32_t get_count() {
uint32_t new_count;
uint32_t cur_count;
do {
uint32_t cur_count = count;
if (cur_count == std::numeric_limits::max()) {
new_count = 1;
} else {
new_count = cur_count + 1;
}
} while(!std::atomic_compare_exchange_weak(, _count, new_count));
return new_count;
}
{code}
{code}

> client xid overflow is not handled
> --
>
> Key: ZOOKEEPER-1485
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1485
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: c client, java client
>Affects Versions: 3.4.3, 3.3.5
>Reporter: Michi Mutsuzaki
>Assignee: Martin Kuchta
> Attachments: ZOOKEEPER-1485.patch
>
>
> Both Java and C clients use signed 32-bit int as XIDs. XIDs are assumed to be 
> non-negative, and zookeeper uses some negative values as special XIDs (e.g. 
> -2 for ping, -4 for auth). However, neither Java nor C client ensures the 
> XIDs it generates are non-negative, and the server doesn't reject negative 
> XIDs.
> Pat had some suggestions on how to fix this:
> - (bin-compat) Expire the session when the client sends a negative XID.
> - (bin-incompat) In addition to expiring the session, use 64-bit int for XID 
> so that overflow will practically never happen.
> --Michi



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ZOOKEEPER-1485) client xid overflow is not handled

[Michael Han (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-1485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15328927#comment-15328927
 ] 

Michael Han commented on ZOOKEEPER-1485:


bq. I didn't see an existing pattern to add test-only code either. Do you have 
any suggestions?
I think zoo_get_current_server could be one existing example of a 'largely for 
testing purposes' code. 

bq. which could cause performance issues with multiple threads making requests 
with different zhandles.
Yeah this was what I thought about. Might not be an issue if the number of 
threads is capped. 

bq. Did you have a specific implementation in mind?
How about this (in C++, could be updated to use gcc intrinsics as well in C):
{code}
#include 
#include 
using namespace std;

std::atomic count;

using namespace std;

uint32_t get_count() {
uint32_t new_count;
uint32_t cur_count;
do {
uint32_t cur_count = count;
if (cur_count == std::numeric_limits::max()) {
new_count = 1;
} else {
new_count = cur_count + 1;
}
} while(!std::atomic_compare_exchange_weak(, _count, new_count));
return new_count;
}
{code}
{code}

> client xid overflow is not handled
> --
>
> Key: ZOOKEEPER-1485
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1485
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: c client, java client
>Affects Versions: 3.4.3, 3.3.5
>Reporter: Michi Mutsuzaki
>Assignee: Martin Kuchta
> Attachments: ZOOKEEPER-1485.patch
>
>
> Both Java and C clients use signed 32-bit int as XIDs. XIDs are assumed to be 
> non-negative, and zookeeper uses some negative values as special XIDs (e.g. 
> -2 for ping, -4 for auth). However, neither Java nor C client ensures the 
> XIDs it generates are non-negative, and the server doesn't reject negative 
> XIDs.
> Pat had some suggestions on how to fix this:
> - (bin-compat) Expire the session when the client sends a negative XID.
> - (bin-incompat) In addition to expiring the session, use 64-bit int for XID 
> so that overflow will practically never happen.
> --Michi



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ZOOKEEPER-1485) client xid overflow is not handled

[Martin Kuchta (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-1485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15328679#comment-15328679
 ] 

Martin Kuchta commented on ZOOKEEPER-1485:
--

I was thinking about the value of a test case around the overflow condition 
here. Like you said, we would need to add a set_xid function and move the xid 
variable outside the get_xid function to let both functions access it. I didn't 
see an existing pattern to add test-only code either. Do you have any 
suggestions?

As for the performance issue, that's also a valid point. I was a bit hesitant 
to replace the atomic operation at first, but I'm not sure it's actually an 
issue. The locking happens once per client request, the lock is held for a very 
short period of time, and each request already performs other locking on the ZK 
handle (enter_critical(), lock_buffer_list()). The one difference here might be 
that this is a global lock not tied to a particular zhandle, which could cause 
performance issues with multiple threads making requests with different 
zhandles.

I don't think you can correctly implement this using a plain CAS and atomic 
add. Any way you combine those operations, I think there's a chance the CAS 
won't trigger on the value you're checking against. Did you have a specific 
implementation in mind? I might be approaching it from the wrong angle when 
trying to reason about possible implementations.

> client xid overflow is not handled
> --
>
> Key: ZOOKEEPER-1485
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1485
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: c client, java client
>Affects Versions: 3.4.3, 3.3.5
>Reporter: Michi Mutsuzaki
>Assignee: Martin Kuchta
> Attachments: ZOOKEEPER-1485.patch
>
>
> Both Java and C clients use signed 32-bit int as XIDs. XIDs are assumed to be 
> non-negative, and zookeeper uses some negative values as special XIDs (e.g. 
> -2 for ping, -4 for auth). However, neither Java nor C client ensures the 
> XIDs it generates are non-negative, and the server doesn't reject negative 
> XIDs.
> Pat had some suggestions on how to fix this:
> - (bin-compat) Expire the session when the client sends a negative XID.
> - (bin-incompat) In addition to expiring the session, use 64-bit int for XID 
> so that overflow will practically never happen.
> --Michi



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

ZooKeeper-trunk - Build # 2959 - Failure

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper-trunk/2959/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 385306 lines...]
[junit] 2016-06-13 23:28:06,916 [myid:] - INFO  [main:ClientBase@562] - 
fdcount after test is: 47 at start it was 47
[junit] 2016-06-13 23:28:06,917 [myid:] - INFO  [main:ZKTestCase$1@65] - 
SUCCEEDED testWatcherAutoResetWithLocal
[junit] 2016-06-13 23:28:06,917 [myid:] - INFO  [main:ZKTestCase$1@60] - 
FINISHED testWatcherAutoResetWithLocal
[junit] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
28.934 sec, Thread: 1, Class: org.apache.zookeeper.test.WatcherTest
[junit] 2016-06-13 23:30:01,759 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@82] - Memory used 26511
[junit] 2016-06-13 23:30:01,759 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@87] - Number of threads 55
[junit] 2016-06-13 23:30:01,759 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@102] - FINISHED TEST METHOD 
testManyChildWatchersAutoReset
[junit] 2016-06-13 23:30:01,759 [myid:] - INFO  [main:ClientBase@537] - 
tearDown starting
[junit] 2016-06-13 23:30:01,760 [myid:] - INFO  [ProcessThread(sid:0 
cport:16611)::PrepRequestProcessor@647] - Processed session termination for 
sessionid: 0x100a4fd0210
[junit] 2016-06-13 23:30:01,767 [myid:] - INFO  
[NIOWorkerThread-24:MBeanRegistry@128] - Unregister MBean 
[org.apache.ZooKeeperService:name0=StandaloneServer_port16611,name1=Connections,name2=127.0.0.1,name3=0x100a4fd0210]
[junit] 2016-06-13 23:30:01,767 [myid:] - INFO  [main:ZooKeeper@1313] - 
Session: 0x100a4fd0210 closed
[junit] 2016-06-13 23:30:01,768 [myid:] - INFO  
[NIOWorkerThread-24:NIOServerCnxn@607] - Closed socket connection for client 
/127.0.0.1:52724 which had sessionid 0x100a4fd0210
[junit] 2016-06-13 23:30:01,767 [myid:] - INFO  
[main-EventThread:ClientCnxn$EventThread@513] - EventThread shut down for 
session: 0x100a4fd0210
[junit] 2016-06-13 23:30:01,768 [myid:] - INFO  [ProcessThread(sid:0 
cport:16611)::PrepRequestProcessor@647] - Processed session termination for 
sessionid: 0x100a4fd0211
[junit] 2016-06-13 23:30:01,775 [myid:] - INFO  
[main-EventThread:ClientCnxn$EventThread@513] - EventThread shut down for 
session: 0x100a4fd0211
[junit] 2016-06-13 23:30:01,775 [myid:] - INFO  
[NIOWorkerThread-28:MBeanRegistry@128] - Unregister MBean 
[org.apache.ZooKeeperService:name0=StandaloneServer_port16611,name1=Connections,name2=127.0.0.1,name3=0x100a4fd0211]
[junit] 2016-06-13 23:30:01,776 [myid:] - INFO  [main:ZooKeeper@1313] - 
Session: 0x100a4fd0211 closed
[junit] 2016-06-13 23:30:01,777 [myid:] - INFO  [main:ClientBase@507] - 
STOPPING server
[junit] 2016-06-13 23:30:01,776 [myid:] - INFO  
[NIOWorkerThread-28:NIOServerCnxn@607] - Closed socket connection for client 
/127.0.0.1:52720 which had sessionid 0x100a4fd0211
[junit] 2016-06-13 23:30:01,777 [myid:] - INFO  
[ConnnectionExpirer:NIOServerCnxnFactory$ConnectionExpirerThread@583] - 
ConnnectionExpirerThread interrupted
[junit] 2016-06-13 23:30:01,781 [myid:] - INFO  
[NIOServerCxnFactory.AcceptThread:0.0.0.0/0.0.0.0:16611:NIOServerCnxnFactory$AcceptThread@219]
 - accept thread exitted run method
[junit] 2016-06-13 23:30:01,781 [myid:] - INFO  
[NIOServerCxnFactory.SelectorThread-0:NIOServerCnxnFactory$SelectorThread@420] 
- selector thread exitted run method
[junit] 2016-06-13 23:30:01,782 [myid:] - INFO  
[NIOServerCxnFactory.SelectorThread-1:NIOServerCnxnFactory$SelectorThread@420] 
- selector thread exitted run method
[junit] 2016-06-13 23:30:01,782 [myid:] - INFO  [main:ZooKeeperServer@498] 
- shutting down
[junit] 2016-06-13 23:30:01,782 [myid:] - INFO  
[main:SessionTrackerImpl@232] - Shutting down
[junit] 2016-06-13 23:30:01,782 [myid:] - INFO  
[main:PrepRequestProcessor@965] - Shutting down
[junit] 2016-06-13 23:30:01,783 [myid:] - INFO  
[main:SyncRequestProcessor@191] - Shutting down
[junit] 2016-06-13 23:30:01,783 [myid:] - INFO  [ProcessThread(sid:0 
cport:16611)::PrepRequestProcessor@154] - PrepRequestProcessor exited loop!
[junit] 2016-06-13 23:30:01,783 [myid:] - INFO  
[SyncThread:0:SyncRequestProcessor@169] - SyncRequestProcessor exited!
[junit] 2016-06-13 23:30:01,783 [myid:] - INFO  
[main:FinalRequestProcessor@479] - shutdown of request processor complete
[junit] 2016-06-13 23:30:01,784 [myid:] - INFO  [main:MBeanRegistry@128] - 
Unregister MBean 
[org.apache.ZooKeeperService:name0=StandaloneServer_port16611,name1=InMemoryDataTree]
[junit] 2016-06-13 23:30:01,784 [myid:] - INFO  [main:MBeanRegistry@128] - 
Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port16611]
[junit] 2016-06-13 23:30:01,784 [myid:] - INFO  
[main:FourLetterWordMain@85] - 

[jira] [Commented] (ZOOKEEPER-1485) client xid overflow is not handled

[Michael Han (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-1485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15328384#comment-15328384
 ] 

Michael Han commented on ZOOKEEPER-1485:


Should we add a test case about this if there is no existing tests? We could 
add a "test only" method like set_xid if it is hard to simulate the overflow 
condition in a test. 

I am also wondering what is the performance implication by changing get_xid 
from using a compiler intrinsic / atomic operation to use explicit locks. I 
think it is possible to implement the same 'wrap if overflow' operation using 
CAS. Generally lock free atomic operation provides better performance comparing 
to locks, but it of course depends on the use cases such as the level of 
contention / number of threads, so without benchmark in a specific context it 
might hard to tell. Not saying we should over optimize, just want to bring this 
up see if this is a real concern or not.



> client xid overflow is not handled
> --
>
> Key: ZOOKEEPER-1485
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1485
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: c client, java client
>Affects Versions: 3.4.3, 3.3.5
>Reporter: Michi Mutsuzaki
>Assignee: Martin Kuchta
> Attachments: ZOOKEEPER-1485.patch
>
>
> Both Java and C clients use signed 32-bit int as XIDs. XIDs are assumed to be 
> non-negative, and zookeeper uses some negative values as special XIDs (e.g. 
> -2 for ping, -4 for auth). However, neither Java nor C client ensures the 
> XIDs it generates are non-negative, and the server doesn't reject negative 
> XIDs.
> Pat had some suggestions on how to fix this:
> - (bin-compat) Expire the session when the client sends a negative XID.
> - (bin-incompat) In addition to expiring the session, use 64-bit int for XID 
> so that overflow will practically never happen.
> --Michi



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ZOOKEEPER-1485) client xid overflow is not handled

[Martin Kuchta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-1485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Martin Kuchta updated ZOOKEEPER-1485:
-
Attachment: ZOOKEEPER-1485.patch

I am uploading a patch that takes the approach of wrapping the XID around when 
the max value is reached. The Java client and both multi- and single-threaded C 
implementations have been modified. They also now all initialize the XID to 1 
to match the Java client's behavior (the C implementations used epoch time 
before), though I don't think it matters.

We're likely going to deploy this fix internally, since we're hitting the 
overflow issue somewhat frequently given the number of ZooKeeper deployments we 
have and the volume of requests they're processing. It sounds like [~fanster.z] 
fixed the issue in a similar way.

We can discuss whether mainline ZooKeeper should take a different approach 
toward fixing this based on the points made above.

> client xid overflow is not handled
> --
>
> Key: ZOOKEEPER-1485
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1485
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: c client, java client
>Affects Versions: 3.4.3, 3.3.5
>Reporter: Michi Mutsuzaki
>Assignee: Martin Kuchta
> Attachments: ZOOKEEPER-1485.patch
>
>
> Both Java and C clients use signed 32-bit int as XIDs. XIDs are assumed to be 
> non-negative, and zookeeper uses some negative values as special XIDs (e.g. 
> -2 for ping, -4 for auth). However, neither Java nor C client ensures the 
> XIDs it generates are non-negative, and the server doesn't reject negative 
> XIDs.
> Pat had some suggestions on how to fix this:
> - (bin-compat) Expire the session when the client sends a negative XID.
> - (bin-incompat) In addition to expiring the session, use 64-bit int for XID 
> so that overflow will practically never happen.
> --Michi



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

ZooKeeper_branch34_openjdk7 - Build # 1107 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper_branch34_openjdk7/1107/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 210766 lines...]
[junit] 2016-06-13 15:27:34,305 [myid:] - INFO  [main:JMXEnv@246] - 
expect:StandaloneServer_port
[junit] 2016-06-13 15:27:34,305 [myid:] - INFO  [main:JMXEnv@250] - 
found:StandaloneServer_port 
org.apache.ZooKeeperService:name0=StandaloneServer_port11221
[junit] 2016-06-13 15:27:34,306 [myid:] - INFO  [main:ClientBase@490] - 
STOPPING server
[junit] 2016-06-13 15:27:34,306 [myid:] - INFO  
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxnFactory@219] - 
NIOServerCnxn factory exited run method
[junit] 2016-06-13 15:27:34,306 [myid:] - INFO  [main:ZooKeeperServer@469] 
- shutting down
[junit] 2016-06-13 15:27:34,307 [myid:] - INFO  
[main:SessionTrackerImpl@225] - Shutting down
[junit] 2016-06-13 15:27:34,307 [myid:] - INFO  
[main:PrepRequestProcessor@765] - Shutting down
[junit] 2016-06-13 15:27:34,307 [myid:] - INFO  
[main:SyncRequestProcessor@209] - Shutting down
[junit] 2016-06-13 15:27:34,307 [myid:] - INFO  [ProcessThread(sid:0 
cport:11221)::PrepRequestProcessor@143] - PrepRequestProcessor exited loop!
[junit] 2016-06-13 15:27:34,307 [myid:] - INFO  
[SyncThread:0:SyncRequestProcessor@187] - SyncRequestProcessor exited!
[junit] 2016-06-13 15:27:34,308 [myid:] - INFO  
[main:FinalRequestProcessor@402] - shutdown of request processor complete
[junit] 2016-06-13 15:27:34,308 [myid:] - INFO  
[main:FourLetterWordMain@62] - connecting to 127.0.0.1 11221
[junit] 2016-06-13 15:27:34,309 [myid:] - INFO  [main:JMXEnv@146] - 
ensureOnly:[]
[junit] 2016-06-13 15:27:34,311 [myid:] - INFO  [main:ClientBase@443] - 
STARTING server
[junit] 2016-06-13 15:27:34,311 [myid:] - INFO  [main:ClientBase@364] - 
CREATING server instance 127.0.0.1:11221
[junit] 2016-06-13 15:27:34,311 [myid:] - INFO  
[main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:11221
[junit] 2016-06-13 15:27:34,312 [myid:] - INFO  [main:ClientBase@339] - 
STARTING server instance 127.0.0.1:11221
[junit] 2016-06-13 15:27:34,312 [myid:] - INFO  [main:ZooKeeperServer@170] 
- Created server with tickTime 3000 minSessionTimeout 6000 maxSessionTimeout 
6 datadir 
/x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_openjdk7/branch-3.4/build/test/tmp/test4760210756161738102.junit.dir/version-2
 snapdir 
/x1/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_openjdk7/branch-3.4/build/test/tmp/test4760210756161738102.junit.dir/version-2
[junit] 2016-06-13 15:27:34,317 [myid:] - INFO  
[main:FourLetterWordMain@62] - connecting to 127.0.0.1 11221
[junit] 2016-06-13 15:27:34,318 [myid:] - INFO  
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxnFactory@192] - 
Accepted socket connection from /127.0.0.1:44865
[junit] 2016-06-13 15:27:34,318 [myid:] - INFO  
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn@827] - Processing 
stat command from /127.0.0.1:44865
[junit] 2016-06-13 15:27:34,319 [myid:] - INFO  
[Thread-4:NIOServerCnxn$StatCommand@663] - Stat command output
[junit] 2016-06-13 15:27:34,319 [myid:] - INFO  
[Thread-4:NIOServerCnxn@1008] - Closed socket connection for client 
/127.0.0.1:44865 (no session established for client)
[junit] 2016-06-13 15:27:34,319 [myid:] - INFO  [main:JMXEnv@229] - 
ensureParent:[InMemoryDataTree, StandaloneServer_port]
[junit] 2016-06-13 15:27:34,322 [myid:] - INFO  [main:JMXEnv@246] - 
expect:InMemoryDataTree
[junit] 2016-06-13 15:27:34,322 [myid:] - INFO  [main:JMXEnv@250] - 
found:InMemoryDataTree 
org.apache.ZooKeeperService:name0=StandaloneServer_port11221,name1=InMemoryDataTree
[junit] 2016-06-13 15:27:34,322 [myid:] - INFO  [main:JMXEnv@246] - 
expect:StandaloneServer_port
[junit] 2016-06-13 15:27:34,322 [myid:] - INFO  [main:JMXEnv@250] - 
found:StandaloneServer_port 
org.apache.ZooKeeperService:name0=StandaloneServer_port11221
[junit] 2016-06-13 15:27:34,323 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@58] - Memory used 26415
[junit] 2016-06-13 15:27:34,323 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@63] - Number of threads 20
[junit] 2016-06-13 15:27:34,323 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@78] - FINISHED TEST METHOD testQuota
[junit] 2016-06-13 15:27:34,324 [myid:] - INFO  [main:ClientBase@520] - 
tearDown starting
[junit] 2016-06-13 15:27:34,388 [myid:] - INFO  [main:ZooKeeper@684] - 
Session: 0x1554a60021c closed
[junit] 2016-06-13 15:27:34,388 [myid:] - INFO  
[main-EventThread:ClientCnxn$EventThread@519] - EventThread shut down for 
session: 0x1554a60021c
[junit] 2016-06-13 15:27:34,388 [myid:] - INFO  [main:ClientBase@490] - 
STOPPING server
[junit] 2016-06-13 15:27:34,388 [myid:] - INFO  

Re: [jira] [Commented] (ZOOKEEPER-1044) Allow dynamic changes to roles of a peer

[Alexander Shraer]

You can issue a reconfig -add command where that server keeps all current
parameters except it's role. See manual:
https://zookeeper.apache.org/doc/trunk/zookeeperReconfig.html

On Jun 13, 2016 5:21 PM, "Nomar Morado (JIRA)"  wrote:

>
> [
> https://issues.apache.org/jira/browse/ZOOKEEPER-1044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327467#comment-15327467
> ]
>
> Nomar Morado commented on ZOOKEEPER-1044:
> -
>
> I am looking at QuorumPeer and trying to figure out how to get ZooKeeper
> to be able to execute the reconfig API to change role of a peer.
>
> It does not seem straightforward.
>
> I have 4 nodes - one is observer - and in some cases i wanted to be able
> to promote it to participant and vice versa.
>
> Can someone please help?
>
> Thanks.
>
>
>
> > Allow dynamic changes to roles of a peer
> > 
> >
> > Key: ZOOKEEPER-1044
> > URL:
> https://issues.apache.org/jira/browse/ZOOKEEPER-1044
> > Project: ZooKeeper
> >  Issue Type: Sub-task
> >  Components: quorum
> >Affects Versions: 3.3.0
> >Reporter: Vishal Kher
> >Assignee: Alexander Shraer
> > Fix For: 3.5.0
> >
> >
> > Requirement: functionality that will reconfigure
> > a OBSERVER to become a voting member and vice versa.
> > Example of usage:
> > 1. Maintain the Quorum size without changing the cluster size - in a 5
> > node cluster with 2 observers, I decide to decommission a voting
> > member. Then, I would like to configure one of my observers to be a
> > follower without any down time.
> > 2. Added a new server to the cluster that has better resources than
> > one of the voting peers. Make the new node as voting peer and the old
> > one as observer.
> > 3. Reduce the size of voting member for performance reasons.
> > Fix to ZOOKEEPER-107 might automatically give us this functionality.
> > It will be good to confirm that, and if needed, highlight work
> > that might be needed in addition to ZOOKEEPER-107.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
>

[jira] [Commented] (ZOOKEEPER-1044) Allow dynamic changes to roles of a peer

[Nomar Morado (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-1044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327467#comment-15327467
 ] 

Nomar Morado commented on ZOOKEEPER-1044:
-

I am looking at QuorumPeer and trying to figure out how to get ZooKeeper to be 
able to execute the reconfig API to change role of a peer.

It does not seem straightforward.

I have 4 nodes - one is observer - and in some cases i wanted to be able to 
promote it to participant and vice versa.

Can someone please help?

Thanks. 



> Allow dynamic changes to roles of a peer
> 
>
> Key: ZOOKEEPER-1044
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1044
> Project: ZooKeeper
>  Issue Type: Sub-task
>  Components: quorum
>Affects Versions: 3.3.0
>Reporter: Vishal Kher
>Assignee: Alexander Shraer
> Fix For: 3.5.0
>
>
> Requirement: functionality that will reconfigure
> a OBSERVER to become a voting member and vice versa.
> Example of usage:
> 1. Maintain the Quorum size without changing the cluster size - in a 5
> node cluster with 2 observers, I decide to decommission a voting
> member. Then, I would like to configure one of my observers to be a
> follower without any down time.
> 2. Added a new server to the cluster that has better resources than
> one of the voting peers. Make the new node as voting peer and the old
> one as observer.
> 3. Reduce the size of voting member for performance reasons.
> Fix to ZOOKEEPER-107 might automatically give us this functionality.
> It will be good to confirm that, and if needed, highlight work
> that might be needed in addition to ZOOKEEPER-107.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

ZooKeeper-trunk-jdk8 - Build # 633 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper-trunk-jdk8/633/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 392044 lines...]
[junit] 2016-06-13 11:53:13,528 [myid:] - INFO  [main:ZooKeeperServer@498] 
- shutting down
[junit] 2016-06-13 11:53:13,528 [myid:] - INFO  
[main:SessionTrackerImpl@232] - Shutting down
[junit] 2016-06-13 11:53:13,528 [myid:] - INFO  
[main:PrepRequestProcessor@965] - Shutting down
[junit] 2016-06-13 11:53:13,528 [myid:] - INFO  
[main:SyncRequestProcessor@191] - Shutting down
[junit] 2016-06-13 11:53:13,528 [myid:] - INFO  [ProcessThread(sid:0 
cport:11228)::PrepRequestProcessor@154] - PrepRequestProcessor exited loop!
[junit] 2016-06-13 11:53:13,528 [myid:] - INFO  
[SyncThread:0:SyncRequestProcessor@169] - SyncRequestProcessor exited!
[junit] 2016-06-13 11:53:13,528 [myid:] - INFO  
[main:FinalRequestProcessor@479] - shutdown of request processor complete
[junit] 2016-06-13 11:53:13,528 [myid:] - INFO  [main:MBeanRegistry@128] - 
Unregister MBean 
[org.apache.ZooKeeperService:name0=StandaloneServer_port11228,name1=InMemoryDataTree]
[junit] 2016-06-13 11:53:13,529 [myid:] - INFO  [main:MBeanRegistry@128] - 
Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port11228]
[junit] 2016-06-13 11:53:13,529 [myid:] - INFO  
[main:FourLetterWordMain@85] - connecting to 127.0.0.1 11228
[junit] 2016-06-13 11:53:13,529 [myid:] - INFO  [main:JMXEnv@146] - 
ensureOnly:[]
[junit] 2016-06-13 11:53:13,532 [myid:] - INFO  [main:ClientBase@562] - 
fdcount after test is: 51 at start it was 51
[junit] 2016-06-13 11:53:13,532 [myid:] - INFO  [main:ZKTestCase$1@65] - 
SUCCEEDED testWatcherAutoResetWithLocal
[junit] 2016-06-13 11:53:13,532 [myid:] - INFO  [main:ZKTestCase$1@60] - 
FINISHED testWatcherAutoResetWithLocal
[junit] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
29.542 sec, Thread: 1, Class: org.apache.zookeeper.test.WatcherTest
[junit] 2016-06-13 11:55:11,434 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@82] - Memory used 66378
[junit] 2016-06-13 11:55:11,435 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@87] - Number of threads 55
[junit] 2016-06-13 11:55:11,435 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@102] - FINISHED TEST METHOD 
testManyChildWatchersAutoReset
[junit] 2016-06-13 11:55:11,435 [myid:] - INFO  [main:ClientBase@537] - 
tearDown starting
[junit] 2016-06-13 11:55:11,436 [myid:] - INFO  [ProcessThread(sid:0 
cport:13918)::PrepRequestProcessor@647] - Processed session termination for 
sessionid: 0x100a2cc3416
[junit] 2016-06-13 11:55:11,443 [myid:] - INFO  [main:ZooKeeper@1313] - 
Session: 0x100a2cc3416 closed
[junit] 2016-06-13 11:55:11,443 [myid:] - INFO  
[NIOWorkerThread-31:MBeanRegistry@128] - Unregister MBean 
[org.apache.ZooKeeperService:name0=StandaloneServer_port13918,name1=Connections,name2=127.0.0.1,name3=0x100a2cc3416]
[junit] 2016-06-13 11:55:11,443 [myid:] - INFO  
[main-EventThread:ClientCnxn$EventThread@513] - EventThread shut down for 
session: 0x100a2cc3416
[junit] 2016-06-13 11:55:11,443 [myid:] - INFO  [ProcessThread(sid:0 
cport:13918)::PrepRequestProcessor@647] - Processed session termination for 
sessionid: 0x100a2cc34160001
[junit] 2016-06-13 11:55:11,444 [myid:] - INFO  
[NIOWorkerThread-31:NIOServerCnxn@607] - Closed socket connection for client 
/127.0.0.1:47000 which had sessionid 0x100a2cc3416
[junit] 2016-06-13 11:55:11,451 [myid:] - INFO  [main:ZooKeeper@1313] - 
Session: 0x100a2cc34160001 closed
[junit] 2016-06-13 11:55:11,451 [myid:] - INFO  [main:ClientBase@507] - 
STOPPING server
[junit] 2016-06-13 11:55:11,451 [myid:] - INFO  
[NIOWorkerThread-5:MBeanRegistry@128] - Unregister MBean 
[org.apache.ZooKeeperService:name0=StandaloneServer_port13918,name1=Connections,name2=127.0.0.1,name3=0x100a2cc34160001]
[junit] 2016-06-13 11:55:11,452 [myid:] - INFO  
[ConnnectionExpirer:NIOServerCnxnFactory$ConnectionExpirerThread@583] - 
ConnnectionExpirerThread interrupted
[junit] 2016-06-13 11:55:11,451 [myid:] - INFO  
[main-EventThread:ClientCnxn$EventThread@513] - EventThread shut down for 
session: 0x100a2cc34160001
[junit] 2016-06-13 11:55:11,452 [myid:] - INFO  
[NIOWorkerThread-5:NIOServerCnxn@607] - Closed socket connection for client 
/127.0.0.1:47019 which had sessionid 0x100a2cc34160001
[junit] 2016-06-13 11:55:11,454 [myid:] - INFO  
[NIOServerCxnFactory.AcceptThread:0.0.0.0/0.0.0.0:13918:NIOServerCnxnFactory$AcceptThread@219]
 - accept thread exitted run method
[junit] 2016-06-13 11:55:11,457 [myid:] - INFO  
[NIOServerCxnFactory.SelectorThread-1:NIOServerCnxnFactory$SelectorThread@420] 
- selector thread exitted run method
[junit] 2016-06-13 11:55:11,457 [myid:] - INFO  

ZooKeeper-trunk-solaris - Build # 1193 - Failure

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper-trunk-solaris/1193/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 351180 lines...]
[junit] 2016-06-13 08:14:28,107 [myid:] - INFO  [main:JMXEnv@146] - 
ensureOnly:[]
[junit] 2016-06-13 08:14:28,108 [myid:] - INFO  [main:ClientBase@460] - 
STARTING server
[junit] 2016-06-13 08:14:28,108 [myid:] - INFO  [main:ClientBase@380] - 
CREATING server instance 127.0.0.1:11222
[junit] 2016-06-13 08:14:28,108 [myid:] - INFO  
[main:NIOServerCnxnFactory@673] - Configuring NIO connection handler with 10s 
sessionless connection timeout, 2 selector thread(s), 16 worker threads, and 64 
kB direct buffers.
[junit] 2016-06-13 08:14:28,110 [myid:] - INFO  
[main:NIOServerCnxnFactory@686] - binding to port 0.0.0.0/0.0.0.0:11222
[junit] 2016-06-13 08:14:28,110 [myid:] - INFO  [main:ClientBase@355] - 
STARTING server instance 127.0.0.1:11222
[junit] 2016-06-13 08:14:28,110 [myid:] - INFO  [main:ZooKeeperServer@858] 
- minSessionTimeout set to 6000
[junit] 2016-06-13 08:14:28,111 [myid:] - INFO  [main:ZooKeeperServer@867] 
- maxSessionTimeout set to 6
[junit] 2016-06-13 08:14:28,111 [myid:] - INFO  [main:ZooKeeperServer@156] 
- Created server with tickTime 3000 minSessionTimeout 6000 maxSessionTimeout 
6 datadir 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper-trunk-solaris/trunk/build/test/tmp/test8113626091555253991.junit.dir/version-2
 snapdir 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper-trunk-solaris/trunk/build/test/tmp/test8113626091555253991.junit.dir/version-2
[junit] 2016-06-13 08:14:28,112 [myid:] - INFO  [main:FileSnap@83] - 
Reading snapshot 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper-trunk-solaris/trunk/build/test/tmp/test8113626091555253991.junit.dir/version-2/snapshot.b
[junit] 2016-06-13 08:14:28,114 [myid:] - INFO  [main:FileTxnSnapLog@298] - 
Snapshotting: 0xb to 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper-trunk-solaris/trunk/build/test/tmp/test8113626091555253991.junit.dir/version-2/snapshot.b
[junit] 2016-06-13 08:14:28,115 [myid:] - INFO  
[main:FourLetterWordMain@85] - connecting to 127.0.0.1 11222
[junit] 2016-06-13 08:14:28,116 [myid:] - INFO  
[NIOServerCxnFactory.AcceptThread:0.0.0.0/0.0.0.0:11222:NIOServerCnxnFactory$AcceptThread@296]
 - Accepted socket connection from /127.0.0.1:49243
[junit] 2016-06-13 08:14:28,116 [myid:] - INFO  
[NIOWorkerThread-1:NIOServerCnxn@485] - Processing stat command from 
/127.0.0.1:49243
[junit] 2016-06-13 08:14:28,117 [myid:] - INFO  
[NIOWorkerThread-1:StatCommand@49] - Stat command output
[junit] 2016-06-13 08:14:28,117 [myid:] - INFO  
[NIOWorkerThread-1:NIOServerCnxn@607] - Closed socket connection for client 
/127.0.0.1:49243 (no session established for client)
[junit] 2016-06-13 08:14:28,117 [myid:] - INFO  [main:JMXEnv@228] - 
ensureParent:[InMemoryDataTree, StandaloneServer_port]
[junit] 2016-06-13 08:14:28,119 [myid:] - INFO  [main:JMXEnv@245] - 
expect:InMemoryDataTree
[junit] 2016-06-13 08:14:28,119 [myid:] - INFO  [main:JMXEnv@249] - 
found:InMemoryDataTree 
org.apache.ZooKeeperService:name0=StandaloneServer_port11222,name1=InMemoryDataTree
[junit] 2016-06-13 08:14:28,119 [myid:] - INFO  [main:JMXEnv@245] - 
expect:StandaloneServer_port
[junit] 2016-06-13 08:14:28,119 [myid:] - INFO  [main:JMXEnv@249] - 
found:StandaloneServer_port 
org.apache.ZooKeeperService:name0=StandaloneServer_port11222
[junit] 2016-06-13 08:14:28,119 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@82] - Memory used 17717
[junit] 2016-06-13 08:14:28,120 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@87] - Number of threads 24
[junit] 2016-06-13 08:14:28,120 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@102] - FINISHED TEST METHOD 
testQuota
[junit] 2016-06-13 08:14:28,120 [myid:] - INFO  [main:ClientBase@537] - 
tearDown starting
[junit] 2016-06-13 08:14:28,192 [myid:] - INFO  [main:ZooKeeper@1313] - 
Session: 0x121c5d0 closed
[junit] 2016-06-13 08:14:28,192 [myid:] - INFO  
[main-EventThread:ClientCnxn$EventThread@513] - EventThread shut down for 
session: 0x121c5d0
[junit] 2016-06-13 08:14:28,192 [myid:] - INFO  [main:ClientBase@507] - 
STOPPING server
[junit] 2016-06-13 08:14:28,192 [myid:] - INFO  
[ConnnectionExpirer:NIOServerCnxnFactory$ConnectionExpirerThread@583] - 
ConnnectionExpirerThread interrupted
[junit] 2016-06-13 08:14:28,192 [myid:] - INFO  
[NIOServerCxnFactory.AcceptThread:0.0.0.0/0.0.0.0:11222:NIOServerCnxnFactory$AcceptThread@219]
 - accept thread exitted run method
[junit] 2016-06-13 08:14:28,193 [myid:] - INFO