Hi Frank,
I'm struggling with a performance issue when querying large numbers of devices
from a single process.
If, for example, I simultaneously issue 20 separate 'get' requests to each of
100 different addresses and repeatedly do that on a half minute cycle, I see
the CPU usage of my process plateau at a high level. If I increase the number
of devices, I don't see much more of an increase in CPU but the time taken to
fulfill each request increases on a roughly linear scale in relation to the
number of devices/requests.
I have tried using more than one Snmp instance with (as I expected) no obvious
benefit and I have tried asynchronous requests (where I can reduce CPU load by
introducing a wait for a response but this comes at the expense of overall
response times for a given set of 'gets').
Is it that ultimately all the responses to my many 'get' requests must arrive
at a single port (161) on my network interface and that the Snmp reading thread
can only process them so fast? Is there something I can do to increase
throughput short of bulking up my queries?
Any help appreciated.
Thanks
Jp
________________________________
From: "[email protected]" <[email protected]>
To: [email protected]
Sent: Monday, 3 June 2013, 20:43
Subject: SNMP4J Digest, Vol 111, Issue 3
Send SNMP4J mailing list submissions to
[email protected]
To subscribe or unsubscribe via the World Wide Web, visit
http://lists.agentpp.org/mailman/listinfo/snmp4j
or, via email, send a message with subject or body 'help' to
[email protected]
You can reach the person managing the list at
[email protected]
When replying, please edit your Subject line so it is more specific
than "Re: Contents of SNMP4J digest..."
Today's Topics:
1. Re: Performance hotspot in Cipher.getInstance() method call
(Praveen Jain)
2. Re: V3 Not In Time Window and Client Clock Drift (Elise Atkins)
3. Re: V3 Not In Time Window and Client Clock Drift (Frank Fock)
4. Re: V3 Not In Time Window and Client Clock Drift (Elise Atkins)
----------------------------------------------------------------------
Message: 1
Date: Mon, 3 Jun 2013 18:24:19 +0530
From: "Praveen Jain" <[email protected]>
To: "'Frank Fock'" <[email protected]>, <[email protected]>
Subject: Re: [SNMP4J] Performance hotspot in Cipher.getInstance()
method call
Message-ID: <[email protected]>
Content-Type: text/plain; charset="us-ascii"
Hi Frank,
Thanks a lot for quick reply. I would really appreciate if you backport the
fix into 1.x branch. Please let me know if there is a place where I can
track that (any issue tracking system). In case the backporting idea is
dropped for 1.x branch, please provide a patch so that I can apply the same
on source code. Thanks.
Regards,
Praveen.
-----Original Message-----
From: [email protected] [mailto:[email protected]] On
Behalf Of Frank Fock
Sent: Monday, June 03, 2013 2:59 AM
To: [email protected]
Subject: Re: [SNMP4J] Performance hotspot in Cipher.getInstance() method
call
Hi Praveen,
SNMP4J is not caching the cipher object. Apparently, the performance
overhead of retrieving a Cipher instance is not that great with newer Java
versions.
Nevertheless, I am currently working on a caching solution, which does not
introduce a new bootleneek for multi-threaded usage for SNMP4J 2.x.
If it will possible, I will also provide a backport to the 1.x branch.
Best regards,
Frank
Am 30.05.2013 13:03, schrieb Praveen Jain:
We are using SNMP4J version 1.11.4 for a legacy application running in
Java
1.4 environment. The application behaved slower than expected and
profiling showed that there is hotspot in PrivDES.encrypt() method.
This method in turn calls javax.crypto.Cipher.getInstance(String)
method (apparently on each message transmission) and
Cipher.getInstance(String) is the expensive and CPU intensive call. Can
you please suggest if:
1. This might be due wrong way of calling SNMP4J api (and also point
to a reference code for correct usage to prevent this problem)
2. Is this an inherent problem with Java 1.4 code?
3. Is this a problem with SNMP 1.11.4 code (it is not caching the
obtained cipher object)?
Thanks in advance for the help.
-Praveen
_______________________________________________
SNMP4J mailing list
[email protected]
http://lists.agentpp.org/mailman/listinfo/snmp4j
--
---
AGENT++
Maximilian-Kolbe-Str. 10
73257 Koengen, Germany
https://agentpp.com
Phone: +49 7024 8688230
Fax: +49 7024 8688231
_______________________________________________
SNMP4J mailing list
[email protected]
http://lists.agentpp.org/mailman/listinfo/snmp4j
------------------------------
Message: 2
Date: Mon, 03 Jun 2013 11:52:14 -0400
From: Elise Atkins <[email protected]>
To: Frank Fock <[email protected]>, [email protected]
Cc: chuckc <[email protected]>
Subject: Re: [SNMP4J] V3 Not In Time Window and Client Clock Drift
Message-ID: <[email protected]>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Frank,
After looking at RFC 3414, I have a different interpretation. In the
case 1) the RFC states that the local notion of the value of the
snmpEngineTime is to be updated to the msgAuthoritativeEngineTime. This
should mean that the response is in time regardless of how slow or fast
the authoritative engine's time is running as long as it moves in the
forward direction when the time check is done in 2).
b) If the extracted value of msgAuthoritativeEngineID is not the
same as the value snmpEngineID of the processing SNMP engine
(meaning this is not the authoritative SNMP engine), then:
1) if at least one of the following conditions is true:
- the extracted value of the msgAuthoritativeEngineBoots
field is greater than the local notion of the value of
snmpEngineBoots; or,
- the extracted value of the msgAuthoritativeEngineBoots
field is equal to the local notion of the value of
snmpEngineBoots, and the extracted value of
msgAuthoritativeEngineTime field is greater than the
value of latestReceivedEngineTime,
then the LCD entry corresponding to the extracted value of
the msgAuthoritativeEngineID field is updated, by setting:
- the local notion of the value of snmpEngineBoots to the
value of the msgAuthoritativeEngineBoots field,*
*
*- the local notion of the value of snmpEngineTime to the
value of the msgAuthoritativeEngineTime field, and*
- the latestReceivedEngineTime to the value of the value of
the msgAuthoritativeEngineTime field
2) if any of the following conditions is true, then the
message is considered to be outside of the Time Window:
- the local notion of the value of snmpEngineBoots is
2147483647;
- the value of the msgAuthoritativeEngineBoots field is
less than the local notion of the value of
snmpEngineBoots; or,
- the value of the msgAuthoritativeEngineBoots field is
equal to the local notion of the value of
snmpEngineBoots and the value of the
msgAuthoritativeEngineTime field is more than 150
seconds less than the local notion of the value of
snmpEngineTime.
From the above, I think if the reboots are the same and the
msgAuthoritativeEngineTime is greater then the
latestReceivedTimeEngineTime, the latestReceivedEngineTime should be
updated. At that point if you fall down into 2) then the since the
From release 2.1.2 UsmTimeTable starting at line 174
if ((entry.getEngineBoots() < time.getEngineBoots()) ||
((entry.getEngineBoots() == time.getEngineBoots()) &&
(time.getTimeDiff() + now >
entry.getLatestReceivedTime() + 150)) ||
(time.getEngineBoots() == 2147483647)) {
if (logger.isDebugEnabled()) {
logger.debug(
"CheckTime: received message outside time window (non
authoritative)");
}
return SnmpConstants.SNMPv3_USM_NOT_IN_TIME_WINDOW;
}
else {
if ((entry.getEngineBoots() > time.getEngineBoots()) ||
((entry.getEngineBoots() == time.getEngineBoots()) &&
(entry.getLatestReceivedTime() >
time.getLatestReceivedTime()))) {
/* time ok, update values */
time.setEngineBoots(entry.getEngineBoots());
time.setLatestReceivedTime(entry.getLatestReceivedTime());
time.setTimeDiff(entry.getLatestReceivedTime() - now);
}
Frank Fock wrote:
Hi Elise,
I would like to give you an update on the issue below.
I have checked the implementation, your findings, and the current RFC
3414
and found out, that RFC 3414 precised the description of the incoming
message handling (section 3.2.7 pages 27-28).
SNMP4J follows that description and is therefore conforming.
No changes will be applied.
Could you please verify this on your side, too?
Best regards,
Frank
Am 23.05.2013 17:52, schrieb Elise Atkins:
We have been successfully using snmp4j using v3 for both requests and
traps but have recently run into a problem with a client whose engine
time clock runs slow.
Initial time synchronization goes ok and the SNMP v3 requests and
responses flow properly. Eventually this client's engine time
becomes more 150 seconds behind the time SNMP4j is expecting and the
responses are marked as Not In Time. I have looked at RFC 2574
Section 3.2 subsection 7b and the code in the UsmTimeTable class,
checkTime method and have questions about the order of testing for
timeliness.
Based on the RFC, I would expect the code to test for the conditions
in 1 first and update if needed before testing the conditions in 2
but the code seems to test in the reverse order. Testing 1 and
updating reboots and time first will allow the client's clock to
drift (fast or slow) as long as it is always increasing and remain in
the time window. Am I missing something here?
Elise Atkins
The RFC says:
3.2. Processing an Incoming SNMP Message
7) If the securityLevel indicates an authenticated message, then
the local values of snmpEngineBoots, snmpEngineTime
and latestReceivedEngineTime
corresponding to the value of the msgAuthoritativeEngineID
field are extracted from the Local Configuration Datastore.
b) If the extracted value of msgAuthoritativeEngineID is not the
same as the value snmpEngineID of the processing SNMP engine
(meaning this is not the authoritative SNMP engine), then:
1) if at least one of the following conditions is true:
- the extracted value of the msgAuthoritativeEngineBoots
field is greater than the local notion of the value of
snmpEngineBoots; or,
- the extracted value of the msgAuthoritativeEngineBoots
field is equal to the local notion of the value of
snmpEngineBoots, and the extracted value of
msgAuthoritativeEngineTime field is greater than the
value of latestReceivedEngineTime,
then the LCD entry corresponding to the extracted value
of the msgAuthoritativeEngineID field is updated, by
setting:
- the local notion of the value of snmpEngineBoots to
the value of the msgAuthoritativeEngineBoots field,
- the local notion of the value of snmpEngineTime to
the value of the msgAuthoritativeEngineTime field,
and
- the latestReceivedEngineTime to the value of the
value of the msgAuthoritativeEngineTime field.
2) if any of the following conditions is true, then the
message is considered to be outside of the Time Window:
- the local notion of the value of snmpEngineBoots is
2147483647;
- the value of the msgAuthoritativeEngineBoots field is
less than the local notion of the value of
snmpEngineBoots; or,
- the value of the msgAuthoritativeEngineBoots field is
equal to the local notion of the value of
snmpEngineBoots and the value of the
msgAuthoritativeEngineTime field is more than 150
seconds less than the local notion of the value of
snmpEngineTime.
If the message is considered to be outside of the Time
Window then an error indication (notInTimeWindow) is
returned to the calling module.
_______________________________________________
SNMP4J mailing list
[email protected]
http://lists.agentpp.org/mailman/listinfo/snmp4j
------------------------------
Message: 3
Date: Mon, 03 Jun 2013 21:17:57 +0200
From: Frank Fock <[email protected]>
To: [email protected]
Cc: chuckc <[email protected]>, Elise Atkins <[email protected]>,
[email protected]
Subject: Re: [SNMP4J] V3 Not In Time Window and Client Clock Drift
Message-ID: <[email protected]>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Elise,
OK, I am convinced. I reservley checked the implementation with RFC3414 and
overseen your initial point about the order of checks.
I will provide a fix for it (ticket ID is SFJ-75).
The SNMP4J 2.2.2 release (with that fix) is then planned for next week.
Best regards,
Frank
Am 03.06.2013 17:52, schrieb Elise Atkins:
Frank,
After looking at RFC 3414, I have a different interpretation. In the
case 1) the RFC states that the local notion of the value of the
snmpEngineTime is to be updated to the msgAuthoritativeEngineTime.
This should mean that the response is in time regardless of how slow
or fast the authoritative engine's time is running as long as it moves
in the forward direction when the time check is done in 2).
b) If the extracted value of msgAuthoritativeEngineID is not the
same as the value snmpEngineID of the processing SNMP engine
(meaning this is not the authoritative SNMP engine), then:
1) if at least one of the following conditions is true:
- the extracted value of the msgAuthoritativeEngineBoots
field is greater than the local notion of the value of
snmpEngineBoots; or,
- the extracted value of the msgAuthoritativeEngineBoots
field is equal to the local notion of the value of
snmpEngineBoots, and the extracted value of
msgAuthoritativeEngineTime field is greater than the
value of latestReceivedEngineTime,
then the LCD entry corresponding to the extracted value of
the msgAuthoritativeEngineID field is updated, by setting:
- the local notion of the value of snmpEngineBoots to the
value of the msgAuthoritativeEngineBoots field,*
*
*- the local notion of the value of snmpEngineTime to the
value of the msgAuthoritativeEngineTime field, and*
- the latestReceivedEngineTime to the value of the value of
the msgAuthoritativeEngineTime field
2) if any of the following conditions is true, then the
message is considered to be outside of the Time Window:
- the local notion of the value of snmpEngineBoots is
2147483647;
- the value of the msgAuthoritativeEngineBoots field is
less than the local notion of the value of
snmpEngineBoots; or,
- the value of the msgAuthoritativeEngineBoots field is
equal to the local notion of the value of
snmpEngineBoots and the value of the
msgAuthoritativeEngineTime field is more than 150
seconds less than the local notion of the value of
snmpEngineTime.
From the above, I think if the reboots are the same and the
msgAuthoritativeEngineTime is greater then the
latestReceivedTimeEngineTime, the latestReceivedEngineTime should be
updated. At that point if you fall down into 2) then the since the
From release 2.1.2 UsmTimeTable starting at line 174
if ((entry.getEngineBoots() < time.getEngineBoots()) ||
((entry.getEngineBoots() == time.getEngineBoots()) &&
(time.getTimeDiff() + now >
entry.getLatestReceivedTime() + 150)) ||
(time.getEngineBoots() == 2147483647)) {
if (logger.isDebugEnabled()) {
logger.debug(
"CheckTime: received message outside time window (non
authoritative)");
}
return SnmpConstants.SNMPv3_USM_NOT_IN_TIME_WINDOW;
}
else {
if ((entry.getEngineBoots() > time.getEngineBoots()) ||
((entry.getEngineBoots() == time.getEngineBoots()) &&
(entry.getLatestReceivedTime() >
time.getLatestReceivedTime()))) {
/* time ok, update values */
time.setEngineBoots(entry.getEngineBoots());
time.setLatestReceivedTime(entry.getLatestReceivedTime());
time.setTimeDiff(entry.getLatestReceivedTime() - now);
}
Frank Fock wrote:
Hi Elise,
I would like to give you an update on the issue below.
I have checked the implementation, your findings, and the current RFC
3414
and found out, that RFC 3414 precised the description of the incoming
message handling (section 3.2.7 pages 27-28).
SNMP4J follows that description and is therefore conforming.
No changes will be applied.
Could you please verify this on your side, too?
Best regards,
Frank
Am 23.05.2013 17:52, schrieb Elise Atkins:
We have been successfully using snmp4j using v3 for both requests
and traps but have recently run into a problem with a client whose
engine time clock runs slow.
Initial time synchronization goes ok and the SNMP v3 requests and
responses flow properly. Eventually this client's engine time
becomes more 150 seconds behind the time SNMP4j is expecting and the
responses are marked as Not In Time. I have looked at RFC 2574
Section 3.2 subsection 7b and the code in the UsmTimeTable class,
checkTime method and have questions about the order of testing for
timeliness.
Based on the RFC, I would expect the code to test for the conditions
in 1 first and update if needed before testing the conditions in 2
but the code seems to test in the reverse order. Testing 1 and
updating reboots and time first will allow the client's clock to
drift (fast or slow) as long as it is always increasing and remain
in the time window. Am I missing something here?
Elise Atkins
The RFC says:
3.2. Processing an Incoming SNMP Message
7) If the securityLevel indicates an authenticated message, then
the local values of snmpEngineBoots, snmpEngineTime
and latestReceivedEngineTime
corresponding to the value of the msgAuthoritativeEngineID
field are extracted from the Local Configuration Datastore.
b) If the extracted value of msgAuthoritativeEngineID is not the
same as the value snmpEngineID of the processing SNMP engine
(meaning this is not the authoritative SNMP engine), then:
1) if at least one of the following conditions is true:
- the extracted value of the msgAuthoritativeEngineBoots
field is greater than the local notion of the value of
snmpEngineBoots; or,
- the extracted value of the msgAuthoritativeEngineBoots
field is equal to the local notion of the value of
snmpEngineBoots, and the extracted value of
msgAuthoritativeEngineTime field is greater than the
value of latestReceivedEngineTime,
then the LCD entry corresponding to the extracted value
of the msgAuthoritativeEngineID field is updated, by
setting:
- the local notion of the value of snmpEngineBoots to
the value of the msgAuthoritativeEngineBoots field,
- the local notion of the value of snmpEngineTime to
the value of the msgAuthoritativeEngineTime field,
and
- the latestReceivedEngineTime to the value of the
value of the msgAuthoritativeEngineTime field.
2) if any of the following conditions is true, then the
message is considered to be outside of the Time Window:
- the local notion of the value of snmpEngineBoots is
2147483647;
- the value of the msgAuthoritativeEngineBoots field is
less than the local notion of the value of
snmpEngineBoots; or,
- the value of the msgAuthoritativeEngineBoots field is
equal to the local notion of the value of
snmpEngineBoots and the value of the
msgAuthoritativeEngineTime field is more than 150
seconds less than the local notion of the value of
snmpEngineTime.
If the message is considered to be outside of the Time
Window then an error indication (notInTimeWindow) is
returned to the calling module.
_______________________________________________
SNMP4J mailing list
[email protected]
http://lists.agentpp.org/mailman/listinfo/snmp4j
