Re: [AOLSERVER] Help with Dotted IP conversion
When we evaluated 3.4 performance (TCL 7.6 and 8x) vs. 2.3.3 (TCL 7.4), one thing I compared was a 10-line loop to load an ns_share array. The loop contained maybe 8-10 string operations on a string of around 200 characters, and a single set command with an ns_share array lvalue. One execution of the procedure takes something like 3 minutes, i.e., TCL 8x does one compile and then executes for several minutes. I couldn't find the exact benchmark results just now, but it was something along the lines of 3.4 w/7.6 being 80-90% faster than 2.3.3/7.4, and 3.4 w/8x being only 25% faster. So in this case, the TCL 8x compiling and faster string operations were greatly outweighed by just 1 ns_share variable being set. This performance difference is the reason we didn't migrate to 8x. Just a data point that sites making heavy use of ns_share may want to consider. Jim Ah yes, the lifespan of Tcl 7.6 has certainly been longer than many expected, or perhaps wanted ... There are so many reasons to upgrade, and very, very few not to. At the Tcl level, script compatability was near 100% (it was Tk 4.x - 8.x that was more sensitive). The main reason is speed. Check out the data at: http://wiki.tcl.tk/1611.html That indicates a 6x overall speedup from 7.6 - 8.0. I'm now working on 8.4 to push that boundary even further (already beating 8.0 by 20% and we're not done yet). Then of course there are the new features. More advanced regexp, new string functions, transparent unicode support throughout, 'binary', 'namespace', more 'file' commands, ... Recall that the last patch release of 7.6 was in January 1997! A lot has happened since then. Jeff Hobbs The Tcl Guy Senior Developer http://www.ActiveState.com/ Tcl Support and Productivity Solutions
Re: [AOLSERVER] Help with Dotted IP conversion
Jim Wilcoxson wrote: ... I couldn't find the exact benchmark results just now, but it was something along the lines of 3.4 w/7.6 being 80-90% faster than 2.3.3/7.4, and 3.4 w/8x being only 25% faster. So in this case, the TCL 8x compiling and faster string operations were greatly outweighed by just 1 ns_share variable being set. This performance difference is the reason we didn't migrate to 8x. Just a data point that sites making heavy use of ns_share may want to consider. The ns_share stuff is very sensitive to the performance of the memory allocator across threads. This has fluctuated over time, and I believe the one in the current aolserver head is the best so far. Earlier 8.x versions were not as good as the 7.6 version, and the default one in Tcl was really bad for threads. I'm currently working with Jim Davidson to incorporate the much faster threaded allocator into the core for 8.4. Another thing that might be noted is that Zoran Vasiljevic (main guy doing the Tcl Thread extension) mentioned, I believe, that he had a somewhat different shared variable implementation in the Thread extension that was faster than the AOLServer one. I'm not really sure, as it was just a newsgroup comment in passing. Also, I believe you implied that you were using a proc, but it is important to note that toplevel code will be slower if large loops are used without putting them in procs. This is by design. The feeling was that toplevel code is only ever evaled once, and thus doesn't need to be compiled. However large loops would always gain a compilation benefit, so they should be in procs. Some other pointers about this at http://wiki.tcl.tk/348.html. Perhaps we should always compile loops ... Jeff Hobbs The Tcl Guy Senior Developer http://www.ActiveState.com/ Tcl Support and Productivity Solutions
Re: [AOLSERVER] Help with Dotted IP conversion
Thanks for the info Jeff. My understanding is that the TCL 7.6 ns_share stuff is hacked into the TCL interpreter variable handler, while the 8x ns_share routines use variable traces. I had wondered about using the C variable facility to do ns_shares but haven't messed with it. It seems possible, although there is no way to unset a C variable but an ns_share can be unset. I'm sure many more issues than this... LOL. FYI, the loop was contained in a proc in a TCL file that is loaded during startup. So I assume it was compiled. We're using the standard allocator, not -z. Jim Jim Wilcoxson wrote: ... I couldn't find the exact benchmark results just now, but it was something along the lines of 3.4 w/7.6 being 80-90% faster than 2.3.3/7.4, and 3.4 w/8x being only 25% faster. So in this case, the TCL 8x compiling and faster string operations were greatly outweighed by just 1 ns_share variable being set. This performance difference is the reason we didn't migrate to 8x. Just a data point that sites making heavy use of ns_share may want to consider. The ns_share stuff is very sensitive to the performance of the memory allocator across threads. This has fluctuated over time, and I believe the one in the current aolserver head is the best so far. Earlier 8.x versions were not as good as the 7.6 version, and the default one in Tcl was really bad for threads. I'm currently working with Jim Davidson to incorporate the much faster threaded allocator into the core for 8.4. Another thing that might be noted is that Zoran Vasiljevic (main guy doing the Tcl Thread extension) mentioned, I believe, that he had a somewhat different shared variable implementation in the Thread extension that was faster than the AOLServer one. I'm not really sure, as it was just a newsgroup comment in passing. Also, I believe you implied that you were using a proc, but it is important to note that toplevel code will be slower if large loops are used without putting them in procs. This is by design. The feeling was that toplevel code is only ever evaled once, and thus doesn't need to be compiled. However large loops would always gain a compilation benefit, so they should be in procs. Some other pointers about this at http://wiki.tcl.tk/348.html. Perhaps we should always compile loops ... Jeff Hobbs The Tcl Guy Senior Developer http://www.ActiveState.com/ Tcl Support and Productivity Solutions
Re: [AOLSERVER] Help with Dotted IP conversion
+-- On Feb 26, Jeff Hobbs said:
Also, I believe you implied that you were using a proc, but it is
important to note that toplevel code will be slower if large loops
are used without putting them in procs. This is by design.
And boy was that annoying when we (ArsDigita) wanted to cache .tcl page
bytecode, because .tcl pages are normally executed with the source
command, which doesn't compile. We ended up taking advantage of the
fact that for (in Tcl_ForObjCmd) compiles its first argument, e.g.
for [util_file_contents_cached $filename] {0} {} {}
...compiles the string returned by util_file_contents_cached.
Re: [AOLSERVER] Help with Dotted IP conversion
The Tcl 'expr' command works with signed integers, whatever the
size of an int is on your platform. However, it is just a stream
of bits, so when you have your number, just do:
format %u -1
However, note that this number is really for viewing only, because
as soon as expr gets it again, it will be signed internally. Again
this likely isn't a problem as you can put the end result through
format again. Here is an example:
(tkcon) 49 % format %u -1
4294967295
(tkcon) 50 % expr {[format %u -1]-1}
-2
(tkcon) 51 % format %u [expr {[format %u -1]-1}]
4294967294
Note that Tcl 8.4 may be slightly different for some. It is adding
wide integers to the core, mostly to handle 64 bit file systems,
but also for calculations with large numbers. In 8.4, the above
command at 50 will actually return 4294967294 directly. This means
long longs are used when necessary (or int64 on Windows).
Jeff Hobbs The Tcl Guy
Senior Developer http://www.ActiveState.com/
Tcl Support and Productivity Solutions
JamesRanson wrote:
...
can only give it in Long format. I have come up with a function to convert
the Long IP into Dotted Quad format, but the first division returns a
negative value whenever the Long IP provided is greater than 2147483647.
Re: [AOLSERVER] Help with Dotted IP conversion
You can try these functions:
proc inet_addr { ipaddr } {
set addr [binary format c4 [split $ipaddr .]]
binary scan $addr i bin
return $bin
}
proc inet_ntoa { ipaddr } {
set bin [binary format i $ipaddr]
binary scan $bin a b c d
set a [expr { ($a + 0x100) % 0x100 }]
set b [expr { ($b + 0x100) % 0x100 }]
set c [expr { ($c + 0x100) % 0x100 }]
set d [expr { ($d + 0x100) % 0x100 }]
return $a.$b.$c.$d
}
On Mon, Feb 25, 2002 at 02:09:51PM -0500, [EMAIL PROTECTED] wrote:
I am working on a back-end logging system and am having a bit of difficulty
with a number conversion. The resource that provides the IP address to log
can only give it in Long format. I have come up with a function to convert
the Long IP into Dotted Quad format, but the first division returns a
negative value whenever the Long IP provided is greater than 2147483647. Can
anyone help figure out why expr will not accept any values bigger than that,
and what my options are? I have looked through the AOLServer functions to see
if there was anything built-in that would work but nothing caught my eye.
Thanks!
proc dotted_ip_from_long { long_ip } {
set n 1
set ip_parts [list]
while { $n 5 } {
set denom [expr int(pow(256,(4 - $n)))]
set ip_part [expr int(floor($long_ip / $denom ))]
set long_ip [expr $long_ip % $denom ]
lappend ip_parts $ip_part
incr n
}
return [join $ip_parts .]
}
Re: [AOLSERVER] Help with Dotted IP conversion
On 2002.02.25, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote:
I have come up with a function to convert
the Long IP into Dotted Quad format, [...]
I've been thinking about writing a proc net for a while that
exposes [net inet_ntoa $long] for a while, but ... I'm lazy.
proc inet_ntoa {long} {
set hex [format %08x $long]
for {set i 0} {$i 8} {incr i 2} {
lappend dec [format %u 0x[string range $hex $i [expr $i + 1]]]
}
return [join $dec .]
}
To test:
231.123.213.132 - 3883652484
% inet_ntoa 3883652484
231.123.213.132
127.0.0.1 - 2130706433
% inet_ntoa 2130706433
127.0.0.1
255.255.255.255 - 4294967295
% inet_ntoa 4294967295
255.255.255.255
Performance isn't too bad under Tcl8, either:
% puts $tcl_patchLevel
8.3.3
% time {inet_ntoa 4294967295} 1
771 microseconds per iteration
% expr 1 / (711 * pow(10, -6))
1406.4697609
Unless I'm reading that wrong, it's approximately able to execute
1400 per second.
Compare to your dotted_ip_from_long:
% dotted_ip_from_long 2130706433
127.0.0.1
% time {dotted_ip_from_long 2130706433} 1
1265 microseconds per iteration
Just 200 usec short of twice as slow.
-- Dossy
--
Dossy Shiobara mail: [EMAIL PROTECTED]
Panoptic Computer Network web: http://www.panoptic.com/
He realized the fastest way to change is to laugh at your own
folly -- then you can let go and quickly move on. (p. 70)
Re: [AOLSERVER] Help with Dotted IP conversion
On 2002.02.25, Rob Mayoff [EMAIL PROTECTED] wrote:
+-- On Feb 25, [EMAIL PROTECTED] said:
Can anyone help figure out why expr will not accept any values bigger
than that,
Because your Tcl interpreter uses 32-bit signed integers.
proc dotted_ip_from_long {n} {
binary scan [binary format I $n] c4 signed_octets
set octets {}
foreach octet $signed_octets {
lappend octets [expr {$octet 0xff}]
}
join $octets .
}
Damn you. I couldn't remember binary -- too lazy to fire
off apropos :-). That is, of course, the right solution,
similar to the one I've used in Perl which uses pack() and
unpack().
(using Rob's version of dotted_ip_from_long):
% time {dotted_ip_from_long 2130706433} 1
194 microseconds per iteration
Of course, Rob's version only works under Tcl8, where I
(think) my version will also work on Tcl7 unmodified ...
-- Dossy
--
Dossy Shiobara mail: [EMAIL PROTECTED]
Panoptic Computer Network web: http://www.panoptic.com/
He realized the fastest way to change is to laugh at your own
folly -- then you can let go and quickly move on. (p. 70)
Re: [AOLSERVER] Help with Dotted IP conversion
Thanks Dossy -- worked like a charm. I tried the binary method but apparently binary is not an available command in TCL 7.6 (don't shoot me I'm not the web admin, I just write the code ;) As always, you guys are a great help!
