On Thu, Aug 7, 2008 at 5:33 PM, Álvaro J. Iradier <[EMAIL PROTECTED]>wrote:
> On Thu, Aug 7, 2008 at 11:15 PM, Youness Alaoui
> <[EMAIL PROTECTED]> wrote:
> > Yep, nice to see you back! :)
> >
> > Anyways, here are my ideas :
> >
> > 1 - for the wrapping of socket, you could do something like :
> >
> > proc socket { host port } {
> >
> > set var "::socket_wait_var_$host"
> >
> > set $var ""
> >
> > resolve_hostname $ip [list socket_resolver_callback $var]
> >
> > tkwait $var
> >
> > set sock [::tk::socket -async $var $port]
> >
> > unset $var
> >
> > return $sock
> >
> > }
>
> Yes, I already thought about that too. But that would just update the
> UI (process events on tkwait), not really "asynchronous" (the call
> wouldn't return inmediately). However, it's a better approach. I'll
> commit tomorrow the asyncresolver package, that when loaded,
> automatically replaces the socket command. I might need some help for
> porting to windows (and maybe other unixes, I'm using gethostbyname_r
> to allow reentrant calls), and to include it in the main Makefile.
Well.. the problem is not the "function returns immediatly".. it's just that
we need UI events to get processed to avoid the windows freezing...
You also told me on MSN that the host resolving is fast.. yes it IS fast...
I already tested this, it takes very few milliseconds to get the ip of a
host.. BUT it is VERY dependant on your connection...
in other words, if you have a very very slow connection (or a lot of packet
drops.. since it's UDP), then it will be slow... but most importantly...
when you have no connection at all, it will completely block until it times
out (5 or 10 seconds iirc)...
which means that when you loose your internet connection, and amsn tries to
auto-reconnect for example, it will freeze for 5-10 seconds completely until
the dns resolution times out...and that's the real problem...
In any case, it will be asynchronous since the 'socket' will not 'block', it
will create a thread, resolve asynchronously, and the 'update' will get you
back into the Tcl main loop which will continue processing events, this is
exactly what we need...
Anyways, about gethostbyname_r, as I told you before, I wrote some code for
resolving the hostname in the tcl_farsight extension.. and it is portable to
windows/mac/etc...
here's the code :
static char *host2ip(char *hostname)
{
struct addrinfo * result;
static char ip[30];
char * ret;
int error;
error = getaddrinfo(hostname, NULL, NULL, &result);
if (error != 0) {
return NULL;
}
if (result) {
ret = inet_ntop (AF_INET,
&((struct sockaddr_in *) result->ai_addr)->sin_addr,
ip, INET_ADDRSTRLEN);
freeaddrinfo (result);
if (ret == NULL) {
return NULL;
}
}
return ip;
}
Note that the function returns the ip as a string and from a static
variable, so you don't need to free any allocated memory by the function,
but calling the function twice will erase the previous value, so once you
call that, you need to strdup() the string before storing it anywhere... I
think that if you do a Tcl_NewStringObj(str), it will copy it already
though...
>
>
> > proc socket_resolver_callback { var ip } {
> >
> > set $var $ip
> >
> > }
> >
> > About the SOAP problem, I thought (stupidly) that whenever a tcl/tk
> function
> > gets called, that tcl would update the UI.. but this is of course wrong,
> > it's actually waiting on 'idle', then once it calls a function, it needs
> to
> > wait for it and all its underlying functions to finish processing before
> it
> > can process the window events..
> >
> > so a simple solution would be to look at which functions are huge, then
> call
> > 'update' in there... so for example, before and after parsing the XML in
> the
> > soap library.. then before and after calling the Soap callback... then
> maybe
> > a few times inside the SOAP callback itself if we're in a loop that might
> > take some time to process... (like, when we loop through every contact in
> > the soap xml to parse it and build our CL lists...).
> >
> > What do you think ?
>
>
> Well, it's a possible fix, but I prefer to avoid the idea of polling
> events when it's not absolutely necessary. I'll take a look at that
> funcions, maybe the XML parsing can be improved? We will see.
It's not polling events, it's just the way the main loop works, you tell it
to "process pending events" which means "refresh the window and don't
freeze"....
And I don't think the XML parsing can be improved.. maybe a little but you
can't do much about it.. M$ decided to do stupid SOAP requests.. which is
HUGE.. for example, in my "very small" contact list, I get a 100KB xml for
my contacts, and 60KB for my 'AL/BL/RL' lists...
I have an xml here provided by a user on the forums, who had a 1MB xml for
the 'AL/BL/RL' lists.. and I don't know how much for the whole contact
list... so, imagine.. 2MB of xml that need to be parsed from xml into a Tcl
list, then loop through every tag and initialize aMSN with it...
ok, we can do it in a much better way.. for example, right now, the code
looks like this :
set i 0
while { 1 } {
set subnode [GetXmlNode $xml $path $i]
incr i
if {$subnode == "" } { break }
...
}
where the GetXmlNode parses from the beginning and compares the index with
the requested one.. so the first iteration will parse until it finds $path..
the second iteration will parse until it finds $path, then skip it then
parse again.. the 100 iteration will reparse the 99 first $path tags that
were already parsed 99 times before... So maybe we could do something like
this :
set subnodes [GetAllXmlNodes $xml $path]
for {set i 0 } {$i < [llength $subnodes] } { incr i } {
set subnode [lindex $subnodes $i]
....
}
This way the XML will only get parsed once and will get all the paths in one
shot (the equivalent of calling the GetXmlNode for the last iteration
only...), so this can be an improvement, yes.. but I don't think it will be
enough to make it "wow, much better"... although I might be wrong, of
course...
KaKaRoTo
>
>
> > KaKaRoTo
> >
> > On Thu, Aug 7, 2008 at 4:33 PM, Boris Faure (aka billiob)
> > <[EMAIL PROTECTED]> wrote:
> >>
> >> thanks for your work, and welcome back :)
> >>
> >> On Thu, Aug 7, 2008 at 20:18, Álvaro J. Iradier <[EMAIL PROTECTED]>
> >> wrote:
> >> > Sorry for the previous blank email...
> >> >
> >> > ok, further investigation shows most time in GotSOAPReply is spent in
> >> > 'xml2list' (XML time) and in the callbacks (CB time):
> >> >
> >> > SOAPRequest time: 250623 microseconds per iteration
> >> > XML time: 132933 microseconds per iteration
> >> > CB ::SSOAuthentication1 AuthenticateCallback sso_authenticated
> >> > CB time: 594444 microseconds per iteration
> >> > SOAPReply time: 745462 microseconds per iteration
> >> >
> >> > SOAPRequest time: 246234 microseconds per iteration
> >> > XML time: 629189 microseconds per iteration
> >> > CB ::Addressbook1 FindMembershipCallback {::Addressbook1
> >> > FindMembershipDone {::MSN::ABSynchronizationDone 1}}
> >> > CB time: 667627 microseconds per iteration
> >> > SOAPReply time: 1329095 microseconds per iteration
> >> >
> >> > SOAPRequest time: 286000 microseconds per iteration
> >> > XML time: 1117484 microseconds per iteration
> >> > CB ::Addressbook1 ABFindAllCallback {::Addressbook1 ABFindAllDone
> >> > {::MSN::ABSynchronizationDone 1}}
> >> > CB time: 6183310 microseconds per iteration
> >> > SOAPReply time: 7402998 microseconds per iteration
> >> >
> >> > SOAPRequest time: 244129 microseconds per iteration
> >> > XML time: 15804 microseconds per iteration
> >> > CB ::ContentRoaming1 GetProfileCallback {::ns setInitialNicknameCB
> HDN
> >> > HDN} {}
> >> > CB time: 2674448 microseconds per iteration
> >> > SOAPReply time: 2692246 microseconds per iteration
> >> >
> >> >
> >> > On Thu, Aug 7, 2008 at 7:31 PM, Álvaro J. Iradier <[EMAIL PROTECTED]
> >
> >> > wrote:
> >> >> HI, the other day, talking to youness we thought amsn was being quite
> >> >> unresponssive on start and connection due to sockets blocking while
> >> >> resolving the host names (that is, the dns resolution was being
> >> >> synchronous and blocking, even when the sockets are asynchronous).
> >> >>
> >> >> I made a small tcl extension to allow for asynchronous resolution.
> The
> >> >> idea was to replace the socket command, and when -async option was
> >> >> used, return inmediately, launch a thread to resolve hostname, and
> >> >> when resolution was done, call the original socket command with the
> >> >> resolved IP instead of the hostname.
> >> >>
> >> >> The problem is this can't be done that easy, because when socket
> >> >> -async is used, it should return inmediately and return a channel
> >> >> identifier. As we don't know the channel identifier until the real
> >> >> socket is created, what should the extension socket wrapper return if
> >> >> the socket won't be created until hostname is resolved?
> >> >>
> >> >> So, i just made a new command that resolves a hostname in background
> >> >> and callbacks a procedure with the resolved IP when finished.
> >> >>
> >> >> Before starting to change the connection code to use the resolver, I
> >> >> decided to measure the time spen on socket creation. I made a
> >> >> socket_wrapper function like this in amsncore.tcl:
> >> >>
> >> >> --------------
> >> >>
> >> >> proc socket_wrapper { args } {
> >> >> puts "socket_wrapper $args"
> >> >>
> >> >> puts [time {
> >> >> set sock [eval [linsert $args 0 _oldsocket] ]}]
> >> >>
> >> >> return $sock
> >> >> }
> >> >>
> >> >> rename socket _oldsocket
> >> >> rename socket_wrapper socket
> >> >>
> >> >> --------------
> >> >>
> >> >> running and connection amsn displays this:
> >> >>
> >> >> socket_wrapper -server phony 60671
> >> >> 415 microseconds per iteration
> >> >> socket_wrapper -myaddr 127.0.0.1 -server lockSvrNew 64123
> >> >> 323 microseconds per iteration
> >> >> socket_wrapper -async 207.46.111.90 1863
> >> >> 339 microseconds per iteration
> >> >> socket_wrapper login.live.com 443
> >> >> 343022 microseconds per iteration
> >> >> socket_wrapper contacts.msn.com 443
> >> >> 289655 microseconds per iteration
> >> >> socket_wrapper contacts.msn.com 443
> >> >> 251321 microseconds per iteration
> >> >> socket_wrapper -server abook::dummysocketserver 6891
> >> >> 148 microseconds per iteration
> >> >> socket_wrapper -async firewall.amsn-project.net 80
> >> >> 59207 microseconds per iteration
> >> >> socket_wrapper storage.msn.com 443
> >> >> 298166 microseconds per iteration
> >> >> socket_wrapper contacts.msn.com 443
> >> >> 254806 microseconds per iteration
> >> >> socket_wrapper -async 207.46.27.197 1863
> >> >> 353 microseconds per iteration
> >> >> socket_wrapper -async 207.46.26.66 1863
> >> >> 335 microseconds per iteration
> >> >>
> >> >>
> >> >> So, it looks like connections with name resolution are taking about
> >> >> 0,06 seconds (like http connection to firewall.amsn-project.net),
> and
> >> >> https connections are taking about 0,25-0,3 seconds. That's pretty
> >> >> quick. However, it looks like things are locking longer, amsn seems
> >> >> quite unresponsive while connection.
> >> >>
> >> >> It looks like maybe name resolution is not the bottleneck. The TLS
> >> >> layer is taking some time, for sure, but I think problem is SOAP
> >> >> replies are read synchronously, and blocking. These are the new
> >> >> timings:
> >> >>
> >> >> ocket_wrapper -server phony 61199 time: 238 microseconds per
> iteration
> >> >> socket_wrapper -myaddr 127.0.0.1 -server lockSvrNew 62966 time: 282
> >> >> microseconds per iteration
> >> >> socket_wrapper -async 207.46.111.90 1863 time: 20350 microseconds
> per
> >> >> iteration
> >> >> socket_wrapper login.live.com 443 time: 228129 microseconds per
> >> >> iteration
> >> >> SOAPRequest time: 295017 microseconds per iteration
> >> >> SOAPReply time: 819305 microseconds per iteration
> >> >> socket_wrapper contacts.msn.com 443 time: 236107 microseconds per
> >> >> iteration
> >> >> SOAPRequest time: 243931 microseconds per iteration
> >> >> socket_wrapper contacts.msn.com 443 time: 232883 microseconds per
> >> >> iteration
> >> >> SOAPRequest time: 240097 microseconds per iteration
> >> >> socket_wrapper -server abook::dummysocketserver 6891 time: 177
> >> >> microseconds per iteration
> >> >> socket_wrapper -async firewall.amsn-project.net 80 time: 2317
> >> >> microseconds per iteration
> >> >> SOAPReply time: 1672309 microseconds per iteration
> >> >> socket_wrapper storage.msn.com 443 time: 241007 microseconds per
> >> >> iteration
> >> >> SOAPRequest time: 248864 microseconds per iteration
> >> >> SOAPReply time: 10365145 microseconds per iteration
> >> >> SOAPReply time: 2872543 microseconds per iteration
> >> >>
> >> >> Look at that SOAPReply times!! 0,8 seconds, 1,67 seconds, 2,87
> seconds
> >> >> and 10,36 seconds!!!! During that time AMSN is completely blocked!!
> >> >>
> >> >> So, I'm going to fix this first, and then i'll think about the dns
> >> >> resolve, maybe it's not worth the effort.
> >> >>
> >> >> If you have any questions or want to help me, just find me at amsn :)
> >> >>
> >> >> Greets.
> >> >>
> >> >> --
> >> >> (:===========================================:)
> >> >> Alvaro J. Iradier Muro - [EMAIL PROTECTED]
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > (:===========================================:)
> >> > Alvaro J. Iradier Muro - [EMAIL PROTECTED]
> >> >
> >> >
> >> >
> -------------------------------------------------------------------------
> >> > This SF.Net email is sponsored by the Moblin Your Move Developer's
> >> > challenge
> >> > Build the coolest Linux based applications with Moblin SDK & win great
> >> > prizes
> >> > Grand prize is a trip for two to an Open Source event anywhere in the
> >> > world
> >> > http://moblin-contest.org/redirect.php?banner_id=100&url=/
> >> > _______________________________________________
> >> > Amsn-devel mailing list
> >> > Amsn-devel@lists.sourceforge.net
> >> > https://lists.sourceforge.net/lists/listinfo/amsn-devel
> >> >
> >>
> >>
> >>
> >> --
> >> Boris FAURE (aka billiob)
> >>
> -------------------------------------------------------------------------
> >> This SF.Net email is sponsored by the Moblin Your Move Developer's
> >> challenge
> >> Build the coolest Linux based applications with Moblin SDK & win great
> >> prizes
> >> Grand prize is a trip for two to an Open Source event anywhere in the
> >> world
> >> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> >> _______________________________________________
> >> Amsn-devel mailing list
> >> Amsn-devel@lists.sourceforge.net
> >> https://lists.sourceforge.net/lists/listinfo/amsn-devel
> >
> >
> > -------------------------------------------------------------------------
> > This SF.Net email is sponsored by the Moblin Your Move Developer's
> challenge
> > Build the coolest Linux based applications with Moblin SDK & win great
> > prizes
> > Grand prize is a trip for two to an Open Source event anywhere in the
> world
> > http://moblin-contest.org/redirect.php?banner_id=100&url=/
> > _______________________________________________
> > Amsn-devel mailing list
> > Amsn-devel@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/amsn-devel
> >
> >
>
>
>
> --
> (:===========================================:)
> Alvaro J. Iradier Muro - [EMAIL PROTECTED]
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's
> challenge
> Build the coolest Linux based applications with Moblin SDK & win great
> prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Amsn-devel mailing list
> Amsn-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/amsn-devel
>
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Amsn-devel mailing list
Amsn-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amsn-devel
