Re: About Automated Unit Test for Wget
2008/4/5, Micah Cowan [EMAIL PROTECTED]:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Daniel Stenberg wrote:
This system allows us to write unit-tests if we'd like to, but mostly so
far we've focused to test it system-wide. It is hard enough for us!
Yeah, I thought I'd seen something like that; I was thinking we might
even be able to appropriate some of that, if that looked doable. Except
that I preferred faking the server completely, so I could deal better
with cross-site issues, which AFAICT are significantly more important to
Wget than they are to Curl.
It seems that abstraction of network API needs more discussion,
so I would focus on the server emulation
By the way, How about using LD_PRELOAD ?
I tested a little and it seems to be working. If we use this, we can test
by overriding socket interface, and still we don't change wget real source
code.
--main.c --
#include stdio.h
int main(void)
{
puts(Helow Wgets\n);
return 0;
}
--testputs.c
#include stdio.h
int puts(const char *str)
{
while(*str)
putchar(*str++);
printf(This is a test module);
putchar('\n');
}
-
--Compile like below:
[EMAIL PROTECTED] Test]$ gcc main.c -o main
[EMAIL PROTECTED] Test]$ gcc -fPIC -shared -o testputs.so testputs.c
--Execute like below:
[EMAIL PROTECTED] Test]$ ./main
Helow Wgets
[EMAIL PROTECTED] Test]$ LD_PRELOAD=./testputs.so ./main
Helow Wgets
This is a test module
--
I found this way on the net, and sample was using wget !! they are overriding
socket, close, and connect.
http://www.t-dori.net/forensics/hook_tcp.cpp
--
Yoshihiro TANAKA
Re: About Automated Unit Test for Wget
2008/4/5, Micah Cowan [EMAIL PROTECTED]:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Daniel Stenberg wrote:
This system allows us to write unit-tests if we'd like to, but mostly so
far we've focused to test it system-wide. It is hard enough for us!
Yeah, I thought I'd seen something like that; I was thinking we might
even be able to appropriate some of that, if that looked doable. Except
that I preferred faking the server completely, so I could deal better
with cross-site issues, which AFAICT are significantly more important to
Wget than they are to Curl.
It seems that abstraction of network API needs more discussion,
so I would focus on the server emulation
By the way, How about using LD_PRELOAD ?
I tested a little and it seems to be working. If we use this, we can test
by overriding socket interface, and still we don't change wget real source
code.
I found this way on the net, and sample was using wget !! they are overriding
socket, close, connect.
--main.c --
#include stdio.h
int main(void)
{
puts(Helow Wgets\n);
return 0;
}
--testputs.c
#include stdio.h
int puts(const char *str)
{
while(*str)
putchar(*str++);
printf(This is a test module);
putchar('\n');
}
-
--Compile like below:
[EMAIL PROTECTED] Test]$ gcc main.c -o main
[EMAIL PROTECTED] Test]$ gcc -fPIC -shared -o testputs.so testputs.c
--Execute like below:
[EMAIL PROTECTED] Test]$ ./main
Helow Wgets
[EMAIL PROTECTED] Test]$ LD_PRELOAD=./testputs.so ./main
Helow Wgets
This is a test module
--
Yoshihiro TANAKA
SFSU CS Department
Re: About Automated Unit Test for Wget
Micah Cowan [EMAIL PROTECTED] writes: I don't see what you see wrt making the code harder to follow and reason about (true abstraction rarely does, AFAICT, I was referring to the fact that adding an abstraction layer requires learning about the abstraction layer, both its concepts and its implementation, including its quirks and limitations. Too general abstractions added to application software are typically to be underspecified (for the domain they attempt to cover) and incomplete. Programmers tend to ignore the hidden cost of adding an abstraction layer until the cost becomes apparent, by which time it is too late. Application-specific abstractions are usually worth it because they are well-justified: they directly benefit the application by making the code base simpler and removing duplication. Some general abstractions are worth it because the alternative is worse; you wouldn't want to have two versions of SSL-using code, one for regular sockets, and one for SSL, since the whole point of SSL is that you're supposed to use it as if it were sockets behind the scenes. But adding a whole new abstraction layer over something as general as Berkely sockets to facilitate an automated test suite definitely sounds like ignoring the costs of such an abstraction layer. I _am_ thinking that it'd probably be best to forgo the idea of one-to-one correspondence of Berkeley sockets, and pass around a struct net_connector * (and struct net_listener *), so we're not forced to deal with file descriptor silliness (where obviously we'd have wanted to avoid the values 0 through 2, and I was even thinking it might _possibly_ be worthwhile to allocate real file descriptors to get the numbers, just to avoid clashes). I have no idea what file descriptor silliness with values 0-2 you're referring to. :-) I do agree that an application-specific struct is better than a more general abstraction because it is easier to design and more useful to Wget in the long run. This would mean we'd need to separate uses of read() and write() on normal files (which should continue to use the real calls, until we replace them with the file I/O abstractions), from uses of read(), write(), etc on sockets, which would be using our emulated versions. Unless you're willing to spend a lot of time in careful design of these abstractions, I think this is a mistake. Why? Because implementing a file I/O abstraction is much harder and more time-consuming than it sounds. To paraphrase Greenspun, it would appear that every sufficiently large code base contains an ad-hoc, informally-specified, bug-ridden implementation of a streaming layer. There are streaming libraries out there; maybe we should consider using some of them.
Re: About Automated Unit Test for Wget
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Micah Cowan wrote: Yeah. But we're not doing streaming. And you still haven't given much explanation for _why_ it's as hard and time-consuming as you say. Making a claim and demonstrating it are different things, I think. To be clear, I'm not trying to say, I don't believe you; I'm saying, argue the case, please, don't just make assertions. Clearly, you're concerned about something I'm unable to see: help me to see it! If I ignore your warnings, and wind up running headlong into what you saw in the first place, you can't claim you gave fair warning if you didn't provide examples of what I might run into. For my part, I see something which, at least for first cut, I could whip up in a couple of hours (the server emulation and associated state-tracking, of course, would be _quite_ a bit more work). What is it that causes our two perspectives to differ so wildly? - -- Micah J. Cowan Programmer, musician, typesetting enthusiast, gamer, and GNU Wget Project Maintainer. http://micah.cowan.name/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFH+KfI7M8hyUobTrERAt4YAKCKSfG/1HtV29mm1MSdDyzFuS8lRQCfdVla EIpSSdKhguieVxgYXln+XiQ= =mMj2 -END PGP SIGNATURE-
