On Tue, Nov 26, 2013 at 04:19:31PM -0800, Andrei Alexandrescu wrote: > On 11/26/13 3:40 PM, Adam D. Ruppe wrote: > >Again though, my position is that curl is ok and I use it myself, but > >let's not assume we need the entire kitchen sink that curl offers, > >especially given the fact that std.net.curl barely scratches it > >anyway. If someone does need one of curl's less common features, it > >is still there for them, but for the 99% we can do with the > >std.net.curl api, I'm sure we could make that happen in ~1000 lines > >of curl replacement. > > Care to make it interesting? I may have some funds for this. But we're > looking for no less than a glorious 100% replacement. [...]
This piqued my interest. Please excuse another of my TL;DR stories / rants. Some time earlier this year, I wanted to write a simple utility to (1) login to a website with a preconfigured username/password; (2) navigate to a specific page on the site; (3) extract a specific HTML table from that page, and (4) parse and format the table into an ASCII-graphics text format. Sounds like the perfect job for a quick-n-dirty D program, right? In the old days, I'd hack up a Perl script to do this in a couple o' hours, but this time, I thought, since I have D at my disposal, let's do this the Right Way(tm). Where in the past I'd just run wget or curl (the CLI utility) to fetch the web pages, I thought it'd be much cleaner to do the web interaction directly so that I didn't have to mess around with parsing temporary cookie files and what-not just to keep track of my login session. So I scrolled through the Phobos docs, and lo and behold, there was std.net.curl sitting right there, looking all pretty and beckoning to me. I glanced over its API, and found that it was just the right fit for what I needed to do. Or so I thought... About 30 minutes into the coding, I ran into a blocker: std.net.curl didn't let me specify Content-Type of POST requests!!! The server simply refused to accept anything that wasn't explicitly stated to be x/www-urlencoded, but std.net.curl only allows text/plain or application/octet-stream! I simply couldn't login. Thinking that I must've missed something obvious, I spent the rest of the day (and probably several more hours the next day) combing through the docs to find out how to set the Content-Type to something else. Finally I resorted to reading the std.net.curl source code, only to discover, to my dismay, that there was NO way to do this unless I bypassed all the nice high-level syntactic sugar, and got my hands down and dirty with low-level libcurl API calls (or their thin wrappers thereof). At that point, I threw up my hands. The original task was far too simple for something this heavy-handed, so I backed off and said, it was nice knowing you, std.net.curl, but I think we're better off going our separate ways. TL;DR: I would *love* to see a replacement for std.net.curl that isn't so straitjacketed that I can't even do something so basic as logging on to a website. I'm sure there are very good reasons why the current API is the way it is, but the fact of the matter is that, from an end-user's POV, its limitations were a deal-breaker. So, I'm all for a D-only replacement of std.net.curl with a superior API. Getting rid of the external dependence on libcurl would be an added bonus. T -- "I'm running Windows '98." "Yes." "My computer isn't working now." "Yes, you already said that." -- User-Friendly
