I have OpenSSL running on a Solaris 8 system. I have installed the /dev/random
patch and can connect from the command line via:
openssl s_client -host myhost -port 443
Without setting any environement variables, like RANDFILE, or creating .rnd files.
I have htdig/htdig-3.2.0b4-20020721 compiled with SSL. I have tried this with
OpenSSL-0.9.6d and OpenSSL-0.9.7-beta2. I can get the robots.txt file from an
SSL enabled web server, but htdig core dumps when it trys to the get next file.
I have tried this with and without requiring a username/password to the web
server.
I need the features of 3.2.0b4 in order for my project to use HtDig over
spending money on a commerical version. But I also need to connect to remote
https web servers. I am also anxiously waiting the support for
search_rewrite_rules. Other than that, we are very impressed with HtDig, thanks!
./rundig -vvvvvvv -s
ht://dig Start Time: Thu Jul 25 10:37:35 2002
1:1:https://myhost.com/
New server: myhost.com, 443
- Persistent connections: enabled
- HEAD before GET: disabled
- Timeout: 30
- Connection space: 0
- Max Documents: -1
- TCP retries: 1
- TCP wait time: 5
- Accept-Language:
Trying to retrieve robots.txt file
Creating an HtHTTPSecure object
Making HTTPS request on https://myhost.com/robots.txt
Try to get through to host myhost.com (port 443)
1 - Open of the connection ok
Assigning the server (myhost.com) to the TCP connection
Assigned the remote host myhost.com
Assigning the port (443) to the TCP connection
Assigned the port 443
Connecting via TCP to (myhost.com:443)
New connection open successfully
Request
GET /robots.txt HTTP/1.1
Host: myhost.com
User-Agent: htdig
Header line: HTTP/1.1 200 OK
Header line: Server: Netscape-Enterprise/6.0
Header line: Date: Thu, 25 Jul 2002 17:37:35 GMT
Header line: Content-type: text/plain
Header line: Etag: "de7b21ea-8-0-1e"
Discarded header line: Etag: "de7b21ea-8-0-1e"
Header line: Last-modified: Thu, 25 Jul 2002 16:46:46 GMT
Header line: Content-length: 30
Header line: Accept-ranges: bytes
Discarded header line: Accept-ranges: bytes
Retrieving document /robots.txt on host: myhost.com:443
Http version : HTTP/1.1
Server : HTTP/1.1
Status Code : 200
Reason : OK
Access Time : Thu, 25 Jul 2002 17:37:35 PST
Modification Time : Thu, 25 Jul 2002 16:46:46 PST
Content-type : text/plain
Persistent connection: would be accepted
Reading the body of the response
Contents:
User-agent: *
Disallow: /tmp/
Connection stays up ... (Persistent connection)
Request time: 0 secs
Contents:
User-agent: *
Disallow: /tmp/
Content Type: text/plain
Content Length: 30
Modification Time: 2002-07-25 16:46:46 PST
Parsing robots.txt file using myname = htdig
Robots.txt line: User-agent: *
Found 'user-agent' line: *
Robots.txt line: Disallow: /tmp/
Found 'disallow' line: /tmp/
Pattern: /tmp/
1 - Closing previous connection with the remote host
pushed
pick: myhost.com, # servers = 1
> myhost.com supports HTTP persistent connections (infinite)
0:2:0:https://myhost.com/: Creating an HtHTTPSecure object
Making HTTPS request on https://myhost.com/
Try to get through to host myhost.com (port 443)
2 - Open of the connection ok
Assigning the server (myhost.com) to the TCP connection
Assigned the remote host myhost.com
Assigning the port (443) to the TCP connection
Assigned the port 443
Connecting via TCP to (myhost.com:443)
Segmentation Fault - core dumped
htdig.conf
database_dir: /search/cassini/db
start_url: https://myhost.com/
limit_urls_to: ${start_url}
exclude_urls: /cgi-bin/ .cgi
bad_extensions: .wav .gz .z .sit .au .zip .tar .hqx .exe .com .gif \
.jpg .jpeg .aiff .class .map .ram .tgz .bin .rpm .mpg .mov .avi .css
maintainer: [EMAIL PROTECTED]
max_head_length: 10000
max_doc_size: 200000
no_excerpt_show_top: true
search_algorithm: exact:1 synonyms:0.5 endings:0.1
--
Rob Kremer
JPL Cassini SA
818-393-1283 Fax: 393-4658
Office 230-311 M/S 230-310
--
-------------------------------------------------------
This sf.net email is sponsored by: Jabber - The world's fastest growing
real-time communications platform! Don't just IM. Build it in!
http://www.jabber.com/osdn/xim
_______________________________________________
htdig-general mailing list <[EMAIL PROTECTED]>
To unsubscribe, send a message to <[EMAIL PROTECTED]> with a
subject of unsubscribe
FAQ: http://htdig.sourceforge.net/FAQ.html
