On Mon, 13 Jun 2005, Clift, Robert T CTR K55-Branch wrote:
I installed htdig 3.2.0b5 as a precompiled binary on Solaris 9 and am having problems with db.words.db once it reaches the 2GB file size. I recompiled with --enable-bigfile and still see the same problem. Does
Are you certain that none of the components from the precompiled binary are still floating around the system somewhere? If they are, then you might inadvertently be calling something not compiled with the bigfile support. When you built the new copy from source, did you ever run configure without the --enable-bigfile option? If so, did you run a make clean or better yet, a make distclean before trying to build with the bigfile support? If not, perhaps some components were not rebuilt. I seem to recall at least a couple people reporting success with large files when using the --enable-bigfile option; however I have not recently tested this option myself. Unless there is a reason that you need to stick with 3.2.0b5, I would suggest upgrading to 3.2.0b6. There are a number of known bugs in b5 that have been fixed in the new version, including some involving database corruption.
anyone know how to make multiple smaller databases and run htdig against those. Thanks in advance,
There are a couple approaches, both requiring a config file for each database that you want to build. Building the databases works the same either way. Create a configuration file for each set of databases that you want to build and pass that configuration file to rundig, or whatever you use to perform the indexing. If you use rundig, you will probably need to make some modifications to the script since the default version hard codes the path to the database directory. As for the configuration file, you will need customize the database location and the URL's to be indexed. See the following for more information. http://www.htdig.org/attrs.html#database_base http://www.htdig.org/attrs.html#database_dir http://www.htdig.org/attrs.html#start_url http://www.htdig.org/attrs.html#limit_urls_to If the grouping is such that it makes sense to limit search to a single database, then you can either create one search page for each grouping with a reference to the corresponding config file or use a single search page with a select list that allows a choice between databases. See http://www.htdig.org/attrs.html#build_select_lists for information on building select lists. If you want the search to encompass all of the databases, then you can simply add each config file to the search page using hidden input elements (e.g. <input type="hidden" name="config" value="configname">). This should work out of the box with the 3.2.x code. The option is also available with the 3.1.6 code using the collections patch. ------------------------------------------------------- SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click _______________________________________________ ht://Dig general mailing list: <htdig-general@lists.sourceforge.net> ht://Dig FAQ: http://htdig.sourceforge.net/FAQ.html List information (subscribe/unsubscribe, etc.) https://lists.sourceforge.net/lists/listinfo/htdig-general
