Pradeep,
Let me just start by saying that I haven't read all the other responses to this question. If my response overlaps with others, please excuse me.
Are you restricted to using a single machine, or the .NET Framework itself, in order to solve this problem? If, not, then I want to suggest that this looks like a case for a data parallel application. Have you looked at the MPI (Message Passing Interface) library? This is a portable parallel programming library designed to solve complex (data-parallel) problems on multiple computers. There are both commercial and freeware implementations of the MPI specification. And, there are C and Fortran based implementations of the library. You could use Managed C++ in order to use the library calls from your .NET code, for example.
Srihari
Pradeep Tapadiya wrote:
.NETters,
For the .NET project that I am working on, I need to index about 100 million strings. Each string is associated with a list of integers. The requirements are that the memory footprint must stay low and that we cannot use any database. It is okay to use a file-based memory management scheme. I am wondering if anyone has any suggestions on the algorithm to use (and if there already is code out there:-)).
Thank you in advance for your help.
Pradeep
=================================== This list is hosted by DevelopMentorŪ http://www.develop.com
View archives and manage your subscription(s) at http://discuss.develop.com
-- Srihari Angaluri Verari Systems Software, Inc. Phone +1-603-891-4766 http://www.verarisoft.com
The information contained in this communication may be confidential and is intended only for the use of the recipient(s) named above. If the reader of this communication is not the intended recipient(s), you are hereby notified that any dissemination, distribution, or copying of this communication, or any of its contents, is strictly prohibited. If you are not a named recipient or received this communication by mistake, please notify the sender and delete the communication and all copies of it.
=================================== This list is hosted by DevelopMentorŪ http://www.develop.com
View archives and manage your subscription(s) at http://discuss.develop.com
