I need some explanation on how ValuesSource and related classes work. There are already implemented ExternalFileField, example on how to load data from database ( <http://sujitpal.blogspot.com/2011/05/custom-sorting-in-solr-using-external. html> http://sujitpal.blogspot.com/2011/05/custom-sorting-in-solr-using-external.h tml)
But they all fetch ALL data into memory which may consume large amounts of this resource. Also documents are referenced by 'doc' integer value. My questions: 1) Is the 'doc' value pointing to document in whole index? If so - how to get value of such documents field (for example: field named 'id')? 2) Is there possibility to create ValuesSource, FieldType (or similar interface which will provide external data to sort and in query results) which will work only on some subset of documents and use external source capabilities to fetch document related data? 3) How does it all work (memory consumption, hashtable access speed, etc), when there is a lot of documents in index (tens of millions for example)? 4) Are there any other examples on loading external data from database (I want to have numerical 'rate' from simple table having two columns: 'document unique key' string, 'rate' integer/float) which are not just proof of concept but real-life examples? Any help and hints appreciated TIA -- Maciek