Hi, Are you going to generate a report with 30000 records in it? That will be a very large report - will anyone really want to read through that? If you want/need 'summary' reports - i.e. stats on on the 30k records, it is much more efficient to setup faceting and/or server-side analysis to do this, rather than download 30000 records to a client, then do statistical analysis on the result. It will take a while to stream 30000 records over an http connection, and, if you're building, say, a PDF table for 30k records, that will take some time as well. Server-side analysis then just send the results will work better, if that fits your remit for reporting.
Peter On Thu, Sep 23, 2010 at 4:14 PM, Adeel Qureshi <adeelmahm...@gmail.com> wrote: > Thank you for your suggestions .. makes sense and I didnt knew about the > XsltResponseWriter .. that opens up door to all kind of possibilities ..so > its great to know about that > > but before I go that route .. what about performance .. In Solr Wiki it > mentions that XSLT transformation isnt so bad in terms of memory usage but I > guess its all relative to the amount of data and obviously system resources > .. > > my data set will be around 15000 - 30'000 records at the most ..I do have > about 30 some fields but all fields are either small strings (less than 500 > chars) or dates, int, booleans etc .. so should I be worried about > performances problems while doing the XSLT translations .. secondly for > reports Ill have to request solr to send all 15000 some records at the same > time to be entered in report output files .. is there a way to kind of > stream that process .. well I think Solr native xml is already streamed to > you but sounds like for the translation it will have to load the whole thing > in RAM .. > > and again what about SolrJ .. isnt that supposed to provide better > performance since its in java .. well I guess it shouldnt be much different > since it also uses the HTTP calls to communicate to Solr .. > > Thanks for your help > Adeel > > On Thu, Sep 23, 2010 at 7:16 AM, kenf_nc <ken.fos...@realestate.com> wrote: > >> >> keep in mind that the <str name="id"> paradigm isn't completely useless, >> the >> str is a data type (string), it can be int, float, double, date, and >> others. >> So to not lose any information you may want to do something like: >> >> <id type="int">123</id> >> <title type="str">xyz</title> >> >> Which I agree makes more sense to me. The name of the field is more >> important than it's datatype, but I don't want to lose track of the data >> type. >> >> Ken >> -- >> View this message in context: >> http://lucene.472066.n3.nabble.com/Solr-Reporting-tp1565271p1567604.html >> Sent from the Solr - User mailing list archive at Nabble.com. >> >
