Re: Field Size per document in Solr
I want to get the field size (in kb or mb) as is It is stored on disk. That approach might not give that info. On Monday, January 4, 2016, Upayavirawrote: > > Solr does store the term positions, but you won't find it easy to > extract them, as they are stored against terms not fields. > > Your best bet is to index field lengths into Solr alongside the field > values. You could use an UpdateProcessor to do this if you want to do it > in Solr. > > Upayavira > > On Tue, Jan 5, 2016, at 12:39 AM, KNitin wrote: > > Hi, > > > > I want to get the size of individual fields per document (or per index) > > in > > solrcloud. Is there a way to do this using exiting solr or lucene api? > > > > *Use case*: I have a few dynamic fields which may or may not be populated > > everyday depending on certain conditions. I also do faceting and some > > custom processing on these fields (using custom solr components). I want > > to > > be able to plot the per field size of an index in realtime so that I can > > try to identify the trend between fields & latencies. > > > > Thanks a lot in advance! > > Nitin >
Re: Field Size per document in Solr
The field is not stored in a discrete place, rather it is mixed up with all other field/document data. Therefore, I would suggest that attempting to discern the disk space consumed by a single field would be a futile endeavour. Upayavira On Tue, Jan 5, 2016, at 12:04 PM, KNitin wrote: > I want to get the field size (in kb or mb) as is It is stored on disk. > That > approach might not give that info. > > On Monday, January 4, 2016, Upayavirawrote: > > > > > Solr does store the term positions, but you won't find it easy to > > extract them, as they are stored against terms not fields. > > > > Your best bet is to index field lengths into Solr alongside the field > > values. You could use an UpdateProcessor to do this if you want to do it > > in Solr. > > > > Upayavira > > > > On Tue, Jan 5, 2016, at 12:39 AM, KNitin wrote: > > > Hi, > > > > > > I want to get the size of individual fields per document (or per index) > > > in > > > solrcloud. Is there a way to do this using exiting solr or lucene api? > > > > > > *Use case*: I have a few dynamic fields which may or may not be populated > > > everyday depending on certain conditions. I also do faceting and some > > > custom processing on these fields (using custom solr components). I want > > > to > > > be able to plot the per field size of an index in realtime so that I can > > > try to identify the trend between fields & latencies. > > > > > > Thanks a lot in advance! > > > Nitin > >
Field Size per document in Solr
Hi, I want to get the size of individual fields per document (or per index) in solrcloud. Is there a way to do this using exiting solr or lucene api? *Use case*: I have a few dynamic fields which may or may not be populated everyday depending on certain conditions. I also do faceting and some custom processing on these fields (using custom solr components). I want to be able to plot the per field size of an index in realtime so that I can try to identify the trend between fields & latencies. Thanks a lot in advance! Nitin
Re: Field Size per document in Solr
Solr does store the term positions, but you won't find it easy to extract them, as they are stored against terms not fields. Your best bet is to index field lengths into Solr alongside the field values. You could use an UpdateProcessor to do this if you want to do it in Solr. Upayavira On Tue, Jan 5, 2016, at 12:39 AM, KNitin wrote: > Hi, > > I want to get the size of individual fields per document (or per index) > in > solrcloud. Is there a way to do this using exiting solr or lucene api? > > *Use case*: I have a few dynamic fields which may or may not be populated > everyday depending on certain conditions. I also do faceting and some > custom processing on these fields (using custom solr components). I want > to > be able to plot the per field size of an index in realtime so that I can > try to identify the trend between fields & latencies. > > Thanks a lot in advance! > Nitin