Number of dynamic fields will be in thousands (millions of users + thousands of events shared between subsets of users).
We also thought about indexing in one field with value being fieldname_fieldvalue. Since we support range queries for dates and numbers, it won't work out of box. On Mon, Jun 26, 2017 at 1:05 PM, Erick Erickson <[email protected]> wrote: > How many distinct fields do you expect across _all_ documents? That > is, if doc1 has 10 dynamic fields and doc2 has 10 dynamic fields, will > there be exactly 10 fields total or more than 10 when you consider > both documents? > > 100s of fields total across all documents is a tractable problem. > thousands of dynamic fields total is going to be a problem. > > One technique that people do use is to index one field with a prefix > rather than N dynamic fields. So you have something like > dyn1_val1 > dyn1_val2 > dyn4_val67 > > Only really works with string fields of course. > > Best, > Erick > > On Mon, Jun 26, 2017 at 10:11 AM, Saurabh Sethi > <[email protected]> wrote: > > We have two requirements: > > > > 1. Indexing and storing event id and its timestamp. > > 2. Indexing and storing custom field name and value. The fields can be of > > any type, but for now lets say they are of types string, date and number. > > > > The events and custom fields for any solr document can easily be in > > hundreds. > > > > We are looking at two different approaches to handle these scenarios: > > > > 1. *Dynamic fields* - Have the fields name start with a particular > pattern > > like for string, the pattern could be like str_* and for event could be > > eventid_* > > 2. *Parent/child fields* - This seems to be an overkill for our use case > > since it's more for hierarchical data. Also, the parent and all its > > children need to be reindexed on update which defeats the purpose - we > are > > now reindexing multiple docs instead of one with dynamic fields. But it > > allows us to store custom field name along with its value unlike dynamic > > fields where we will have to map user supplied custom field to some other > > name based on type. > > > > Has anyone handled similar scenarios with Solr? If so, which approach > would > > you recommend based on your experience? > > > > We are using solr 6.6 > > > > Thanks, > > Saurabh > -- Saurabh Sethi Principal Engineer I | Engineering
