[
https://issues.apache.org/jira/browse/SOLR-13312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16789077#comment-16789077
]
Noble Paul edited comment on SOLR-13312 at 3/11/19 2:53 AM:
------------------------------------------------------------
[~dsmiley] SolrDocument is an expensive data structure. Yes, we may need a more
efficient data structure to actually accomplish this. HashMaps are extremely
memory inefficient.
Skipping transformation is something we can't do now without backward
incompatibility. We can probably rewrite Transformers like ChildDocTransformer
to adapt to the new format. We may need to create the SolrDocument objects
where other transformers are used. But most requests never use any
transformers. They are paying a huge price
One problem we have in Solr is we never explore new Data structures suitable
for our needs. We just keep using List, Map, Namedlist irrespective of how many
objects we create .We don't ever differentiate between a usecase where 10 maps
are created Vs. 10 million Maps are created. We never stop to think how this
all adds up. Lucene keeps thinking about this all the time and it shows .
Lucene is extremely memory efficient while Solr is not
was (Author: noble.paul):
[~dsmiley] SolrDocument is an expensive data structure. Yes, we may need a more
efficient data structure to actually accomplish this. HashMaps are extremely
memory inefficient.
Skipping transformation is something we can't do now without backward
incompatibility. We can probably rewrite Transformers like ChildDocTransformer
to adapt to the new format. We may need to create the SolrDocument objects
where other transformers are used. But most requests never use any
transformers. They are paying a huge price
> write out responses without creating SolrDocument objects
> ---------------------------------------------------------
>
> Key: SOLR-13312
> URL: https://issues.apache.org/jira/browse/SOLR-13312
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Noble Paul
> Priority: Major
>
> Once we get a document from lucene there is no need to create a SolrDocument
> object to write out the response, if there are no transformers
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
