[jira] [Commented] (LUCENE-8558) Adding NumericDocValuesFields is slowing down the indexing process significantly
[
https://issues.apache.org/jira/browse/LUCENE-8558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16678605#comment-16678605
]
Kranthi commented on LUCENE-8558:
-
There was no change in functionality. The change was a slight adjustment to
existing code. Existing test cases for this functionality should be enough test
the change.
I reran my indexing job and the performance was back to normal
> Adding NumericDocValuesFields is slowing down the indexing process
> significantly
>
>
> Key: LUCENE-8558
> URL: https://issues.apache.org/jira/browse/LUCENE-8558
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/index
>Affects Versions: 7.4, 7.5
>Reporter: Kranthi
>Priority: Major
> Labels: patch, performance
> Fix For: 7.4, 7.5
>
> Attachments: LUCENE-8558.patch
>
>
> The indexing time for my ~2M documents has gone up significantly when I
> started adding fields of type NumericDocValuesField.
>
> Upon debugging found the bottleneck to be in the
> PerFieldMergeState#FilterFieldInfos constructor. The contains check in the
> below code snippet was the culprit.
> {code:java}
> this.filteredNames = new HashSet<>(filterFields);
> this.filtered = new ArrayList<>(filterFields.size());
> for (FieldInfo fi : src) {
> if (filterFields.contains(fi.name)) {
> {code}
> A simple change as below seems to have fixed my issue
> {code:java}
> this.filteredNames = new HashSet<>(filterFields);
> this.filtered = new ArrayList<>(filterFields.size());
> for (FieldInfo fi : src) {
> if (this.filteredNames.contains(fi.name)) {
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8558) Adding NumericDocValuesFields is slowing down the indexing process significantly
[
https://issues.apache.org/jira/browse/LUCENE-8558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kranthi updated LUCENE-8558:
Attachment: LUCENE-8558.patch
> Adding NumericDocValuesFields is slowing down the indexing process
> significantly
>
>
> Key: LUCENE-8558
> URL: https://issues.apache.org/jira/browse/LUCENE-8558
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/index
>Affects Versions: 7.4, 7.5
>Reporter: Kranthi
>Priority: Major
> Labels: patch, performance
> Fix For: 7.4, 7.5
>
> Attachments: LUCENE-8558.patch
>
>
> The indexing time for my ~2M documents has gone up significantly when I
> started adding fields of type NumericDocValuesField.
>
> Upon debugging found the bottleneck to be in the
> PerFieldMergeState#FilterFieldInfos constructor. The contains check in the
> below code snippet was the culprit.
> {code:java}
> this.filteredNames = new HashSet<>(filterFields);
> this.filtered = new ArrayList<>(filterFields.size());
> for (FieldInfo fi : src) {
> if (filterFields.contains(fi.name)) {
> {code}
> A simple change as below seems to have fixed my issue
> {code:java}
> this.filteredNames = new HashSet<>(filterFields);
> this.filtered = new ArrayList<>(filterFields.size());
> for (FieldInfo fi : src) {
> if (this.filteredNames.contains(fi.name)) {
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8558) Adding NumericDocValuesFields is slowing down the indexing process significantly
[
https://issues.apache.org/jira/browse/LUCENE-8558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673879#comment-16673879
]
Kranthi commented on LUCENE-8558:
-
[~dnhatn] I'm preparing the patch. Will submit it soon
> Adding NumericDocValuesFields is slowing down the indexing process
> significantly
>
>
> Key: LUCENE-8558
> URL: https://issues.apache.org/jira/browse/LUCENE-8558
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/index
>Affects Versions: 7.4, 7.5
>Reporter: Kranthi
>Priority: Major
> Labels: patch, performance
> Fix For: 7.4, 7.5
>
>
> The indexing time for my ~2M documents has gone up significantly when I
> started adding fields of type NumericDocValuesField.
>
> Upon debugging found the bottleneck to be in the
> PerFieldMergeState#FilterFieldInfos constructor. The contains check in the
> below code snippet was the culprit.
> {code:java}
> this.filteredNames = new HashSet<>(filterFields);
> this.filtered = new ArrayList<>(filterFields.size());
> for (FieldInfo fi : src) {
> if (filterFields.contains(fi.name)) {
> {code}
> A simple change as below seems to have fixed my issue
> {code:java}
> this.filteredNames = new HashSet<>(filterFields);
> this.filtered = new ArrayList<>(filterFields.size());
> for (FieldInfo fi : src) {
> if (this.filteredNames.contains(fi.name)) {
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8558) Adding NumericDocValuesFields is slowing down the indexing process significantly
[
https://issues.apache.org/jira/browse/LUCENE-8558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kranthi updated LUCENE-8558:
Description:
The indexing time for my ~2M documents has gone up significantly when I started
adding fields of type NumericDocValuesField.
Upon debugging found the bottleneck to be in the
PerFieldMergeState#FilterFieldInfos constructor. The contains check in the
below code snippet was the culprit.
{code:java}
this.filteredNames = new HashSet<>(filterFields);
this.filtered = new ArrayList<>(filterFields.size());
for (FieldInfo fi : src) {
if (filterFields.contains(fi.name)) {
{code}
A simple change as below seems to have fixed my issue
{code:java}
this.filteredNames = new HashSet<>(filterFields);
this.filtered = new ArrayList<>(filterFields.size());
for (FieldInfo fi : src) {
if (this.filteredNames.contains(fi.name)) {
{code}
was:
The indexing time for my ~2M documents has gone up significantly when I started
adding fields of type NumericDocValuesField.
Upon debugging found the bottleneck to be in the
PerFieldMergeState#FilterFieldInfos constructor. The contains check in the
below code snippet was the culprit.
{code:java}
this.filteredNames = new HashSet<>(filterFields);
this.filtered = new ArrayList<>(filterFields.size());
for (FieldInfo fi : src) {
if (filterFields.contains(fi.name)) {
{code}
A simple change to the following seems to have fixed my issue
{code:java}
this.filteredNames = new HashSet<>(filterFields);
this.filtered = new ArrayList<>(filterFields.size());
for (FieldInfo fi : src) {
if (this.filteredNames.contains(fi.name)) {
{code}
> Adding NumericDocValuesFields is slowing down the indexing process
> significantly
>
>
> Key: LUCENE-8558
> URL: https://issues.apache.org/jira/browse/LUCENE-8558
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/index
>Affects Versions: 7.4, 7.5
>Reporter: Kranthi
>Priority: Major
> Labels: patch, performance
> Fix For: 7.4, 7.5
>
>
> The indexing time for my ~2M documents has gone up significantly when I
> started adding fields of type NumericDocValuesField.
>
> Upon debugging found the bottleneck to be in the
> PerFieldMergeState#FilterFieldInfos constructor. The contains check in the
> below code snippet was the culprit.
> {code:java}
> this.filteredNames = new HashSet<>(filterFields);
> this.filtered = new ArrayList<>(filterFields.size());
> for (FieldInfo fi : src) {
> if (filterFields.contains(fi.name)) {
> {code}
> A simple change as below seems to have fixed my issue
> {code:java}
> this.filteredNames = new HashSet<>(filterFields);
> this.filtered = new ArrayList<>(filterFields.size());
> for (FieldInfo fi : src) {
> if (this.filteredNames.contains(fi.name)) {
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-8558) Adding NumericDocValuesFields is slowing down the indexing process significantly
Kranthi created LUCENE-8558:
---
Summary: Adding NumericDocValuesFields is slowing down the
indexing process significantly
Key: LUCENE-8558
URL: https://issues.apache.org/jira/browse/LUCENE-8558
Project: Lucene - Core
Issue Type: Improvement
Components: core/index
Affects Versions: 7.5, 7.4
Reporter: Kranthi
Fix For: 7.5, 7.4
The indexing time for my ~2M documents has gone up significantly when I started
adding fields of type NumericDocValuesField.
Upon debugging found the bottleneck to be in the
PerFieldMergeState#FilterFieldInfos constructor. The contains check in the
below code snippet was the culprit.
{code:java}
this.filteredNames = new HashSet<>(filterFields);
this.filtered = new ArrayList<>(filterFields.size());
for (FieldInfo fi : src) {
if (filterFields.contains(fi.name)) {
{code}
A simple change to the following seems to have fixed my issue
{code:java}
this.filteredNames = new HashSet<>(filterFields);
this.filtered = new ArrayList<>(filterFields.size());
for (FieldInfo fi : src) {
if (this.filteredNames.contains(fi.name)) {
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-9343) facet not working after implementing filters and stopwords
[ https://issues.apache.org/jira/browse/SOLR-9343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kranthi updated SOLR-9343: -- Priority: Blocker (was: Major) > facet not working after implementing filters and stopwords > -- > > Key: SOLR-9343 > URL: https://issues.apache.org/jira/browse/SOLR-9343 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Facet Module, faceting >Affects Versions: 5.4 >Reporter: kranthi >Priority: Blocker > > i m very new to solr search engine.I m working on solr 5.4 version. > when i m working on faceting its seems everything cool. > after i moved to complex level of searching, i implemented filters and > stopwords facet not working. > can any one suggest is there any relation between filters and faceting. > or any limitations are there for faceting. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-9343) facet not working after implementing filters and stopwords
kranthi created SOLR-9343: - Summary: facet not working after implementing filters and stopwords Key: SOLR-9343 URL: https://issues.apache.org/jira/browse/SOLR-9343 Project: Solr Issue Type: Bug Security Level: Public (Default Security Level. Issues are Public) Components: Facet Module, faceting Affects Versions: 5.4 Reporter: kranthi i m very new to solr search engine.I m working on solr 5.4 version. when i m working on faceting its seems everything cool. after i moved to complex level of searching, i implemented filters and stopwords facet not working. can any one suggest is there any relation between filters and faceting. or any limitations are there for faceting. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
