[jira] [Updated] (SOLR-6803) Pivot Performance
[ https://issues.apache.org/jira/browse/SOLR-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cassandra Targett updated SOLR-6803: Component/s: faceting > Pivot Performance > - > > Key: SOLR-6803 > URL: https://issues.apache.org/jira/browse/SOLR-6803 > Project: Solr > Issue Type: Bug > Components: faceting >Affects Versions: 5.1 >Reporter: Neil Ireson >Priority: Minor > Attachments: PivotPerformanceTest.java > > > I found that my pivot search for terms per day was taking an age so I knocked > up a quick test, using a collection of 1 million documents with a different > number of random terms and times, to compare different ways of getting the > counts. > 1) Combined = combining the term and time in a single field. > 2) Facet = for each term set the query to the term and then get the time > facet > 3) Pivot = use the term/time pivot facet. > The following two tables present the results for version 4.9.1 vs 4.10.1, as > an average of five runs. > 4.9.1 (Processing time in ms) > |Values (#) | Combined (ms)| Facet (ms)| Pivot (ms)| > |100 |22|21|52| > |1000 | 178|57| 115| > |1 | 1363| 211| 310| > |10| 2592| 1009| 978| > |50| 3125| 3753| 2476| > |100 | 3957| 6789| 3725| > 4.10.1 (Processing time in ms) > |Values (#) | Combined (ms)| Facet (ms)| Pivot (ms)| > |100 |21|21|75| > |1000 | 188|60| 265| > |1 | 1438| 215| 1826| > |10| 2768| 1073| 16594| > |50| 3266| 3686| 99682| > |100 | 4080| 6777|208873| > The results show that, as the number of pivot values increases (i.e. number > of terms * number of times), pivot performance in 4.10.1 get progressively > worse. > I tried to look at the code but there was a lot of changes in pivoting > between 4.9 and 4.10, and so it is not clear to me what has cause the > performance issues. However the results seem to indicate that if the pivot > was simply a combined facet search, it could potentially produce better and > more robust performance. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6803) Pivot Performance
[ https://issues.apache.org/jira/browse/SOLR-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neil Ireson updated SOLR-6803: -- Affects Version/s: (was: 4.10.2) 5.1 Pivot Performance - Key: SOLR-6803 URL: https://issues.apache.org/jira/browse/SOLR-6803 Project: Solr Issue Type: Bug Affects Versions: 5.1 Reporter: Neil Ireson Priority: Minor Attachments: PivotPerformanceTest.java I found that my pivot search for terms per day was taking an age so I knocked up a quick test, using a collection of 1 million documents with a different number of random terms and times, to compare different ways of getting the counts. 1) Combined = combining the term and time in a single field. 2) Facet = for each term set the query to the term and then get the time facet 3) Pivot = use the term/time pivot facet. The following two tables present the results for version 4.9.1 vs 4.10.1, as an average of five runs. 4.9.1 (Processing time in ms) |Values (#) | Combined (ms)| Facet (ms)| Pivot (ms)| |100 |22|21|52| |1000 | 178|57| 115| |1 | 1363| 211| 310| |10| 2592| 1009| 978| |50| 3125| 3753| 2476| |100 | 3957| 6789| 3725| 4.10.1 (Processing time in ms) |Values (#) | Combined (ms)| Facet (ms)| Pivot (ms)| |100 |21|21|75| |1000 | 188|60| 265| |1 | 1438| 215| 1826| |10| 2768| 1073| 16594| |50| 3266| 3686| 99682| |100 | 4080| 6777|208873| The results show that, as the number of pivot values increases (i.e. number of terms * number of times), pivot performance in 4.10.1 get progressively worse. I tried to look at the code but there was a lot of changes in pivoting between 4.9 and 4.10, and so it is not clear to me what has cause the performance issues. However the results seem to indicate that if the pivot was simply a combined facet search, it could potentially produce better and more robust performance. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6803) Pivot Performance
[ https://issues.apache.org/jira/browse/SOLR-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neil Ireson updated SOLR-6803: -- Attachment: (was: PivotPerformanceTest.java) Pivot Performance - Key: SOLR-6803 URL: https://issues.apache.org/jira/browse/SOLR-6803 Project: Solr Issue Type: Bug Affects Versions: 4.10.2 Reporter: Neil Ireson Priority: Minor I found that my pivot search for terms per day was taking an age so I knocked up a quick test, using a collection of 1 million documents with a different number of random terms and times, to compare different ways of getting the counts. 1) Combined = combining the term and time in a single field. 2) Facet = for each term set the query to the term and then get the time facet 3) Pivot = use the term/time pivot facet. The following two tables present the results for version 4.9.1 vs 4.10.1, as an average of five runs. 4.9.1 (Processing time in ms) |Values (#) | Combined (ms)| Facet (ms)| Pivot (ms)| |100 |22|21|52| |1000 | 178|57| 115| |1 | 1363| 211| 310| |10| 2592| 1009| 978| |50| 3125| 3753| 2476| |100 | 3957| 6789| 3725| 4.10.1 (Processing time in ms) |Values (#) | Combined (ms)| Facet (ms)| Pivot (ms)| |100 |21|21|75| |1000 | 188|60| 265| |1 | 1438| 215| 1826| |10| 2768| 1073| 16594| |50| 3266| 3686| 99682| |100 | 4080| 6777|208873| The results show that, as the number of pivot values increases (i.e. number of terms * number of times), pivot performance in 4.10.1 get progressively worse. I tried to look at the code but there was a lot of changes in pivoting between 4.9 and 4.10, and so it is not clear to me what has cause the performance issues. However the results seem to indicate that if the pivot was simply a combined facet search, it could potentially produce better and more robust performance. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6803) Pivot Performance
[ https://issues.apache.org/jira/browse/SOLR-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neil Ireson updated SOLR-6803: -- Attachment: PivotPerformanceTest.java Remove some pointless queries and properly output results Pivot Performance - Key: SOLR-6803 URL: https://issues.apache.org/jira/browse/SOLR-6803 Project: Solr Issue Type: Bug Affects Versions: 4.10.2 Reporter: Neil Ireson Priority: Minor Attachments: PivotPerformanceTest.java I found that my pivot search for terms per day was taking an age so I knocked up a quick test, using a collection of 1 million documents with a different number of random terms and times, to compare different ways of getting the counts. 1) Combined = combining the term and time in a single field. 2) Facet = for each term set the query to the term and then get the time facet 3) Pivot = use the term/time pivot facet. The following two tables present the results for version 4.9.1 vs 4.10.1, as an average of five runs. 4.9.1 (Processing time in ms) |Values (#) | Combined (ms)| Facet (ms)| Pivot (ms)| |100 |22|21|52| |1000 | 178|57| 115| |1 | 1363| 211| 310| |10| 2592| 1009| 978| |50| 3125| 3753| 2476| |100 | 3957| 6789| 3725| 4.10.1 (Processing time in ms) |Values (#) | Combined (ms)| Facet (ms)| Pivot (ms)| |100 |21|21|75| |1000 | 188|60| 265| |1 | 1438| 215| 1826| |10| 2768| 1073| 16594| |50| 3266| 3686| 99682| |100 | 4080| 6777|208873| The results show that, as the number of pivot values increases (i.e. number of terms * number of times), pivot performance in 4.10.1 get progressively worse. I tried to look at the code but there was a lot of changes in pivoting between 4.9 and 4.10, and so it is not clear to me what has cause the performance issues. However the results seem to indicate that if the pivot was simply a combined facet search, it could potentially produce better and more robust performance. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6803) Pivot Performance
[ https://issues.apache.org/jira/browse/SOLR-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hoss Man updated SOLR-6803: --- Description: I found that my pivot search for terms per day was taking an age so I knocked up a quick test, using a collection of 1 million documents with a different number of random terms and times, to compare different ways of getting the counts. 1) Combined = combining the term and time in a single field. 2) Facet = for each term set the query to the term and then get the time facet 3) Pivot = use the term/time pivot facet. The following two tables present the results for version 4.9.1 vs 4.10.1, as an average of five runs. 4.9.1 (Processing time in ms) |Values (#) | Combined (ms)| Facet (ms)| Pivot (ms)| |100 |22|21|52| |1000 | 178|57| 115| |1 | 1363| 211| 310| |10| 2592| 1009| 978| |50| 3125| 3753| 2476| |100 | 3957| 6789| 3725| 4.10.1 (Processing time in ms) |Values (#) | Combined (ms)| Facet (ms)| Pivot (ms)| |100 |21|21|75| |1000 | 188|60| 265| |1 | 1438| 215| 1826| |10| 2768| 1073| 16594| |50| 3266| 3686| 99682| |100 | 4080| 6777|208873| The results show that, as the number of pivot values increases (i.e. number of terms * number of times), pivot performance in 4.10.1 get progressively worse. I tried to look at the code but there was a lot of changes in pivoting between 4.9 and 4.10, and so it is not clear to me what has cause the performance issues. However the results seem to indicate that if the pivot was simply a combined facet search, it could potentially produce better and more robust performance. was: I found that my pivot search for terms per day was taking an age so I knocked up a quick test, using a collection of 1 million documents with a different number of random terms and times, to compare different ways of getting the counts. 1) Combined = combining the term and time in a single field. 2) Facet = for each term set the query to the term and then get the time facet 3) Pivot = use the term/time pivot facet. The following two tables present the results for version 4.9.1 vs 4.10.1, as an average of five runs. 4.9.1 | Processing time in ms | Values| Combined| Facet| Pivot| 100 |22|21|52| 1000 | 178|57| 115| 1 | 1363| 211| 310| 10| 2592| 1009| 978| 50| 3125| 3753| 2476| 100 | 3957| 6789| 3725| 4.10.1 | Processing time in ms | Values| Combined| Facet| Pivot| 100 |21|21|75| 1000 | 188|60| 265| 1 | 1438| 215| 1826| 10| 2768| 1073| 16594| 50| 3266| 3686| 99682| 100 | 4080| 6777|208873| The results show that, as the number of pivot values increases (i.e. number of terms * number of times), pivot performance in 4.10.1 get progressively worse. I tried to look at the code but there was a lot of changes in pivoting between 4.9 and 4.10, and so it is not clear to me what has cause the performance issues. However the results seem to indicate that if the pivot was simply a combined facet search, it could potentially produce better and more robust performance. Neil: I tried to fix the issue description so the tables are formatted correctly (as best as i can understand what the data ment) .. please confirm they look the way you ment. Pivot Performance - Key: SOLR-6803 URL: https://issues.apache.org/jira/browse/SOLR-6803 Project: Solr Issue Type: Bug Affects Versions: 4.10.2 Reporter: Neil Ireson Priority: Minor I found that my pivot search for terms per day was taking an age so I knocked up a quick test, using a collection of 1 million documents with a different number of random terms and times, to compare different ways of getting the counts. 1) Combined = combining the term and time in a single field. 2) Facet = for each term set the query to the term and then get the time facet 3) Pivot = use the term/time pivot facet. The following two tables present the results for version 4.9.1 vs 4.10.1, as an average of five runs. 4.9.1 (Processing time in ms) |Values (#) | Combined (ms)| Facet (ms)| Pivot (ms)| |100 |22|21|52| |1000 | 178|57| 115| |1 | 1363| 211| 310| |10| 2592| 1009|
[jira] [Updated] (SOLR-6803) Pivot Performance
[ https://issues.apache.org/jira/browse/SOLR-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neil Ireson updated SOLR-6803: -- Attachment: PivotPerformanceTest.java I've attached my test file... I just used the example solr configuration which comes with the distribution, I used the 4.9.1 version for both tests. Pivot Performance - Key: SOLR-6803 URL: https://issues.apache.org/jira/browse/SOLR-6803 Project: Solr Issue Type: Bug Affects Versions: 4.10.2 Reporter: Neil Ireson Priority: Minor Attachments: PivotPerformanceTest.java I found that my pivot search for terms per day was taking an age so I knocked up a quick test, using a collection of 1 million documents with a different number of random terms and times, to compare different ways of getting the counts. 1) Combined = combining the term and time in a single field. 2) Facet = for each term set the query to the term and then get the time facet 3) Pivot = use the term/time pivot facet. The following two tables present the results for version 4.9.1 vs 4.10.1, as an average of five runs. 4.9.1 (Processing time in ms) |Values (#) | Combined (ms)| Facet (ms)| Pivot (ms)| |100 |22|21|52| |1000 | 178|57| 115| |1 | 1363| 211| 310| |10| 2592| 1009| 978| |50| 3125| 3753| 2476| |100 | 3957| 6789| 3725| 4.10.1 (Processing time in ms) |Values (#) | Combined (ms)| Facet (ms)| Pivot (ms)| |100 |21|21|75| |1000 | 188|60| 265| |1 | 1438| 215| 1826| |10| 2768| 1073| 16594| |50| 3266| 3686| 99682| |100 | 4080| 6777|208873| The results show that, as the number of pivot values increases (i.e. number of terms * number of times), pivot performance in 4.10.1 get progressively worse. I tried to look at the code but there was a lot of changes in pivoting between 4.9 and 4.10, and so it is not clear to me what has cause the performance issues. However the results seem to indicate that if the pivot was simply a combined facet search, it could potentially produce better and more robust performance. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
