Manybubbles has uploaded a new change for review.
https://gerrit.wikimedia.org/r/172941
Change subject: Create offsets for redirects
......................................................................
Create offsets for redirects
This should speed up highlighting for pages with many many many redirects
at the cost of a little disk space.
DEPLOYMENT: This requires an in place reindex to take effect.
DEPLOYMENT: Probably worth doing on wikis that we've seen slow prefix
searches. This is certainly the cause of some of that and might be the
cause of all of it. Wikis are eswiki and frwiki. May as well do dewiki
and enwiki too while we're at it.
Change-Id: I1eacfcb41b622b9ed0462acb69cb4a6b96cf8912
---
M includes/Maintenance/MappingConfigBuilder.php
M tests/browser/features/prefix_search.feature
M tests/browser/features/step_definitions/page_steps.rb
3 files changed, 37 insertions(+), 10 deletions(-)
git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/CirrusSearch
refs/changes/41/172941/1
diff --git a/includes/Maintenance/MappingConfigBuilder.php
b/includes/Maintenance/MappingConfigBuilder.php
index 864728b..fa7c6f5 100644
--- a/includes/Maintenance/MappingConfigBuilder.php
+++ b/includes/Maintenance/MappingConfigBuilder.php
@@ -285,16 +285,6 @@
),
)
);
- if ( $this->optimizeForExperimentalHighlighter ) {
- if ( $options &
MappingConfigBuilder::SPEED_UP_HIGHLIGHTING ) {
- $field[ 'index_options' ] = 'offsets';
- $field[ 'fields' ][ 'plain' ][ 'index_options'
] = 'offsets';
- }
- } else {
- // We use the FVH on all fields so turn on term vectors
- $field[ 'term_vector' ] = 'with_positions_offsets';
- $field[ 'fields' ][ 'plain' ][ 'term_vector' ] =
'with_positions_offsets';
- }
$disableNorms = ( $options & MappingConfigBuilder::ENABLE_NORMS
) === 0;
if ( $disableNorms ) {
$disableNorms = array( 'norms' => array( 'enabled' =>
false ) );
@@ -319,6 +309,26 @@
$field[ 'fields' ][ $extraName ],
$disableNorms );
}
}
+ if ( $this->optimizeForExperimentalHighlighter ) {
+ if ( $options &
MappingConfigBuilder::SPEED_UP_HIGHLIGHTING ) {
+ $field[ 'index_options' ] = 'offsets';
+ $fieldNames = array( 'plain', 'prefix',
'prefix_asciifolding', 'near_match', 'near_match_asciifolding' );
+ foreach ( $fieldNames as $fieldName ) {
+ if ( isset( $field[ 'fields' ][
$fieldName ] ) ) {
+ $field[ 'fields' ][ $fieldName
][ 'index_options' ] = 'offsets';
+ }
+ }
+ }
+ } else {
+ // We use the FVH on all fields so turn on term vectors
+ $field[ 'term_vector' ] = 'with_positions_offsets';
+ $fieldNames = array( 'plain', 'prefix',
'prefix_asciifolding', 'near_match', 'near_match_asciifolding' );
+ foreach ( $fieldNames as $fieldName ) {
+ if ( isset( $field[ 'fields' ][ $fieldName ] )
) {
+ $field[ 'fields' ][ $fieldName ][
'term_vector' ] = 'with_positions_offsets';
+ }
+ }
+ }
return $field;
}
diff --git a/tests/browser/features/prefix_search.feature
b/tests/browser/features/prefix_search.feature
index e593453..5954919 100644
--- a/tests/browser/features/prefix_search.feature
+++ b/tests/browser/features/prefix_search.feature
@@ -83,3 +83,11 @@
| áccent Sorting | Áccent Sorting | Accent Sorting |
| Accent Sorting | Accent Sorting | Áccent Sorting |
| accent Sorting | Accent Sorting | Áccent Sorting |
+
+ # Just take too long to run on a regular basis
+ # @redirect @huge
+ # Scenario: Prefix search on pages with tons of redirects is reasonably fast
+ # Given a page named IHaveTonsOfRedirects exists
+ # And there are 1000 redirects to IHaveTonsOfRedirects of the form
TonsOfRedirects%s
+ # When I type TonsOfRedirects into the search box
+ # Then suggestions should appear
diff --git a/tests/browser/features/step_definitions/page_steps.rb
b/tests/browser/features/step_definitions/page_steps.rb
index 2ed2819..4b76117 100644
--- a/tests/browser/features/step_definitions/page_steps.rb
+++ b/tests/browser/features/step_definitions/page_steps.rb
@@ -8,6 +8,15 @@
edit_page(title, description, false) # Make sure the description is
correct
end
+Given(/^there are (\d+) redirects to (.+) of the form (.+)$/) do |count,
target, form|
+ count = count.to_i
+ text = '#REDIRECT [[%s]]' % target
+ for i in 1..count do
+ new_tital = form % i
+ edit_page(new_tital, text, false)
+ end
+end
+
Given(/^a page named (.*) doesn't exist$/) do |title|
step("I delete #{title}")
end
--
To view, visit https://gerrit.wikimedia.org/r/172941
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I1eacfcb41b622b9ed0462acb69cb4a6b96cf8912
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/CirrusSearch
Gerrit-Branch: master
Gerrit-Owner: Manybubbles <[email protected]>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
