This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch publish-airflow-docs-latest-layout-2025-12 in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 4cfb5cf0be4004726a08948cd3e42c9f8901b20a Author: Kaxil Naik <[email protected]> AuthorDate: Sat Dec 20 13:01:09 2025 +0000 Add fast client-side search to Airflow documentation (#59658) I have been frustrated by Sphinx search for a long-long time. So after adding dark-mode, this was next in my list! This PR/commit introduces a fast, fully client-side search experience for the Apache Airflow documentation, powered by [Pagefind](https://pagefind.app/). The new search is keyboard-accessible (Cmd+K / Ctrl+K), works offline, and requires no external services. Search indexes are generated automatically at documentation build time and loaded entirely in the browser, enabling sub-50 ms queries even on large docs. I have kept the Sphinx search too as a backup and it will keep functioning. ---- Add keyboard-accessible search (Cmd+K) to Apache Airflow documentation with automatic indexing and offline support. New Sphinx extension: `pagefind_search` Located in `devel-common/src/sphinx_exts/pagefind_search/`: - __init__.py: Extension setup with configuration values and event handlers - builder.py: Automatic index building with graceful fallback - static/css/pagefind.css: Search modal and button styling with dark mode support - static/js/search.js: Search functionality with keyboard shortcuts - templates/search-modal.html: Search modal HTML template - Keyboard shortcut (Cmd+K/Ctrl+K) opens search modal - Arrow key navigation through results - Works offline (no external services) - Automatic indexing during documentation build - Dark mode support - Sub-50ms search performance - Configurable content indexing via conf.py Users can now: - Press Cmd+K from any documentation page to search - Navigate results with arrow keys, Enter to select, Esc to close - Search works immediately without network requests - Results show page title, breadcrumb, and excerpt Available in conf.py: - pagefind_enabled: Toggle search indexing - pagefind_verbose: Enable build logging - pagefind_root_selector: Define searchable content area - pagefind_exclude_selectors: Exclude navigation, headers, footers - pagefind_custom_records: Index non-HTML content (PDFs, etc.) (cherry picked from commit d0bd2df6d194a8b923fb55b2f37107642c261b40) --- Dockerfile | 8 +- Dockerfile.ci | 8 +- airflow-core/docs/conf.py | 8 + devel-common/pyproject.toml | 1 + devel-common/src/docs/utils/conf_constants.py | 1 + .../src/sphinx_exts/pagefind_search/README.md | 165 +++++++ .../src/sphinx_exts/pagefind_search/__init__.py | 103 ++++ .../src/sphinx_exts/pagefind_search/builder.py | 215 +++++++++ .../pagefind_search/static/css/pagefind.css | 529 +++++++++++++++++++++ .../pagefind_search/static/js/search.js | 228 +++++++++ .../pagefind_search/templates/search-modal.html | 48 ++ .../pagefind_search/templates/searchbox.html | 33 ++ .../docker/install_airflow_when_building_images.sh | 8 +- 13 files changed, 1349 insertions(+), 6 deletions(-) diff --git a/Dockerfile b/Dockerfile index 9b1f3afa64b..37e99edd016 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1235,9 +1235,13 @@ function install_airflow_when_building_images() { set +x common::install_packaging_tools echo - echo "${COLOR_BLUE}Running 'pip check'${COLOR_RESET}" + echo "${COLOR_BLUE}Running 'uv pip check'${COLOR_RESET}" echo - pip check + # Here we should use `pip check` not `uv pip check` to detect any incompatibilities that might happen + # between `pip` and `uv` installations + # However, in the current version of `pip` there is a bug that incorrectly detects `pagefind-bin` as unsupported + # https://github.com/pypa/pip/issues/13709 -> once this is fixed, we should bring `pip check` back. + uv pip check } common::get_colors diff --git a/Dockerfile.ci b/Dockerfile.ci index c4921b7df90..5d252318da4 100644 --- a/Dockerfile.ci +++ b/Dockerfile.ci @@ -989,9 +989,13 @@ function install_airflow_when_building_images() { set +x common::install_packaging_tools echo - echo "${COLOR_BLUE}Running 'pip check'${COLOR_RESET}" + echo "${COLOR_BLUE}Running 'uv pip check'${COLOR_RESET}" echo - pip check + # Here we should use `pip check` not `uv pip check` to detect any incompatibilities that might happen + # between `pip` and `uv` installations + # However, in the current version of `pip` there is a bug that incorrectly detects `pagefind-bin` as unsupported + # https://github.com/pypa/pip/issues/13709 -> once this is fixed, we should bring `pip check` back. + uv pip check } common::get_colors diff --git a/airflow-core/docs/conf.py b/airflow-core/docs/conf.py index 74c09b31c7a..4d505bf6654 100644 --- a/airflow-core/docs/conf.py +++ b/airflow-core/docs/conf.py @@ -268,6 +268,14 @@ global_substitutions = { "experimental": "This is an :ref:`experimental feature <experimental>`.", } +# Pagefind search configuration +pagefind_exclude_patterns = [ + "_api/**", # Exclude auto-generated API documentation + "_modules/**", # Exclude source code modules + "release_notes.html", # Exclude changelog aggregation page + "genindex.html", # Exclude generated index +] + # -- Options for sphinx.ext.autodoc -------------------------------------------- # See: https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html diff --git a/devel-common/pyproject.toml b/devel-common/pyproject.toml index ed8c483f4c4..e37be8e44f4 100644 --- a/devel-common/pyproject.toml +++ b/devel-common/pyproject.toml @@ -76,6 +76,7 @@ dependencies = [ "rich-click>=1.7.1", "click>=8.1.8", "docutils>=0.21", + "pagefind[bin]", "sphinx-airflow-theme@https://github.com/apache/airflow-site/releases/download/0.3.0/sphinx_airflow_theme-0.3.0-py3-none-any.whl";, "sphinx-argparse>=0.4.0", "sphinx-autoapi>=3", diff --git a/devel-common/src/docs/utils/conf_constants.py b/devel-common/src/docs/utils/conf_constants.py index 1b8285460e8..77b4426363c 100644 --- a/devel-common/src/docs/utils/conf_constants.py +++ b/devel-common/src/docs/utils/conf_constants.py @@ -91,6 +91,7 @@ BASIC_SPHINX_EXTENSIONS = [ "redirects", "substitution_extensions", "sphinx_design", + "pagefind_search", ] SPHINX_REDOC_EXTENSIONS = [ diff --git a/devel-common/src/sphinx_exts/pagefind_search/README.md b/devel-common/src/sphinx_exts/pagefind_search/README.md new file mode 100644 index 00000000000..b01e3c077ac --- /dev/null +++ b/devel-common/src/sphinx_exts/pagefind_search/README.md @@ -0,0 +1,165 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + --> + +# Pagefind Search Extension + +A Sphinx extension providing fast, self-hosted search for Apache Airflow documentation using [Pagefind](https://pagefind.app/). + +## Features + +- **Automatic indexing** when docs are built +- **Cmd+K search** with modern UI and keyboard shortcuts +- **Self-hosted** - No third-party services +- **Content weighting** - Prioritizes titles and headings for better relevance +- **Optimized ranking** - Tuned for exact phrase matching and title matches +- **Playground support** - Debug and tune search behavior + +## Usage + +### Search Interface + +- **Keyboard**: Press `Cmd+K` (Mac) or `Ctrl+K` (Windows/Linux) +- **Mouse**: Click the search button in the header +- **Navigation**: Arrow keys to navigate, Enter to select, Esc to close + +## Configuration + +In Sphinx's `conf.py`: + +```python +# Enable/disable search (default: True) +pagefind_enabled = True + +# Verbose logging (default: False) +pagefind_verbose = False + +# Content selector (default: "main") +pagefind_root_selector = "main" + +# Exclude selectors (default: see below) +# These elements won't be included in the search index +pagefind_exclude_selectors = [ + ".headerlink", # Permalink icons + ".toctree-wrapper", # Table of contents navigation + "nav", # All navigation elements + "footer", # Footer content + ".td-sidebar", # Left sidebar + ".breadcrumb", # Breadcrumb navigation + ".navbar", # Top navigation bar + ".dropdown-menu", # Dropdown menus (version selector, etc.) + ".docs-version-selector", # Version selector widget + "[role='navigation']", # ARIA navigation landmarks + ".d-print-none", # Print-hidden elements (usually UI controls) + ".pagefind-search-button", # Search button itself +] + +# File pattern (default: "**/*.html") +pagefind_glob = "**/*.html" + +# Exclude patterns (default: []) +# Path patterns to exclude from indexing (e.g., auto-generated API docs) +# Note: File-by-file indexing is used when patterns are specified (slightly slower but precise) +# Pagefind does NOT automatically exclude underscore-prefixed directories +pagefind_exclude_patterns = [ + "_api/**", # Exclude API documentation + "_modules/**", # Exclude source code modules + "release_notes.html", # Exclude specific files + "genindex.html", # Exclude generated index +] + +# Content weighting (default: True) +# Uses lightweight regex to add data-pagefind-weight attributes to titles and headings +pagefind_content_weighting = True + +# Enable playground (default: False) +# Creates a playground at /_pagefind/playground/ for debugging search +pagefind_enable_playground = False + +# Custom records for non-HTML content (default: []) +pagefind_custom_records = [ + { + "url": "/downloads/guide.pdf", + "content": "PDF content...", + "language": "en", + "meta": {"title": "Guide PDF"}, + } +] +``` + +### Ranking Optimization + +The extension uses optimized ranking parameters in `search.js`: + +- **termFrequency: 1.0** - Standard term occurrence weighting +- **termSaturation: 0.7** - Moderate saturation to prevent over-rewarding repetition +- **termSimilarity: 7.5** - Maximum boost for exact phrase matches and similar terms +- **pageLength: 0** - No penalty for longer pages (important for reference documentation) + +Combined with content weighting (10x for titles, 9x for h1, 7x for h2), these settings ensure exact title matches rank highly even for very long pages. + +## Architecture + +### Build Process + +1. Sphinx builds HTML documentation +2. Extension copies CSS/JS to `_static/` +3. Extension injects search modal HTML into pages +4. (Optional) Extension adds content weights to HTML files +5. Extension builds Pagefind index with configured options + +### Runtime + +1. User presses Cmd+K or clicks search button +2. JavaScript loads Pagefind library dynamically +3. Search executes client-side +4. Results rendered in modal + +## Troubleshooting + +### Search not working + +1. Check Pagefind is installed: `python -c "import pagefind; print('OK')"` +2. Check index exists: `ls generated/_build/docs/apache-airflow/stable/_pagefind/` +3. Enable verbose logging: `pagefind_verbose = True` in `conf.py` + +### Index not created + +- Ensure `pagefind_enabled = True` in `conf.py` +- Check build logs for errors +- If `pagefind[bin]` unavailable, use: `npx pagefind --site <build-dir>` + +### Poor search ranking + +1. Enable playground: `pagefind_enable_playground = True` in `conf.py` +2. Rebuild docs and access playground at `/_pagefind/playground/` +3. Test queries and analyze ranking scores +4. Ensure `pagefind_content_weighting = True` (default) +5. Check that titles and headings contain expected keywords + +### Debugging with Playground + +The playground provides detailed insights: + +- View all indexed pages +- See ranking scores for each result +- Analyze impact of different search terms +- Verify content weighting is applied +- Test ranking parameter changes + +Example, access at: `http://localhost:8000/docs/apache-airflow/stable/_pagefind/playground/` diff --git a/devel-common/src/sphinx_exts/pagefind_search/__init__.py b/devel-common/src/sphinx_exts/pagefind_search/__init__.py new file mode 100644 index 00000000000..acfe12ea1d8 --- /dev/null +++ b/devel-common/src/sphinx_exts/pagefind_search/__init__.py @@ -0,0 +1,103 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +"""Sphinx extension for Pagefind search integration.""" + +from __future__ import annotations + +from pathlib import Path +from typing import TYPE_CHECKING, Any + +if TYPE_CHECKING: + from sphinx.application import Sphinx + +from sphinx_exts.pagefind_search.builder import build_index_finished, copy_static_files + + +def register_templates(app: Sphinx, config) -> None: + """Register template directory for searchbox override.""" + template_dir = str(Path(__file__).parent / "templates") + # Prepend so our template overrides Sphinx's default searchbox + if template_dir not in config.templates_path: + config.templates_path.insert(0, template_dir) + + +def inject_search_html(app: Sphinx, pagename: str, templatename: str, context: dict, doctree) -> None: + """Inject Pagefind search modal HTML into page context.""" + template_dir = Path(__file__).parent / "templates" + modal_file = template_dir / "search-modal.html" + + if not modal_file.exists(): + return + + modal_content = modal_file.read_text() + + from jinja2 import Template + + modal_template = Template(modal_content) + search_modal_html = modal_template.render(pathto=context.get("pathto")) + + context["pagefind_search_modal"] = search_modal_html + + if "body" in context: + context["body"] = search_modal_html + context["body"] + + +def setup(app: Sphinx) -> dict[str, Any]: + """Setup the Pagefind search extension.""" + app.add_config_value("pagefind_enabled", True, "html", [bool]) + app.add_config_value("pagefind_verbose", False, "html", [bool]) + app.add_config_value("pagefind_root_selector", "main", "html", [str]) + app.add_config_value( + "pagefind_exclude_selectors", + [ + ".headerlink", + ".toctree-wrapper", + "nav", + "footer", + ".td-sidebar", + ".breadcrumb", + ".navbar", + ".dropdown-menu", + ".docs-version-selector", + "[role='navigation']", + ".d-print-none", + ".pagefind-search-button", + ], + "html", + [list], + ) + app.add_config_value("pagefind_glob", "**/*.html", "html", [str]) + app.add_config_value("pagefind_exclude_patterns", [], "html", [list]) + app.add_config_value("pagefind_custom_records", [], "html", [list]) + app.add_config_value("pagefind_content_weighting", True, "html", [bool]) + app.add_config_value("pagefind_enable_playground", False, "html", [bool]) + + app.add_css_file("css/pagefind.css") + app.add_js_file("js/search.js") + + # Register template directory for searchbox override + app.connect("config-inited", register_templates) + app.connect("html-page-context", inject_search_html) + app.connect("build-finished", copy_static_files, priority=100) + app.connect("build-finished", build_index_finished, priority=900) + + return { + "version": "1.0.0", + "parallel_read_safe": True, + "parallel_write_safe": False, + } diff --git a/devel-common/src/sphinx_exts/pagefind_search/builder.py b/devel-common/src/sphinx_exts/pagefind_search/builder.py new file mode 100644 index 00000000000..066123bd980 --- /dev/null +++ b/devel-common/src/sphinx_exts/pagefind_search/builder.py @@ -0,0 +1,215 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +"""Pagefind index builder and static file handler.""" + +from __future__ import annotations + +import asyncio +import logging +import re +from pathlib import Path +from typing import TYPE_CHECKING + +from pagefind.index import IndexConfig, PagefindIndex +from sphinx.util.fileutil import copy_asset + +if TYPE_CHECKING: + from sphinx.application import Sphinx + +logger = logging.getLogger(__name__) + + +def add_content_weights_lightweight( + output_dir: Path, glob_pattern: str, exclude_patterns: list[str] | None = None +) -> int: + """Add data-pagefind-weight attributes using simple regex replacement. + + :param output_dir: Output directory + :param glob_pattern: Glob pattern + :param exclude_patterns: Exclude patterns + :return: Number of files processed + """ + files_processed = 0 + exclude_patterns = exclude_patterns or [] + + # Regex patterns to match opening tags without existing weight attribute + # Use maximum valid weights (0.0-10.0 range, quadratic scale) + # https://pagefind.app/docs/weighting/ + # Weight of 10.0 = ~100x impact, 7.0 = ~49x impact (default h1), 5.0 = ~25x impact + patterns = [ + (re.compile(r"<title(?![^>]*data-pagefind-weight)"), '<title data-pagefind-weight="10.0"'), + (re.compile(r"<h1(?![^>]*data-pagefind-weight)"), '<h1 data-pagefind-weight="9.0"'), + ] + + for html_file in output_dir.glob(glob_pattern): + if not html_file.is_file(): + continue + + # Check if file matches any exclude pattern (using simple prefix matching) + relative_path = html_file.relative_to(output_dir) + relative_str = str(relative_path) + if any(relative_str.startswith(pattern.rstrip("/*")) for pattern in exclude_patterns): + continue + + try: + content = html_file.read_text(encoding="utf-8") + modified_content = content + + for pattern, replacement in patterns: + modified_content = pattern.sub(replacement, modified_content) + + if modified_content != content: + html_file.write_text(modified_content, encoding="utf-8") + files_processed += 1 + + except Exception as e: + logger.warning("Failed to add weights to %s: %s", html_file, e) + + return files_processed + + +async def build_pagefind_index(app: Sphinx) -> dict[str, int]: + """Build Pagefind search index using Python API.""" + output_dir = Path(app.builder.outdir) + pagefind_dir = output_dir / "_pagefind" + + # Add content weighting if enabled + if getattr(app.config, "pagefind_content_weighting", True): + logger.info("Adding content weights to HTML files...") + exclude_patterns = getattr(app.config, "pagefind_exclude_patterns", []) + files_processed = add_content_weights_lightweight( + output_dir, app.config.pagefind_glob, exclude_patterns + ) + logger.info("Added content weights to %s files", files_processed) + + config = IndexConfig( + root_selector=app.config.pagefind_root_selector, + exclude_selectors=app.config.pagefind_exclude_selectors, + output_path=str(pagefind_dir), + verbose=app.config.pagefind_verbose, + force_language=app.config.language or "en", + keep_index_url=False, + write_playground=getattr(app.config, "pagefind_enable_playground", False), + ) + + logger.info("Building Pagefind search index...") + + exclude_patterns = getattr(app.config, "pagefind_exclude_patterns", []) + + if exclude_patterns: + # Need to index files individually to apply exclusion patterns + logger.info("Indexing with exclusion patterns: %s", exclude_patterns) + indexed = 0 + skipped = 0 + + async with PagefindIndex(config=config) as index: + for html_file in output_dir.glob(app.config.pagefind_glob): + if not html_file.is_file(): + continue + + relative_path = html_file.relative_to(output_dir) + relative_str = str(relative_path) + + # Check if path matches any exclude pattern (prefix matching) + if any(relative_str.startswith(pattern.rstrip("/*")) for pattern in exclude_patterns): + skipped += 1 + continue + + try: + content = html_file.read_text(encoding="utf-8") + await index.add_html_file( + content=content, + source_path=str(html_file), + url=str(relative_path), + ) + indexed += 1 + except Exception as e: + logger.warning("Failed to index %s: %s", relative_path, e) + + logger.info("Pagefind indexed %s pages (excluded %s)", indexed, skipped) + + if app.config.pagefind_custom_records: + for record in app.config.pagefind_custom_records: + try: + await index.add_custom_record(**record) + except Exception as e: + logger.warning("Failed to add custom record: %s", e) + + return {"page_count": indexed} + else: + # No exclusions - use fast directory indexing + async with PagefindIndex(config=config) as index: + result = await index.add_directory(path=str(output_dir), glob=app.config.pagefind_glob) + page_count = result.get("page_count", 0) + logger.info("Pagefind indexed %s pages", page_count) + + if app.config.pagefind_custom_records: + for record in app.config.pagefind_custom_records: + try: + await index.add_custom_record(**record) + except Exception as e: + logger.warning("Failed to add custom record: %s", e) + + return {"page_count": page_count} + + +def build_index_finished(app: Sphinx, exception: Exception | None) -> None: + """Build Pagefind index after HTML build completes.""" + if exception: + logger.info("Skipping Pagefind indexing due to build errors") + return + + if app.builder.format != "html": + return + + if not app.config.pagefind_enabled: + logger.info("Pagefind indexing disabled (pagefind_enabled=False)") + return + + try: + result = asyncio.run(build_pagefind_index(app)) + page_count = result.get("page_count", 0) + + if page_count == 0: + raise RuntimeError("Pagefind indexing failed: no pages were indexed") + + logger.info("✓ Pagefind index created with %s pages", page_count) + except Exception as e: + logger.exception("Failed to build Pagefind index") + raise RuntimeError(f"Pagefind indexing failed: {e}") from e + + +def copy_static_files(app: Sphinx, exception: Exception | None) -> None: + """Copy CSS and JS files to _static directory.""" + if exception or app.builder.format != "html": + return + + static_dir = Path(app.builder.outdir) / "_static" + extension_static = Path(__file__).parent / "static" + + css_src = extension_static / "css" / "pagefind.css" + css_dest = static_dir / "css" + css_dest.mkdir(parents=True, exist_ok=True) + if css_src.exists(): + copy_asset(str(css_src), str(css_dest)) + + js_src = extension_static / "js" / "search.js" + js_dest = static_dir / "js" + js_dest.mkdir(parents=True, exist_ok=True) + if js_src.exists(): + copy_asset(str(js_src), str(js_dest)) diff --git a/devel-common/src/sphinx_exts/pagefind_search/static/css/pagefind.css b/devel-common/src/sphinx_exts/pagefind_search/static/css/pagefind.css new file mode 100644 index 00000000000..aa72e9440fb --- /dev/null +++ b/devel-common/src/sphinx_exts/pagefind_search/static/css/pagefind.css @@ -0,0 +1,529 @@ +/*! + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +/* + * Pagefind Search Modal - Cmd+K Search + * Minimal styling for search functionality only + */ + +/* Search Modal */ +.search-modal { + position: fixed; + inset: 0; + z-index: 9999; + display: none; + align-items: flex-start; + padding-top: 15vh; +} + +.search-modal.active { + display: flex; +} + +.search-modal__backdrop { + position: absolute; + inset: 0; + background: rgba(0, 0, 0, 0.6); + backdrop-filter: blur(4px); + cursor: pointer; +} + +.search-modal__container { + position: relative; + width: 90%; + max-width: 600px; + margin: 0 auto; + background: white; + border-radius: 8px; + box-shadow: 0 20px 60px rgba(0, 0, 0, 0.3); + overflow: hidden; + border: 1px solid #dee2e6; +} + +[data-bs-theme="dark"] .search-modal__container { + background: #161b22; + border-color: #30363d; +} + +.search-modal__header { + display: flex; + align-items: center; + gap: 12px; + padding: 16px 20px; + border-bottom: 1px solid #dee2e6; +} + +[data-bs-theme="dark"] .search-modal__header { + border-bottom-color: #30363d; +} + +.search-modal__icon { + color: #6c757d; + flex-shrink: 0; + width: 18px; + height: 18px; +} + +.search-modal__input { + flex: 1; + font-size: 16px; + border: none; + background: transparent; + color: #1a1a1a; + font-family: inherit; +} + +[data-bs-theme="dark"] .search-modal__input { + color: #e6edf3; +} + +.search-modal__input:focus { + outline: none; +} + +.search-modal__input::placeholder { + color: #adb5bd; +} + +[data-bs-theme="dark"] .search-modal__input::placeholder { + color: #6e7681; +} + +.search-modal__close { + background: transparent; + border: none; + color: #6c757d; + cursor: pointer; + padding: 6px; + border-radius: 4px; + transition: background 0.2s; + display: flex; + align-items: center; + justify-content: center; + width: 28px; + height: 28px; +} + +.search-modal__close:hover { + background: #f8f9fa; +} + +[data-bs-theme="dark"] .search-modal__close { + color: #8b949e; +} + +[data-bs-theme="dark"] .search-modal__close:hover { + background: #21262d; +} + +.search-modal__close svg { + width: 16px; + height: 16px; +} + +.search-modal__results { + max-height: 400px; + overflow-y: auto; + padding: 8px; +} + +.search-modal__result-item { + padding: 10px 12px; + border-radius: 4px; + cursor: pointer; + display: flex; + gap: 10px; + text-decoration: none; + color: inherit; + transition: background 0.2s; + margin-bottom: 4px; +} + +.search-modal__result-item:hover, +.search-modal__result-item--selected { + background: #f8f9fa; +} + +[data-bs-theme="dark"] .search-modal__result-item:hover, +[data-bs-theme="dark"] .search-modal__result-item--selected { + background: #21262d; +} + +.search-modal__result-icon { + color: #6c757d; + flex-shrink: 0; + margin-top: 2px; + width: 14px; + height: 14px; +} + +.search-modal__result-icon svg { + width: 14px; + height: 14px; +} + +.search-modal__result-content { + flex: 1; + min-width: 0; +} + +.search-modal__result-title { + font-weight: 600; + color: #1a1a1a; + margin-bottom: 2px; + font-size: 14px; +} + +[data-bs-theme="dark"] .search-modal__result-title { + color: #e6edf3; +} + +.search-modal__result-breadcrumb { + font-size: 11px; + color: #6c757d; + margin-bottom: 4px; +} + +[data-bs-theme="dark"] .search-modal__result-breadcrumb { + color: #8b949e; +} + +.search-modal__result-excerpt { + font-size: 12px; + color: #6c757d; + line-height: 1.5; + overflow: hidden; + text-overflow: ellipsis; + display: -webkit-box; + -webkit-line-clamp: 2; + -webkit-box-orient: vertical; +} + +[data-bs-theme="dark"] .search-modal__result-excerpt { + color: #8b949e; +} + +.search-modal__no-results { + text-align: center; + padding: 48px 24px; + color: #6c757d; +} + +[data-bs-theme="dark"] .search-modal__no-results { + color: #8b949e; +} + +.search-modal__no-results p:first-child { + font-size: 15px; + font-weight: 600; + color: #1a1a1a; + margin-bottom: 8px; +} + +[data-bs-theme="dark"] .search-modal__no-results p:first-child { + color: #e6edf3; +} + +.search-modal__no-results-hint { + font-size: 13px; + color: #adb5bd; +} + +[data-bs-theme="dark"] .search-modal__no-results-hint { + color: #6e7681; +} + +.search-modal__footer { + padding: 10px 16px; + border-top: 1px solid #dee2e6; + background: #f8f9fa; +} + +[data-bs-theme="dark"] .search-modal__footer { + border-top-color: #30363d; + background: #0d1117; +} + +.search-modal__shortcuts { + display: flex; + gap: 16px; + justify-content: center; + font-size: 12px; + color: #495057; + font-weight: 500; +} + +[data-bs-theme="dark"] .search-modal__shortcuts { + color: #c9d1d9; +} + +.search-modal__shortcuts span { + display: flex; + align-items: center; + gap: 6px; +} + +.search-modal__shortcuts kbd { + padding: 3px 8px; + background: white; + border: 1px solid #dee2e6; + border-radius: 4px; + font-family: 'SF Mono', 'Monaco', 'Inconsolata', 'Roboto Mono', monospace; + margin: 0; + font-size: 11px; + font-weight: 600; + color: #495057; + box-shadow: 0 1px 2px rgba(0, 0, 0, 0.05); +} + +[data-bs-theme="dark"] .search-modal__shortcuts kbd { + background: #21262d; + border-color: #30363d; + color: #c9d1d9; + box-shadow: 0 1px 2px rgba(0, 0, 0, 0.3); +} + +/* ============================================ + Search Button in Header - FIXED POSITIONING + ============================================ */ + +.search-button { + display: flex; + align-items: center; + gap: 8px; + padding: 6px 12px; + border: 1px solid rgba(255, 255, 255, 0.2); + border-radius: 6px; + background: rgba(255, 255, 255, 0.1); + color: #dee2e6; + cursor: pointer; + transition: all 0.2s ease; + font-size: 14px; + font-family: inherit; + height: 36px; + margin-left: 16px; +} + +.search-button:hover { + background: rgba(255, 255, 255, 0.15); + border-color: rgba(255, 255, 255, 0.3); + color: white; +} + +.search-button__icon { + flex-shrink: 0; + width: 16px; + height: 16px; +} + +.search-button__text { + font-weight: 400; + font-size: 14px; + color: inherit; +} + +.search-button__shortcut { + display: flex; + gap: 2px; + margin-left: 8px; + padding: 2px 6px; + background: rgba(255, 255, 255, 0.15); + border: 1px solid rgba(255, 255, 255, 0.3); + border-radius: 4px; +} + +.search-button__shortcut span { + font-size: 11px; + font-family: 'SF Mono', 'Monaco', 'Inconsolata', 'Roboto Mono', monospace; + font-weight: 600; + line-height: 1.2; + color: rgba(255, 255, 255, 0.9); +} + +/* Light theme adjustments */ +[data-bs-theme="light"] .search-button { + border-color: #dee2e6; + background: white; + color: #495057; +} + +[data-bs-theme="light"] .search-button:hover { + background: #f8f9fa; + border-color: #adb5bd; + color: #212529; +} + +[data-bs-theme="light"] .search-button__shortcut { + background: #e9ecef; + border-color: #adb5bd; +} + +[data-bs-theme="light"] .search-button__shortcut span { + color: #495057; + font-weight: 600; +} + +@media (max-width: 768px) { + .search-button__text, + .search-button__shortcut { + display: none; + } + + .search-button { + padding: 6px; + width: 36px; + justify-content: center; + margin-left: 8px; + } +} + +/* Floating Search Button (Fallback) */ +.pagefind-search-button { + position: fixed; + top: 20px; + right: 20px; + z-index: 9998; + display: flex; + align-items: center; + gap: 8px; + padding: 8px 14px; + background: #017cee; + color: white; + border: 1px solid rgba(1, 124, 238, 0.2); + border-radius: 6px; + box-shadow: 0 2px 8px rgba(0, 0, 0, 0.15); + cursor: pointer; + font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif; + font-size: 14px; + font-weight: 500; + transition: all 0.2s ease; +} + +.pagefind-search-button:hover { + background: #0166c7; + box-shadow: 0 4px 12px rgba(1, 124, 238, 0.25); + transform: translateY(-1px); +} + +.pagefind-search-button__icon { + flex-shrink: 0; + width: 18px; + height: 18px; +} + +.pagefind-search-button__text { + color: white; + font-weight: 500; +} + +.pagefind-search-button__shortcut { + display: flex; + gap: 2px; + padding: 3px 7px; + background: rgba(255, 255, 255, 0.2); + border: 1px solid rgba(255, 255, 255, 0.3); + border-radius: 4px; + font-family: 'SF Mono', 'Monaco', 'Inconsolata', 'Roboto Mono', monospace; + font-size: 11px; + font-weight: 600; + line-height: 1; +} + +.pagefind-search-button__shortcut span { + color: white; +} + +@media (max-width: 768px) { + .pagefind-search-button__text, + .pagefind-search-button__shortcut { + display: none; + } + + .pagefind-search-button { + width: 44px; + height: 44px; + padding: 10px; + justify-content: center; + } +} + +/* Sidebar Search Button - Replaces Sphinx searchbox */ +.sidebar-search-container { + padding: 0; + margin-bottom: 1rem; +} + +.sidebar-search-button { + position: static; + width: 100%; + justify-content: center; + padding: 10px 12px; + border-radius: 6px; +} + +.sidebar-search-button .pagefind-search-button__text { + flex: 1; +} + +/* Light mode - subtle styling */ +[data-bs-theme="light"] .sidebar-search-button { + background: white; + border-color: #dee2e6; + color: #495057; + box-shadow: 0 1px 2px rgba(0, 0, 0, 0.05); +} + +[data-bs-theme="light"] .sidebar-search-button:hover { + background: #f8f9fa; + border-color: #adb5bd; + color: #212529; +} + +[data-bs-theme="light"] .sidebar-search-button .pagefind-search-button__icon { + stroke: #6c757d; +} + +[data-bs-theme="light"] .sidebar-search-button .pagefind-search-button__text { + color: #495057; +} + +[data-bs-theme="light"] .sidebar-search-button .pagefind-search-button__shortcut { + background: #e9ecef; + border-color: #adb5bd; +} + +[data-bs-theme="light"] .sidebar-search-button .pagefind-search-button__shortcut span { + color: #495057; +} + +/* Dark mode - keep blue for visibility */ +[data-bs-theme="dark"] .sidebar-search-button { + background: #0166c7; +} + +[data-bs-theme="dark"] .sidebar-search-button:hover { + background: #017cee; +} + +/* Hide floating button when sidebar button exists */ +body:has(header .search-button) .pagefind-search-button, +body:has(nav .search-button) .pagefind-search-button, +body:has(.sidebar-search-button) .pagefind-search-button:not(.sidebar-search-button) { + display: none; +} diff --git a/devel-common/src/sphinx_exts/pagefind_search/static/js/search.js b/devel-common/src/sphinx_exts/pagefind_search/static/js/search.js new file mode 100644 index 00000000000..4cffcd11c3a --- /dev/null +++ b/devel-common/src/sphinx_exts/pagefind_search/static/js/search.js @@ -0,0 +1,228 @@ +/*! + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +let pagefind = null; +let searchResults = []; +let selectedIndex = 0; + +async function initPagefind() { + if (!pagefind) { + try { + const pagefindPath = window.PAGEFIND_PATH || './_pagefind/pagefind.js'; + const absoluteUrl = new URL(pagefindPath, document.baseURI || window.location.href).href; + + const pf = await import(/* webpackIgnore: true */ absoluteUrl); + pagefind = pf; + await pagefind.options({ + excerptLength: 15, + ranking: { + termFrequency: 1.0, + termSaturation: 0.7, + termSimilarity: 7.5, // Maximum boost for exact/similar matches + pageLength: 0 // No penalty for long pages + } + }); + } catch (e) { + console.error('Failed to load Pagefind:', e); + displaySearchError('Search index not available. Please rebuild the documentation.'); + return null; + } + } + return pagefind; +} + +function displaySearchError(message) { + const resultsDiv = document.getElementById('search-results'); + if (resultsDiv) { + resultsDiv.innerHTML = ` + <div class="search-modal__no-results"> + <p>${message}</p> + </div> + `; + } +} + +function openSearch() { + const modal = document.getElementById('search-modal'); + const input = document.getElementById('search-input'); + if (modal && input) { + modal.style.display = 'flex'; + input.focus(); + document.body.style.overflow = 'hidden'; + } +} + +function closeSearch() { + const modal = document.getElementById('search-modal'); + const input = document.getElementById('search-input'); + const results = document.getElementById('search-results'); + + if (modal) modal.style.display = 'none'; + if (input) input.value = ''; + if (results) results.innerHTML = ''; + + document.body.style.overflow = ''; + searchResults = []; + selectedIndex = 0; +} + +function debounce(func, wait) { + let timeout; + return function executedFunction(...args) { + const later = () => { + clearTimeout(timeout); + func(...args); + }; + clearTimeout(timeout); + timeout = setTimeout(later, wait); + }; +} + +async function performSearch(query) { + const resultsContainer = document.getElementById('search-results'); + if (!resultsContainer) return; + + if (!query || query.length < 2) { + resultsContainer.innerHTML = ''; + return; + } + + const pf = await initPagefind(); + if (!pf) { + resultsContainer.innerHTML = ` + <div class="search-modal__no-results"> + <p>Search index not available</p> + <p class="search-modal__no-results-hint">Index is built automatically during 'make html'</p> + </div> + `; + return; + } + + try { + const search = await pf.search(query); + searchResults = await Promise.all( + search.results.slice(0, 10).map(r => r.data()) + ); + + renderResults(searchResults); + } catch (e) { + console.error('Search error:', e); + resultsContainer.innerHTML = ` + <div class="search-modal__no-results"> + <p>Search temporarily unavailable</p> + <p class="search-modal__no-results-hint">Please try again later</p> + </div> + `; + } +} + +function renderResults(results) { + const container = document.getElementById('search-results'); + if (!container) return; + + if (results.length === 0) { + container.innerHTML = ` + <div class="search-modal__no-results"> + <p>No results found</p> + <p class="search-modal__no-results-hint">Try different keywords or check spelling</p> + </div> + `; + return; + } + + container.innerHTML = results.map((result, index) => ` + <a + href="${result.url}" + class="search-modal__result-item ${index === selectedIndex ? 'search-modal__result-item--selected' : ''}" + data-index="${index}" + role="option" + aria-selected="${index === selectedIndex}" + > + <div class="search-modal__result-icon"> + <svg width="16" height="16" viewBox="0 0 16 16" fill="currentColor"> + <path d="M1 2.5A1.5 1.5 0 0 1 2.5 1h11A1.5 1.5 0 0 1 15 2.5v11a1.5 1.5 0 0 1-1.5 1.5h-11A1.5 1.5 0 0 1 1 13.5v-11zM2.5 2a.5.5 0 0 0-.5.5v11a.5.5 0 0 0 .5.5h11a.5.5 0 0 0 .5-.5v-11a.5.5 0 0 0-.5-.5h-11z"/> + <path d="M4 5.5a.5.5 0 0 1 .5-.5h7a.5.5 0 0 1 0 1h-7a.5.5 0 0 1-.5-.5zm0 3a.5.5 0 0 1 .5-.5h7a.5.5 0 0 1 0 1h-7a.5.5 0 0 1-.5-.5zm0 3a.5.5 0 0 1 .5-.5h4a.5.5 0 0 1 0 1h-4a.5.5 0 0 1-.5-.5z"/> + </svg> + </div> + <div class="search-modal__result-content"> + <div class="search-modal__result-title">${result.meta?.title || 'Untitled'}</div> + <div class="search-modal__result-breadcrumb">${result.url.replace(/^\/docs\//, '').replace(/\.html$/, '')}</div> + ${result.excerpt ? `<div class="search-modal__result-excerpt">${result.excerpt}</div>` : ''} + </div> + </a> + `).join(''); +} + +function handleKeyboardNav(e) { + if (e.key === 'ArrowDown') { + e.preventDefault(); + selectedIndex = Math.min(selectedIndex + 1, searchResults.length - 1); + renderResults(searchResults); + scrollToSelected(); + } else if (e.key === 'ArrowUp') { + e.preventDefault(); + selectedIndex = Math.max(selectedIndex - 1, 0); + renderResults(searchResults); + scrollToSelected(); + } else if (e.key === 'Enter' && searchResults.length > 0) { + e.preventDefault(); + window.location.href = searchResults[selectedIndex].url; + } else if (e.key === 'Escape') { + closeSearch(); + } +} + +function scrollToSelected() { + const selected = document.querySelector('.search-modal__result-item--selected'); + if (selected) { + selected.scrollIntoView({ block: 'nearest', behavior: 'smooth' }); + } +} + +document.addEventListener('DOMContentLoaded', () => { + document.addEventListener('keydown', (e) => { + if ((e.metaKey || e.ctrlKey) && e.key === 'k') { + e.preventDefault(); + openSearch(); + } + }); + + const searchInput = document.getElementById('search-input'); + if (searchInput) { + searchInput.addEventListener('input', debounce((e) => { + selectedIndex = 0; + performSearch(e.target.value); + }, 150)); + + searchInput.addEventListener('keydown', handleKeyboardNav); + } + + const backdrop = document.querySelector('.search-modal__backdrop'); + if (backdrop) { + backdrop.addEventListener('click', closeSearch); + } + + const closeButton = document.querySelector('.search-modal__close'); + if (closeButton) { + closeButton.addEventListener('click', closeSearch); + } +}); + +window.openSearch = openSearch; +window.closeSearch = closeSearch; diff --git a/devel-common/src/sphinx_exts/pagefind_search/templates/search-modal.html b/devel-common/src/sphinx_exts/pagefind_search/templates/search-modal.html new file mode 100644 index 00000000000..b53a1f26a81 --- /dev/null +++ b/devel-common/src/sphinx_exts/pagefind_search/templates/search-modal.html @@ -0,0 +1,48 @@ +{# + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. +#} + +<script> +window.PAGEFIND_PATH = '{{ pathto("", 1) }}_pagefind/pagefind.js'; +</script> + +<div id="search-modal" class="search-modal" role="dialog" aria-modal="true" aria-label="Search documentation"> + <div class="search-modal__backdrop"></div> + <div class="search-modal__container"> + <div class="search-modal__header"> + <svg class="search-modal__icon" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> + <circle cx="9" cy="9" r="7"></circle> + <line x1="14" y1="14" x2="19" y2="19"></line> + </svg> + <input type="text" id="search-input" class="search-modal__input" placeholder="Search documentation..." aria-label="Search documentation input"> + <button class="search-modal__close" onclick="closeSearch()" aria-label="Close search"> + <svg width="16" height="16" viewBox="0 0 16 16" fill="currentColor"> + <path d="M14.354 1.646a.5.5 0 0 0-.708 0L8 7.293 2.354 1.646a.5.5 0 0 0-.708.708L7.293 8l-5.647 5.646a.5.5 0 0 0 .708.708L8 8.707l5.646 5.647a.5.5 0 0 0 .708-.708L8.707 8l5.647-5.646a.5.5 0 0 0 0-.708z"/> + </svg> + </button> + </div> + <div id="search-results" class="search-modal__results" role="listbox" aria-live="polite"></div> + <div class="search-modal__footer"> + <div class="search-modal__shortcuts"> + <span><kbd>↑↓</kbd> Navigate</span> + <span><kbd>⏎</kbd> Select</span> + <span><kbd>Esc</kbd> Close</span> + </div> + </div> + </div> +</div> diff --git a/devel-common/src/sphinx_exts/pagefind_search/templates/searchbox.html b/devel-common/src/sphinx_exts/pagefind_search/templates/searchbox.html new file mode 100644 index 00000000000..0c3ec660b09 --- /dev/null +++ b/devel-common/src/sphinx_exts/pagefind_search/templates/searchbox.html @@ -0,0 +1,33 @@ +{# + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. +#} + +{# Override default Sphinx searchbox with Pagefind button #} +<div class="sidebar-search-container"> + <button class="pagefind-search-button sidebar-search-button" onclick="openSearch()" aria-label="Search documentation" title="Search documentation (Cmd+K or Ctrl+K)"> + <svg class="pagefind-search-button__icon" width="18" height="18" viewBox="0 0 20 20" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> + <circle cx="9" cy="9" r="7"></circle> + <line x1="14" y1="14" x2="19" y2="19"></line> + </svg> + <span class="pagefind-search-button__text">Search docs</span> + <kbd class="pagefind-search-button__shortcut"> + <span>⌘</span> + <span>K</span> + </kbd> + </button> +</div> diff --git a/scripts/docker/install_airflow_when_building_images.sh b/scripts/docker/install_airflow_when_building_images.sh index c93c0387a63..fb7c1d86db1 100644 --- a/scripts/docker/install_airflow_when_building_images.sh +++ b/scripts/docker/install_airflow_when_building_images.sh @@ -200,9 +200,13 @@ function install_airflow_when_building_images() { set +x common::install_packaging_tools echo - echo "${COLOR_BLUE}Running 'pip check'${COLOR_RESET}" + echo "${COLOR_BLUE}Running 'uv pip check'${COLOR_RESET}" echo - pip check + # Here we should use `pip check` not `uv pip check` to detect any incompatibilities that might happen + # between `pip` and `uv` installations + # However, in the current version of `pip` there is a bug that incorrectly detects `pagefind-bin` as unsupported + # https://github.com/pypa/pip/issues/13709 -> once this is fixed, we should bring `pip check` back. + uv pip check } common::get_colors
