[
https://issues.apache.org/jira/browse/TIKA-4595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18047905#comment-18047905
]
ASF GitHub Bot commented on TIKA-4595:
--------------------------------------
nddipiazza commented on PR #2488:
URL: https://github.com/apache/tika/pull/2488#issuecomment-3694844408
## 🔧 Critical Fix Added
Fixed a bug where dynamically created fetchers via gRPC were not available
in the forked PipesServer process.
### The Problem
- `saveFetcher` was only saving to the gRPC server's local FetcherManager
- When `pipesClient.process()` forks a new PipesServer, that process has its
own FetcherManager
- Result: `FetcherNotFoundException: Can't find fetcher for
id=defaultFetcher`
### The Solution
- `saveFetcher` now calls **both** `fetcherManager.saveFetcher()` AND
`pipesClient.saveFetcher()`
- This propagates the fetcher to the forked PipesServer via the socket
protocol
- Also implemented `deleteFetcher` properly (was previously a no-op)
### Testing
The e2e test should now pass - fetchers created via gRPC will be available
for document processing.
> Add dynamic fetcher management API to PipesClient
> -------------------------------------------------
>
> Key: TIKA-4595
> URL: https://issues.apache.org/jira/browse/TIKA-4595
> Project: Tika
> Issue Type: New Feature
> Components: tika-pipes
> Reporter: Nicholas DiPiazza
> Assignee: Nicholas DiPiazza
> Priority: Major
>
> h2. Overview
> Add API to PipesClient for dynamically creating, updating, and deleting
> fetchers at runtime through PipesServer's ConfigStore.
> h2. Current State
> * PipesServer already has ConfigStore infrastructure
> * FetcherManager and EmitterManager support runtime modifications
> * But PipesClient has no API to expose these capabilities to users
> h2. Desired Architecture
> {noformat}
> PipesClient API
> ↓
> PipesServer (forked process)
> ↓
> ConfigStore (memory, Ignite, etc.)
> {noformat}
> h2. Requirements
> # PipesClient provides public API for fetcher CRUD operations
> # All operations are sent to PipesServer via socket protocol
> # PipesServer handles requests and updates ConfigStore
> # Static fetchers from tika-config.xml/json loaded at startup
> # Dynamic fetchers managed through ConfigStore
> # Both static and dynamic fetchers available for use
> h2. Benefits
> * Users can add/modify fetchers without restarting
> * Supports multi-tenant scenarios with isolated fetcher configs
> * Enables programmatic fetcher configuration
> * Maintains backwards compatibility with static config
> h2. Implementation Tasks
> See linked sub-tasks for detailed implementation steps.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
