This is an automated email from the ASF dual-hosted git repository.
ndipiazza pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tika.git
The following commit(s) were added to refs/heads/main by this push:
new fa258938a TIKA-4587: Add development mode support for tika-grpc with
plugin hot-reloading (#2481)
fa258938a is described below
commit fa258938ab492e6994a4fbfbde65f9067a477850
Author: Nicholas DiPiazza <[email protected]>
AuthorDate: Fri Dec 26 09:41:56 2025 -0600
TIKA-4587: Add development mode support for tika-grpc with plugin
hot-reloading (#2481)
* TIKA-4587: Add pf4j development mode support to TikaPluginManager
Enables plugin development without packaging as ZIP files by supporting
pf4j's DEVELOPMENT runtime mode. This allows developers to point
plugin-roots directly at unpackaged plugin directories (e.g.,
target/classes)
for faster iteration during development.
Features:
- Configure via system property: -Dtika.plugin.dev.mode=true
- Configure via environment variable: TIKA_PLUGIN_DEV_MODE=true
- Skips ZIP extraction when in development mode
- Logs mode on startup for visibility
- Defaults to DEPLOYMENT mode for backward compatibility
This aligns with pf4j best practices documented at:
https://pf4j.org/doc/development-mode.html
JIRA: https://issues.apache.org/jira/browse/TIKA-4587
* TIKA-4587: Add development mode documentation to tika-grpc README
Adds comprehensive guide on using pf4j development mode for plugin
development:
- How to enable development mode (system property or env var)
- Configuration examples with plugin-roots pointing to target/classes
- Complete development workflow
- Multiple plugins example
- Troubleshooting tips
- Switching between dev and production modes
* TIKA-4587: Add IntelliJ IDEA setup guide for loading all plugins in dev
mode
Adds comprehensive IntelliJ configuration example:
- Complete dev-config.json with ALL plugin class directories
- Step-by-step IntelliJ Run Configuration setup
- VM options and environment variable configuration
- Hot reload workflow for fast iteration
- Shell script to auto-generate config with all plugins
* TIKA-4587: Clarify that plugin-roots is in Tika config JSON
Added explicit note that plugin-roots is configured in the Tika
configuration JSON file (e.g., tika-config.json, dev-config.json)
* Add Distributed Configuration with Apache Ignite section to tika-grpc
README
Restored documentation about:
- Ignite-based distributed configuration storage
- ConfigStore types (memory, ignite, custom)
- Maven dependency for tika-ignite-config-store
- Running with Ignite
- Cluster behavior and configuration sharing
* TIKA-4587: Fix development mode to prevent scanning subdirectories
Override createPluginRepository() to prevent pf4j from scanning
subdirectories
in development mode. In dev mode, each path in plugin-roots should be
treated
as a complete plugin directory (target/classes), not as a container of
plugins.
This fixes the error:
'No PluginDescriptorFinder for plugin .../target/classes/org'
The default DevelopmentPluginRepository scans for subdirectories, but we
want
each configured path to be the plugin root itself.
* TIKA-4587: Fix ignite plugin compilation and improve dev mode
- Fixed ignite plugin compilation by ensuring tika-pipes-core is built first
- Set pf4j.mode system property in TikaPluginManager.load() to properly
enable pf4j's DEVELOPMENT mode
- Override createPluginDescriptorFinder() to use
PropertiesPluginDescriptorFinder
in development mode (looks for plugin.properties instead of MANIFEST.MF)
- Add Maven dev profile in tika-grpc for easy development server startup
- Add dev-config.json with all 13 plugin directories
Maven dev profile usage:
cd tika-pipes/tika-pipes-plugins && mvn compile
cd ../../tika-grpc && mvn compile exec:java -Pdev
Successfully loads plugins from target/classes in development mode!
* TIKA-4587: Add development mode support for tika-grpc with plugin
hot-reloading
- Add dev profile to tika-grpc/pom.xml with plugin dependencies
- Include all plugin modules in dev profile to make classes available at
runtime
- Set tika.plugin.dev.mode=true system property to enable PF4J development
mode
- Add run-dev.sh convenience script for starting server in dev mode
- Rename test-config.json to test-dev-config.json for clarity
- Allows running tika-grpc with plugins loaded from target/classes
directories
- Enables rapid development without building plugin ZIPs
* Fix README: Remove invalid ${project.basedir} examples and use correct
relative paths
- Remove duplicate/confusing sections with Maven property placeholders
- Update examples to use relative paths (../tika-pipes/...) that actually
work
- Update run-dev.sh to reference correct config file (dev-tika-config.json)
- Simplify documentation and remove redundant content
- Add note that paths are relative to tika-grpc directory
* Expand IntelliJ development guide in README
- Add detailed step-by-step IntelliJ IDEA setup instructions
- Document complete development workflow for making plugin changes
- Explain why server restart is required (PF4J loads plugins at startup)
- Add tips for faster development (build shortcuts, debug mode, keyboard
shortcuts)
- Include common issues and solutions section
- Add alternative Maven exec approach
- Clarify fast iteration cycle: ~20-25 seconds vs 2-3 minutes for ZIP
packaging
* Update TikaGrpcServer run configuration to use development config file
* Remove commented code for properties-based plugin descriptor finder in
development mode
* Remove unused path for Ignite plugin in development configuration, not
yet merged
* Remove unused path for Ignite plugin in development configuration, not
yet merged
* Fix TikaPluginManager to configure PF4J mode in constructor
- Move configurePf4jRuntimeMode() call to happen before super() via helper
method
- Ensures pf4j.mode system property is set based on tika.plugin.dev.mode
- Explicitly set deployment mode when dev mode is false to ensure clean
state
- Fixes testDevelopmentModeViaSystemProperty test failure
- Allows direct constructor usage (not just via load() methods) to work
correctly
---
.run/TikaGrpcServer.run.xml | 5 +-
tika-grpc/README.md | 383 +++++++++++----------
tika-grpc/dev-tika-config.json | 34 ++
tika-grpc/pom.xml | 92 +++++
tika-grpc/run-dev.sh | 20 ++
.../org/apache/tika/plugins/TikaPluginManager.java | 76 +++-
6 files changed, 412 insertions(+), 198 deletions(-)
diff --git a/.run/TikaGrpcServer.run.xml b/.run/TikaGrpcServer.run.xml
index c189e21be..b50b5a856 100644
--- a/.run/TikaGrpcServer.run.xml
+++ b/.run/TikaGrpcServer.run.xml
@@ -2,7 +2,8 @@
<configuration default="false" name="TikaGrpcServer" type="Application"
factoryName="Application" nameIsGenerated="true">
<option name="MAIN_CLASS_NAME"
value="org.apache.tika.pipes.grpc.TikaGrpcServer" />
<module name="tika-grpc" />
- <option name="PROGRAM_PARAMETERS" value="--config
src/test/resources/tika-pipes-test-config.json" />
+ <option name="PROGRAM_PARAMETERS" value="--config dev-tika-config.json" />
+ <option name="VM_PARAMETERS" value="-Dtika.plugin.dev.mode=true" />
<option name="WORKING_DIRECTORY" value="$MODULE_WORKING_DIR$" />
<extension name="coverage">
<pattern>
@@ -14,4 +15,4 @@
<option name="Make" enabled="true" />
</method>
</configuration>
-</component>
\ No newline at end of file
+</component>
diff --git a/tika-grpc/README.md b/tika-grpc/README.md
index aa4898cc0..a4ab950e3 100644
--- a/tika-grpc/README.md
+++ b/tika-grpc/README.md
@@ -11,12 +11,34 @@ This server will manage a pool of Tika Pipes clients.
* Delete
* Fetch + Parse a given Fetch Item
+## Quick Start - Development Mode
+
+The fastest way to run tika-grpc in development mode with plugin hot-reloading:
+
+```bash
+# 1. Build Tika and all plugins (from tika project root)
+mvn clean install -DskipTests
+
+# 2. Run in development mode (from tika-grpc directory)
+cd tika-grpc
+./run-dev.sh
+```
+
+This will:
+- Automatically enable `tika.plugin.dev.mode=true`
+- Load plugins from `../tika-pipes/tika-pipes-plugins/*/target/classes`
directories
+- Start gRPC server on port 50052
+- Use the `dev-tika-config.json` configuration
+
+You can also specify a custom config file:
+```bash
+./run-dev.sh my-custom-config.json
+```
+
## Plugin Development Mode
When developing plugins, you can use pf4j's development mode to load plugins
directly from their `target/classes` directories without needing to package
them as ZIP files. This significantly speeds up the development cycle.
-**Configuration Location:** The `plugin-roots` setting is specified in your
**Tika configuration JSON file** (commonly named `tika-config.json`,
`dev-config.json`, or similar).
-
### Enabling Development Mode
Set one of the following:
@@ -31,23 +53,27 @@ Set one of the following:
export TIKA_PLUGIN_DEV_MODE=true
```
+**Maven Dev Profile:** (Recommended)
+```bash
+mvn exec:java -Pdev -Dconfig.file=dev-tika-config.json
+```
+
### Configuration Example
-In your Tika configuration JSON, point `plugin-roots` to the unpackaged plugin
directories:
+The `dev-tika-config.json` file shows how to configure plugin-roots with
relative paths:
```json
{
"plugin-roots": [
- "/path/to/tika/tika-pipes/tika-pipes-plugins/tika-pipes-s3/target/classes",
-
"/path/to/tika/tika-pipes/tika-pipes-plugins/tika-pipes-file-system/target/classes",
-
"/path/to/tika/tika-pipes/tika-pipes-plugins/tika-pipes-kafka/target/classes"
+ "../tika-pipes/tika-pipes-plugins/tika-pipes-file-system/target/classes",
+ "../tika-pipes/tika-pipes-plugins/tika-pipes-http/target/classes",
+ "../tika-pipes/tika-pipes-plugins/tika-pipes-s3/target/classes"
],
"fetchers": [
{
- "s3": {
- "myS3Fetcher": {
- "region": "us-east-1",
- "bucket": "my-bucket"
+ "fs": {
+ "myFetcher": {
+ "basePath": "/tmp/input"
}
}
}
@@ -55,6 +81,17 @@ In your Tika configuration JSON, point `plugin-roots` to the
unpackaged plugin d
}
```
+**Note:** Paths are relative to the `tika-grpc` directory (where you run the
server from). You can also use absolute paths if preferred:
+
+```json
+{
+ "plugin-roots": [
+
"/home/user/tika/tika-pipes/tika-pipes-plugins/tika-pipes-s3/target/classes",
+
"/home/user/tika/tika-pipes/tika-pipes-plugins/tika-pipes-file-system/target/classes"
+ ]
+}
+```
+
### Development Workflow
1. **Build the plugin modules** (only needed once or when dependencies change):
@@ -63,35 +100,30 @@ In your Tika configuration JSON, point `plugin-roots` to
the unpackaged plugin d
mvn clean compile
```
-2. **Enable development mode** via system property or environment variable:
- ```bash
- export TIKA_PLUGIN_DEV_MODE=true
- ```
-
-3. **Configure plugin-roots** to point to `target/classes` directories in your
config JSON
-
-4. **Run your application** - plugins will be loaded directly from the class
directories:
+2. **Run in development mode** using the convenience script:
```bash
- java -jar tika-grpc-server.jar --config my-config.json
+ cd tika-grpc
+ ./run-dev.sh
```
-5. **Make code changes** to your plugin
+3. **Make code changes** to your plugin
-6. **Recompile just the changed plugin** (much faster than full rebuild):
+4. **Recompile just the changed plugin** (much faster than full rebuild):
```bash
cd tika-pipes/tika-pipes-plugins/tika-pipes-s3
mvn compile
```
-7. **Restart your application** - changes are immediately picked up
+5. **Restart the server** - changes are immediately picked up
### What Happens in Development Mode
- **ZIP extraction is skipped** - TikaPluginManager doesn't try to unzip
plugins
- **Plugins loaded from directories** - pf4j loads classes directly from
`target/classes`
-- **Each plugin directory must contain** `plugin.properties` in the root
(already present after `mvn compile`)
+- **Each plugin directory must contain** `plugin.properties` in the root
(automatically present after `mvn compile`)
+- **Dependencies are available** - The dev profile includes all plugin modules
as dependencies
-### Example Directory Structure
+### Expected Directory Structure
When pointing to `target/classes`, pf4j expects this structure:
@@ -110,199 +142,169 @@ tika-pipes-s3/target/classes/
└── S3FetcherFactory.class
```
-### Multiple Plugins During Development
-
-You can load multiple plugins simultaneously by listing all their
`target/classes` directories:
-
-```json
-{
- "plugin-roots": [
-
"/home/user/tika/tika-pipes/tika-pipes-plugins/tika-pipes-s3/target/classes",
-
"/home/user/tika/tika-pipes/tika-pipes-plugins/tika-pipes-kafka/target/classes",
-
"/home/user/tika/tika-pipes/tika-pipes-plugins/tika-pipes-opensearch/target/classes"
- ]
-}
-```
+### IntelliJ IDEA Setup
-### IntelliJ IDEA Setup - Loading All Plugins
+For IntelliJ IDEA development, here's a complete workflow for developing
plugins:
-For IntelliJ IDEA development, you can load ALL available plugins at once.
Here's a complete configuration example:
+#### Initial Setup
-#### 1. Create a development configuration JSON file
+1. **Import the Tika project** as a Maven project in IntelliJ
-Create `dev-config.json` in your project root with all plugin class
directories:
+2. **Build all plugins once** (required before first run):
+ - Open the Maven tool window: View → Tool Windows → Maven
+ - Navigate to: **tika-pipes-plugins** (the parent module)
+ - Right-click → Lifecycle → **compile**
+ - Or use terminal:
+ ```bash
+ cd tika-pipes/tika-pipes-plugins
+ mvn clean compile
+ ```
-```json
-{
- "plugin-roots": [
-
"${project.basedir}/tika-pipes/tika-pipes-plugins/tika-pipes-az-blob/target/classes",
-
"${project.basedir}/tika-pipes/tika-pipes-plugins/tika-pipes-csv/target/classes",
-
"${project.basedir}/tika-pipes/tika-pipes-plugins/tika-pipes-file-system/target/classes",
-
"${project.basedir}/tika-pipes/tika-pipes-plugins/tika-pipes-gcs/target/classes",
-
"${project.basedir}/tika-pipes/tika-pipes-plugins/tika-pipes-http/target/classes",
-
"${project.basedir}/tika-pipes/tika-pipes-plugins/tika-pipes-ignite/target/classes",
-
"${project.basedir}/tika-pipes/tika-pipes-plugins/tika-pipes-jdbc/target/classes",
-
"${project.basedir}/tika-pipes/tika-pipes-plugins/tika-pipes-json/target/classes",
-
"${project.basedir}/tika-pipes/tika-pipes-plugins/tika-pipes-kafka/target/classes",
-
"${project.basedir}/tika-pipes/tika-pipes-plugins/tika-pipes-microsoft-graph/target/classes",
-
"${project.basedir}/tika-pipes/tika-pipes-plugins/tika-pipes-opensearch/target/classes",
-
"${project.basedir}/tika-pipes/tika-pipes-plugins/tika-pipes-s3/target/classes",
-
"${project.basedir}/tika-pipes/tika-pipes-plugins/tika-pipes-solr/target/classes"
- ],
- "fetchers": [
- {
- "fs": {
- "myFetcher": {
- "basePath": "/tmp/input"
- }
- }
- }
- ]
-}
-```
+3. **Create a Run Configuration** for tika-grpc:
+ - Go to: Run → Edit Configurations
+ - Click `+` → Application
+ - **Name:** Tika gRPC Server (Dev Mode)
+ - **Main class:** `org.apache.tika.pipes.grpc.TikaGrpcServer`
+ - **VM options:** `-Dtika.plugin.dev.mode=true`
+ - **Program arguments:** `--config dev-tika-config.json`
+ - **Working directory:** `$PROJECT_DIR$/tika-grpc`
+ - **Use classpath of module:** `tika-grpc`
+ - Click OK
-**Note:** If using absolute paths instead of `${project.basedir}`, replace
with your actual Tika project path:
+4. **Run the configuration** - You should see output like:
+ ```
+ INFO TikaPluginManager running in DEVELOPMENT mode
+ INFO PF4J version 3.14.0 in 'development' mode
+ INFO Plugin '[email protected]' resolved
+ INFO Plugin '[email protected]' resolved
+ ...
+ INFO Server started, listening on 50052
+ ```
-```json
-{
- "plugin-roots": [
-
"/home/user/tika/tika-pipes/tika-pipes-plugins/tika-pipes-az-blob/target/classes",
-
"/home/user/tika/tika-pipes/tika-pipes-plugins/tika-pipes-csv/target/classes",
-
"/home/user/tika/tika-pipes/tika-pipes-plugins/tika-pipes-file-system/target/classes",
-
"/home/user/tika/tika-pipes/tika-pipes-plugins/tika-pipes-gcs/target/classes",
-
"/home/user/tika/tika-pipes/tika-pipes-plugins/tika-pipes-http/target/classes",
-
"/home/user/tika/tika-pipes/tika-pipes-plugins/tika-pipes-ignite/target/classes",
-
"/home/user/tika/tika-pipes/tika-pipes-plugins/tika-pipes-jdbc/target/classes",
-
"/home/user/tika/tika-pipes/tika-pipes-plugins/tika-pipes-json/target/classes",
-
"/home/user/tika/tika-pipes/tika-pipes-plugins/tika-pipes-kafka/target/classes",
-
"/home/user/tika/tika-pipes/tika-pipes-plugins/tika-pipes-microsoft-graph/target/classes",
-
"/home/user/tika/tika-pipes/tika-pipes-plugins/tika-pipes-opensearch/target/classes",
-
"/home/user/tika/tika-pipes/tika-pipes-plugins/tika-pipes-s3/target/classes",
-
"/home/user/tika/tika-pipes/tika-pipes-plugins/tika-pipes-solr/target/classes"
- ]
-}
-```
+#### Development Workflow: Making Plugin Changes
-#### 2. Configure IntelliJ Run Configuration
+**Scenario:** You want to modify the S3 Fetcher plugin
-1. **Go to:** Run → Edit Configurations
-2. **Add New Configuration:** Click `+` → Application (or your existing run
configuration)
-3. **Set Main Class:** Your application's main class (e.g.,
`org.apache.tika.grpc.TikaGrpcServer`)
-4. **VM Options:** Add development mode flag:
- ```
- -Dtika.plugin.dev.mode=true
- ```
-5. **Program Arguments:** Point to your config file:
+1. **Navigate to the plugin code:**
```
- --config dev-config.json
+ tika-pipes/tika-pipes-plugins/tika-pipes-s3/src/main/java/
```
-6. **Environment Variables:** (Alternative to VM option)
- ```
- TIKA_PLUGIN_DEV_MODE=true
- ```
-7. **Working Directory:** Set to your Tika project root
+
+2. **Make your code changes** in the plugin source files
+ - Edit fetcher, emitter, or iterator classes
+ - Modify configuration handling
+ - Add new features
+
+3. **Build just the modified plugin module:**
+ - In Project view, right-click on **tika-pipes-s3** module
+ - Select **Build Module 'tika-pipes-s3.main'**
+ - Or use terminal:
+ ```bash
+ cd tika-pipes/tika-pipes-plugins/tika-pipes-s3
+ mvn compile
+ ```
+ - Build time: ~5-10 seconds (much faster than full rebuild!)
+
+4. **Restart the tika-grpc server:**
+ - **You MUST restart** for changes to take effect
+ - Click the stop button (red square) in IntelliJ
+ - Click run button (green arrow) to restart
+ - PF4J loads plugins at startup - no hot reload without restart
+
+5. **Verify your changes:**
+ - Check the server logs for successful plugin loading
+ - Test your changes via gRPC client
+ - Make additional changes and repeat steps 3-4 as needed
+
+#### Tips for Faster Development
+
+**Use IntelliJ's Build shortcuts:**
+- `Ctrl+F9` (Windows/Linux) or `Cmd+F9` (Mac) - Build project/module
+- Configure to build only changed files for even faster iteration
+
+**Debug mode:**
+- Use "Debug" instead of "Run" to set breakpoints in plugin code
+- Step through plugin initialization and execution
+- Inspect variables and plugin state
+
+**Multiple terminal windows:**
+- Terminal 1: Run `./run-dev.sh`
+- Terminal 2: Quick builds with `mvn compile` in plugin directory
+- Restart server with Ctrl+C and up-arrow to re-run
+
+**Keyboard shortcut for restart:**
+- Set up: Preferences → Keymap → Search for "Rerun"
+- Assign a hotkey to "Rerun 'Tika gRPC Server (Dev Mode)'"
+- Quick restart: One keystroke instead of mouse clicking
+
+#### Common Issues and Solutions
+
+**"ClassNotFoundException" after changes:**
+- Make sure you ran **Build Module** on the changed plugin
+- Check that `target/classes` was updated (look at file timestamps)
+- Do a clean compile: `mvn clean compile`
+
+**Changes not visible after restart:**
+- Verify you built the correct module (check module name in IntelliJ)
+- Ensure the config file points to the right `target/classes` directory
+- Check that development mode is enabled (look for "DEVELOPMENT mode" in logs)
+
+**Server won't start:**
+- Build ALL plugins first: `cd tika-pipes/tika-pipes-plugins && mvn compile`
+- Check that `dev-tika-config.json` exists in the working directory
+- Verify working directory is set to `$PROJECT_DIR$/tika-grpc`
+
+**Want to see what changed:**
+- Before restarting, add a log statement to verify your change
+- Check the plugin's version in startup logs
+- Use debugger to set breakpoint in modified code
+
+#### Alternative: Maven Exec from IntelliJ
+
+Instead of creating an Application run configuration, you can also run the
Maven goal directly:
+
+1. **Open Maven tool window**
+2. **Navigate to:** tika-grpc → Profiles → dev (check the checkbox)
+3. **Navigate to:** tika-grpc → Plugins → exec → exec:java
+4. **Right-click exec:java** → Run
+5. **To use custom config:** Edit the run configuration and add:
```
- $PROJECT_DIR$
+ -Dconfig.file=my-config.json
```
-#### 3. Build All Plugins Once
+Same workflow applies: make changes → build module → restart Maven goal
-Before running in IntelliJ, compile all plugins:
+### Why You Must Restart
-```bash
-# From Tika project root
-cd tika-pipes/tika-pipes-plugins
-mvn clean compile
-```
-
-Or use IntelliJ's Maven tool window:
-- Open **Maven** tool window (View → Tool Windows → Maven)
-- Expand **tika-pipes-plugins**
-- Right-click → Lifecycle → **compile**
-
-#### 4. Run in IntelliJ
-
-Click the **Run** button with your configured run configuration. You should
see in the console:
-
-```
-INFO TikaPluginManager running in DEVELOPMENT mode
-INFO PF4J version 3.14.0 in 'development' mode
-INFO Plugin '[email protected]' resolved
-INFO Plugin '[email protected]' resolved
-INFO Plugin '[email protected]' resolved
-...
-```
-
-#### 5. Hot Reload During Development
-
-When you modify plugin code:
-
-1. **Make your changes** in the plugin source code
-2. **Build just that module:** In IntelliJ, right-click the module → Build
Module
-3. **Restart your run configuration** - changes are immediately picked up
-
-No need to rebuild ZIPs or the entire project!
-
-### Shell Script to Generate Config with All Plugins
-
-You can also generate the configuration dynamically:
-
-```bash
-#!/bin/bash
-# generate-dev-config.sh
-
-TIKA_ROOT="/path/to/your/tika" # Update this path
-
-cat > dev-config.json << 'EOF'
-{
- "plugin-roots": [
-EOF
-
-# Add all plugin class directories
-for plugin in $(ls -d
$TIKA_ROOT/tika-pipes/tika-pipes-plugins/*/target/classes 2>/dev/null); do
- echo " \"$plugin\"," >> dev-config.json
-done
-
-# Remove trailing comma from last entry
-sed -i '$ s/,$//' dev-config.json
-
-cat >> dev-config.json << 'EOF'
- ],
- "fetchers": [
- {
- "fs": {
- "defaultFetcher": {
- "basePath": "/tmp"
- }
- }
- }
- ]
-}
-EOF
-
-echo "Generated dev-config.json with all available plugins"
-```
-
-Run it:
-```bash
-chmod +x generate-dev-config.sh
-./generate-dev-config.sh
-```
+**PF4J plugin loading happens at server startup:**
+- `TikaPluginManager.loadPlugins()` discovers and loads plugins from
directories
+- Creates `PluginClassLoader` instances for each plugin
+- Instantiates plugin classes and calls `start()` methods
+- This happens **once** when the server starts
+**No hot reload support (yet):**
+- PF4J doesn't automatically detect changed class files
+- ClassLoaders are not refreshed during runtime
+- Server must be restarted to reload plugin code
+**Fast iteration cycle:**
+- Compile plugin: ~5-10 seconds
+- Restart server: ~10-15 seconds
+- Total: ~20-25 seconds per change
+- Much faster than packaging ZIPs (2-3 minutes) or full rebuild (5+ minutes)
### Switching Back to Production Mode
For production deployments, use packaged ZIP files:
-1. **Remove or set development mode to false**:
+1. **Remove or set development mode to false:**
```bash
unset TIKA_PLUGIN_DEV_MODE
# OR
export TIKA_PLUGIN_DEV_MODE=false
```
-2. **Build plugin ZIPs**:
+2. **Build plugin ZIPs:**
```bash
cd tika-pipes/tika-pipes-plugins
mvn clean package
@@ -335,8 +337,11 @@ For production deployments, use packaged ZIP files:
- Restart the application
- Check that you're editing the correct plugin module
+**ClassNotFoundException errors?**
+- Make sure you built all plugins first with `mvn clean install -DskipTests`
+- The dev profile includes all plugin dependencies, but they must be compiled
first
+
### References
- [pf4j Development Mode
Documentation](https://pf4j.org/doc/development-mode.html)
- [JIRA TIKA-4587](https://issues.apache.org/jira/browse/TIKA-4587) -
Development mode implementation
-
diff --git a/tika-grpc/dev-tika-config.json b/tika-grpc/dev-tika-config.json
new file mode 100644
index 000000000..107d7cb0f
--- /dev/null
+++ b/tika-grpc/dev-tika-config.json
@@ -0,0 +1,34 @@
+{
+ "plugin-roots": [
+ "../tika-pipes/tika-pipes-plugins/tika-pipes-az-blob/target/classes",
+ "../tika-pipes/tika-pipes-plugins/tika-pipes-csv/target/classes",
+ "../tika-pipes/tika-pipes-plugins/tika-pipes-file-system/target/classes",
+ "../tika-pipes/tika-pipes-plugins/tika-pipes-gcs/target/classes",
+ "../tika-pipes/tika-pipes-plugins/tika-pipes-http/target/classes",
+ "../tika-pipes/tika-pipes-plugins/tika-pipes-jdbc/target/classes",
+ "../tika-pipes/tika-pipes-plugins/tika-pipes-json/target/classes",
+ "../tika-pipes/tika-pipes-plugins/tika-pipes-kafka/target/classes",
+
"../tika-pipes/tika-pipes-plugins/tika-pipes-microsoft-graph/target/classes",
+ "../tika-pipes/tika-pipes-plugins/tika-pipes-opensearch/target/classes",
+ "../tika-pipes/tika-pipes-plugins/tika-pipes-s3/target/classes",
+ "../tika-pipes/tika-pipes-plugins/tika-pipes-solr/target/classes"
+ ],
+ "fetchers": [
+ {
+ "fs": {
+ "defaultFetcher": {
+ "basePath": "/tmp/input"
+ }
+ }
+ }
+ ],
+ "emitters": [
+ {
+ "fs": {
+ "defaultEmitter": {
+ "basePath": "/tmp/output"
+ }
+ }
+ }
+ ]
+}
diff --git a/tika-grpc/pom.xml b/tika-grpc/pom.xml
index 5a5094a9f..d487c6acb 100644
--- a/tika-grpc/pom.xml
+++ b/tika-grpc/pom.xml
@@ -483,4 +483,96 @@
</plugin>
</plugins>
</build>
+
+ <profiles>
+ <!-- Development profile for running tika-grpc with plugin development
mode -->
+ <profile>
+ <id>dev</id>
+ <dependencies>
+ <!-- Include plugin dependencies so their classes are available at
runtime -->
+ <dependency>
+ <groupId>org.apache.tika</groupId>
+ <artifactId>tika-pipes-file-system</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.tika</groupId>
+ <artifactId>tika-pipes-csv</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.tika</groupId>
+ <artifactId>tika-pipes-json</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.tika</groupId>
+ <artifactId>tika-pipes-jdbc</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.tika</groupId>
+ <artifactId>tika-pipes-kafka</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.tika</groupId>
+ <artifactId>tika-pipes-gcs</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.tika</groupId>
+ <artifactId>tika-pipes-opensearch</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.tika</groupId>
+ <artifactId>tika-pipes-ignite</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.tika</groupId>
+ <artifactId>tika-pipes-solr</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.tika</groupId>
+ <artifactId>tika-pipes-az-blob</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.tika</groupId>
+ <artifactId>tika-pipes-microsoft-graph</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.tika</groupId>
+ <artifactId>tika-pipes-s3</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ </dependencies>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.codehaus.mojo</groupId>
+ <artifactId>exec-maven-plugin</artifactId>
+ <version>${maven.exec.version}</version>
+ <configuration>
+ <mainClass>org.apache.tika.pipes.grpc.TikaGrpcServer</mainClass>
+ <systemProperties>
+ <systemProperty>
+ <key>tika.plugin.dev.mode</key>
+ <value>true</value>
+ </systemProperty>
+ </systemProperties>
+ <arguments>
+ <argument>--config</argument>
+ <argument>${config.file}</argument>
+ </arguments>
+ </configuration>
+ </plugin>
+ </plugins>
+ </build>
+ </profile>
+ </profiles>
</project>
diff --git a/tika-grpc/run-dev.sh b/tika-grpc/run-dev.sh
new file mode 100755
index 000000000..604e70029
--- /dev/null
+++ b/tika-grpc/run-dev.sh
@@ -0,0 +1,20 @@
+#!/bin/bash
+# Run tika-grpc in development mode with plugin hot-reloading
+#
+# Usage:
+# ./run-dev.sh [config-file]
+#
+# If no config file is specified, defaults to dev-tika-config.json
+
+CONFIG_FILE="${1:-dev-tika-config.json}"
+
+echo "🚀 Starting Tika gRPC Server in Development Mode"
+echo "================================================"
+echo "Config file: $CONFIG_FILE"
+echo "Plugin dev mode: ENABLED"
+echo ""
+echo "Plugins will be loaded from target/classes directories"
+echo "Press Ctrl+C to stop the server"
+echo ""
+
+mvn exec:java -Pdev -Dconfig.file="$CONFIG_FILE"
diff --git
a/tika-plugins-core/src/main/java/org/apache/tika/plugins/TikaPluginManager.java
b/tika-plugins-core/src/main/java/org/apache/tika/plugins/TikaPluginManager.java
index 0cd635660..638ed5b59 100644
---
a/tika-plugins-core/src/main/java/org/apache/tika/plugins/TikaPluginManager.java
+++
b/tika-plugins-core/src/main/java/org/apache/tika/plugins/TikaPluginManager.java
@@ -118,6 +118,9 @@ public class TikaPluginManager extends DefaultPluginManager
{
public static TikaPluginManager load(TikaJsonConfig tikaJsonConfig)
throws TikaConfigException, IOException {
+ // Configure pf4j runtime mode before creating the manager
+ configurePf4jRuntimeMode();
+
JsonNode root = tikaJsonConfig.getRootNode();
JsonNode pluginRoots = root.get("plugin-roots");
if (pluginRoots == null) {
@@ -145,17 +148,43 @@ public class TikaPluginManager extends
DefaultPluginManager {
}
public TikaPluginManager(List<Path> pluginRoots) throws IOException {
- super(pluginRoots);
- configureRuntimeMode();
- init();
+ this(pluginRoots, true);
}
- private void configureRuntimeMode() {
- RuntimeMode mode = isDevelopmentMode() ? RuntimeMode.DEVELOPMENT :
RuntimeMode.DEPLOYMENT;
- this.runtimeMode = mode;
- if (mode == RuntimeMode.DEVELOPMENT) {
+ /**
+ * Internal constructor that allows skipping runtime mode configuration.
+ * Used by tests and factory methods that have already configured the mode.
+ */
+ private TikaPluginManager(List<Path> pluginRoots, boolean configureMode)
throws IOException {
+ super(configureMode ? configurePf4jRuntimeModeAndGetRoots(pluginRoots)
: pluginRoots);
+
+ if (getRuntimeMode() == RuntimeMode.DEVELOPMENT) {
LOG.info("TikaPluginManager running in DEVELOPMENT mode");
}
+
+ init();
+ }
+
+ /**
+ * Helper method to configure PF4J runtime mode and return the plugin
roots.
+ * This allows mode configuration before super() is called.
+ */
+ private static List<Path> configurePf4jRuntimeModeAndGetRoots(List<Path>
pluginRoots) {
+ configurePf4jRuntimeMode();
+ return pluginRoots;
+ }
+
+ /**
+ * Set pf4j's runtime mode system property based on Tika's dev mode
setting.
+ * This must be called before creating TikaPluginManager instance.
+ */
+ private static void configurePf4jRuntimeMode() {
+ if (isDevelopmentMode()) {
+ System.setProperty("pf4j.mode",
RuntimeMode.DEVELOPMENT.toString());
+ } else {
+ // Explicitly set to deployment mode to ensure clean state
+ System.setProperty("pf4j.mode", RuntimeMode.DEPLOYMENT.toString());
+ }
}
private static boolean isDevelopmentMode() {
@@ -186,6 +215,39 @@ public class TikaPluginManager extends
DefaultPluginManager {
// This will only discover extensions within the loaded plugin JARs.
return new DefaultExtensionFinder(this);
}
+
+ /**
+ * Override to prevent scanning subdirectories in development mode.
+ * In development mode, the default DevelopmentPluginRepository scans for
subdirectories,
+ * but we want each path in plugin-roots to be treated as a complete
plugin directory.
+ */
+ @Override
+ protected org.pf4j.PluginRepository createPluginRepository() {
+ if (getRuntimeMode() == RuntimeMode.DEVELOPMENT) {
+ // In development mode, return a repository that treats each path
as a plugin
+ return new org.pf4j.BasePluginRepository(getPluginsRoots()) {
+ @Override
+ public List<Path> getPluginPaths() {
+ // Don't scan subdirectories - each configured path IS a
plugin
+ return new java.util.ArrayList<>(pluginsRoots);
+ }
+ };
+ }
+ return super.createPluginRepository();
+ }
+
+ /**
+ * Override to use PropertiesPluginDescriptorFinder in development mode.
+ * In development mode, plugins are in target/classes with
plugin.properties,
+ * not packaged JARs with META-INF/MANIFEST.MF.
+ */
+ @Override
+ protected org.pf4j.PluginDescriptorFinder createPluginDescriptorFinder() {
+ if (getRuntimeMode() == RuntimeMode.DEVELOPMENT) {
+ return new org.pf4j.PropertiesPluginDescriptorFinder();
+ }
+ return super.createPluginDescriptorFinder();
+ }
private void init() throws IOException {
if (getRuntimeMode() == RuntimeMode.DEPLOYMENT) {
