This is an automated email from the ASF dual-hosted git repository. riemer pushed a commit to branch update-docs-transformation-module in repository https://gitbox.apache.org/repos/asf/streampipes.git
commit 6aa29fda4aad65be0679f797ca66a94da2b56977 Author: Dominik Riemer <[email protected]> AuthorDate: Tue May 20 09:30:58 2025 +0200 docs: Update documentation of transformation processors --- .../documentation.md | 155 ++++++++++++++++++--- .../documentation.md | 96 ++++++++++++- .../documentation.md | 155 ++++++++++++++++++++- .../documentation.md | 121 +++++++++++++--- .../documentation.md | 126 +++++++++++++++-- .../documentation.md | 91 +++++++++++- .../documentation.md | 68 ++++++++- .../documentation.md | 129 ++++++++++++----- .../documentation.md | 98 +++++++++---- .../documentation.md | 88 +++++++++++- .../documentation.md | 94 ++++++++++--- .../documentation.md | 62 ++++++++- .../documentation.md | 87 ++++++++++-- .../documentation.md | 102 +++++++++++++- .../documentation.md | 152 ++++++++++++++++++-- .../documentation.md | 131 +++++++++++++++-- .../documentation.md | 85 +++++++++-- .../documentation.md | 110 ++++++++++++--- .../documentation.md | 88 ++++++++++-- .../documentation.md | 123 ++++++++++++---- .../documentation.md | 75 +++++++++- .../documentation.md | 112 ++++++++++++++- .../documentation.md | 107 +++++++++++--- .../documentation.md | 108 ++++++++++++-- .../documentation.md | 107 +++++++++++--- .../documentation.md | 93 +++++++++++-- .../documentation.md | 110 ++++++++++++++- .../documentation.md | 71 +++++++++- 28 files changed, 2582 insertions(+), 362 deletions(-) diff --git a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.booloperator.counter/documentation.md b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.booloperator.counter/documentation.md index bd7b00b9a7..10dd2f5db0 100644 --- a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.booloperator.counter/documentation.md +++ b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.booloperator.counter/documentation.md @@ -26,35 +26,158 @@ ## Description -This processor monitors a boolean value and counts how often the value of the boolean changes. -A user can configure whether the changes from FALSE to TRUE, TRUE to FALSE, or BOTH changes should be counted. +The Boolean Counter processor counts state changes in a boolean field. It supports: +* Rising edge counting (FALSE -> TRUE) +* Falling edge counting (TRUE -> FALSE) +* Both edge counting +* Incremental counting +* State change detection +* Event-based counting + +This processor is essential for: +* Counting state transitions +* Tracking signal changes +* Monitoring event frequency +* Measuring cycle counts +* Analyzing state patterns +* Building event counters *** ## Required input -A boolean value is required in the data stream and can be selected with the field mapping. +The processor requires a data stream containing at least one boolean field to monitor for state changes. + +*** + +## Configuration ### Boolean Field -The boolean value to be monitored. +Select the boolean field to monitor for state changes. This field will be used to detect edges and increment the counter. -*** +### Flank Parameter -## Configuration +Choose which state changes to count: -A user can configure whether the changes from TRUE to FALSE, FALSE to TRUE, or all changes of the -boolean value should be counted. +* **FALSE -> TRUE**: Count rising edges only + * Triggers when signal changes from false to true + * Use for: Counting activations + * Example: Count device startups -### Flank parameter +* **TRUE -> FALSE**: Count falling edges only + * Triggers when signal changes from true to false + * Use for: Counting deactivations + * Example: Count device shutdowns -Either: -* TRUE -> FALSE: Increase counter on a true followed by a false -* FALSE -> TRUE: Increase counter on a false followed by a true -* BOTH: Increase counter on each change of the boolean value on two consecutive events +* **BOTH**: Count all state changes + * Triggers on any state change + * Use for: Counting all transitions + * Example: Count all state changes ## Output -Adds an additional numerical field with the current count value to the event. -Events are just emitted when the counter changes. -Runtime Name: countField \ No newline at end of file +The processor creates a new event containing: +* All original fields from the input event +* A new field named "counter" containing the current count + +### Example + +#### Input Event Stream +```json +[ + { + "deviceId": "machine01", + "isRunning": false, + "timestamp": 1586380104915 + }, + { + "deviceId": "machine01", + "isRunning": true, + "timestamp": 1586380105015 + }, + { + "deviceId": "machine01", + "isRunning": false, + "timestamp": 1586380105115 + } +] +``` + +#### Configuration 1: Rising Edge Counting +* Boolean Field: isRunning +* Flank Parameter: FALSE -> TRUE + +#### Output Event +```json +{ + "deviceId": "machine01", + "isRunning": true, + "timestamp": 1586380105015, + "counter": 1 +} +``` + +#### Configuration 2: Both Edges Counting +* Boolean Field: isRunning +* Flank Parameter: BOTH + +#### Output Events +```json +{ + "deviceId": "machine01", + "isRunning": true, + "timestamp": 1586380105015, + "counter": 1 +} +``` +```json +{ + "deviceId": "machine01", + "isRunning": false, + "timestamp": 1586380105115, + "counter": 2 +} +``` + +## Use Cases + +1. **Equipment Monitoring** + * Count machine cycles + * Track state changes + * Monitor activations + * Count operations + * Track usage patterns + +2. **Process Control** + * Count process steps + * Track state transitions + * Monitor phase changes + * Count operations + * Track cycles + +3. **Event Detection** + * Count button presses + * Track switch changes + * Monitor transitions + * Count events + * Track occurrences + +4. **System Analysis** + * Count state changes + * Track transitions + * Monitor patterns + * Count events + * Track frequencies + +## Notes + +* Only boolean fields can be counted +* Counter is incremental +* Count starts at 0 +* Processing is stateful +* Counter persists between events +* Events are only emitted on state changes +* Counter is reset on pipeline restart +* Multiple counters require chaining +* Edge detection is immediate \ No newline at end of file diff --git a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.booloperator.inverter/documentation.md b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.booloperator.inverter/documentation.md index 496e981a0e..9316b1bf97 100644 --- a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.booloperator.inverter/documentation.md +++ b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.booloperator.inverter/documentation.md @@ -26,20 +26,104 @@ ## Description -This processor requires a boolean value in the data stream and inverts its value. (e.g. true -> flase) +The Boolean Inverter processor inverts the value of a boolean field in a data stream. It supports: +* Single field inversion +* TRUE to FALSE conversion +* FALSE to TRUE conversion +* In-place value modification +* Simple boolean logic +* Direct value negation + +This processor is essential for: +* Negating boolean conditions +* Inverting control signals +* Complementing state values +* Reversing logic gates +* Creating opposite states +* Implementing NOT operations *** ## Required input -### Boolean Field - -The boolean value to be inverted. +The processor requires a data stream containing at least one boolean field to invert. *** ## Configuration -No further configuration required + +### Invert Field + +Select the boolean field to invert. This field's value will be negated (TRUE becomes FALSE, FALSE becomes TRUE). ## Output -The output schema is the same as the input schema. Just the value of the property is changed. \ No newline at end of file + +The processor creates a new event containing: +* All original fields from the input event +* The selected field with its value inverted + +### Example + +#### Input Event +```json +{ + "deviceId": "sensor01", + "isActive": true, + "timestamp": 1586380104915 +} +``` + +#### Configuration +* Invert Field: isActive + +#### Output Event +```json +{ + "deviceId": "sensor01", + "isActive": false, + "timestamp": 1586380104915 +} +``` + +## Use Cases + +1. **Control Systems** + * Invert control signals + * Negate status flags + * Reverse logic gates + * Complement states + * Create opposite conditions + +2. **State Management** + * Invert state values + * Negate status indicators + * Reverse boolean flags + * Complement conditions + * Create inverse states + +3. **Logic Operations** + * Implement NOT gates + * Negate conditions + * Reverse boolean logic + * Complement expressions + * Create opposite states + +4. **Signal Processing** + * Invert digital signals + * Negate binary values + * Reverse boolean states + * Complement conditions + * Create inverse signals + +## Notes + +* Only boolean fields can be inverted +* Inversion is in-place +* Original value is replaced +* Processing is stateless +* Multiple inversions require chaining +* Consider logic implications +* Inversion is immediate +* No delay in processing +* No additional fields created +* Simple boolean negation \ No newline at end of file diff --git a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.booloperator.logical/documentation.md b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.booloperator.logical/documentation.md index 039865443c..5a74bc263e 100644 --- a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.booloperator.logical/documentation.md +++ b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.booloperator.logical/documentation.md @@ -16,21 +16,166 @@ ~ --> -## Boolean Operator +## Boolean Logical Operator + +<p align="center"> + <img src="icon.png" width="150px;" class="pe-image-documentation"/> +</p> + +*** ## Description -This processor performs the logical boolean operation b/w the vales of set of properties. -A user can select the type of operation and set of properties. +The Boolean Logical Operator processor performs logical operations on multiple boolean fields. It supports: +* Standard logical operations (AND, OR, NOT, XOR, XNOR, NOR) +* Multiple input fields +* Single output field +* Truth table based operations +* Complex boolean logic +* State combination + +This processor is essential for: +* Combining multiple conditions +* Implementing decision logic +* Creating composite states +* Building safety interlocks +* Monitoring complex conditions +* Implementing control logic *** ## Required input -Type of logical boolean operator and set of properties on which operator needs to perform. +The processor requires a data stream containing at least one boolean field for NOT operations, or at least two boolean fields for all other operations. *** +## Configuration + +### Properties + +Select one or more boolean fields to use as operands for the logical operation. The number of fields required depends on the selected operator: +* NOT: Exactly one field +* All other operators: Two or more fields + +### Boolean Operator + +Choose the logical operation to perform: + +* **AND**: True only if all inputs are true + * Use for: Requiring all conditions + * Example: All safety switches must be on + +* **OR**: True if any input is true + * Use for: Accepting any condition + * Example: Any warning condition is active + +* **NOT**: Inverts the input value + * Use for: Negating conditions + * Example: Invert a status flag + +* **XOR**: True if odd number of inputs are true + * Use for: Detecting mismatches + * Example: Check if states are different + +* **XNOR**: True if even number of inputs are true + * Use for: Detecting matches + * Example: Verify state agreement + +* **NOR**: True only if all inputs are false + * Use for: Detecting inactive states + * Example: Check if all systems are off + ## Output -Outputs the incoming event while appending the result (``boolean-operations-result``) to the incoming event. +The processor creates a new event containing: +* All original fields from the input event +* A new boolean field named "boolean-operations-result" with the result of the logical operation + +### Example + +#### Input Event +```json +{ + "deviceId": "machine01", + "safetySwitch1": true, + "safetySwitch2": true, + "emergencyStop": false, + "timestamp": 1586380104915 +} +``` + +#### Configuration 1: AND Operation +* Properties: safetySwitch1, safetySwitch2 +* Operator: AND + +#### Output Event +```json +{ + "deviceId": "machine01", + "safetySwitch1": true, + "safetySwitch2": true, + "emergencyStop": false, + "timestamp": 1586380104915, + "boolean-operations-result": true +} +``` + +#### Configuration 2: NOT Operation +* Properties: emergencyStop +* Operator: NOT + +#### Output Event +```json +{ + "deviceId": "machine01", + "safetySwitch1": true, + "safetySwitch2": true, + "emergencyStop": false, + "timestamp": 1586380104915, + "boolean-operations-result": true +} +``` + +## Use Cases + +1. **Safety Systems** + * Combine multiple safety interlocks + * Monitor emergency conditions + * Implement fail-safe logic + * Validate safety states + * Create composite safety checks + +2. **Process Control** + * Combine process conditions + * Implement control logic + * Monitor state combinations + * Create decision points + * Build process interlocks + +3. **Quality Monitoring** + * Combine quality checks + * Implement acceptance criteria + * Monitor multiple conditions + * Create composite quality states + * Build validation rules + +4. **System Monitoring** + * Combine status indicators + * Monitor system states + * Implement health checks + * Create alert conditions + * Build diagnostic rules + +## Notes + +* All input fields must be boolean +* NOT operator requires exactly one input field +* All other operators require at least two input fields +* Operations are evaluated left-to-right +* Result is always boolean +* Null inputs are treated as false +* The output field is always named "boolean-operations-result" +* Processing is stateless +* Multiple operations require chaining processors +* Consider operation precedence in complex logic diff --git a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.booloperator.timekeeping/documentation.md b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.booloperator.timekeeping/documentation.md index a4adacf1b3..e5ddc1f82b 100644 --- a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.booloperator.timekeeping/documentation.md +++ b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.booloperator.timekeeping/documentation.md @@ -26,39 +26,116 @@ ## Description -This processor can be used to measure the time between two boolean sensors. -For example on a conveyor, where one sensor is placed on the left and one senor placed on the right. -Parts are transported on the conveyor and the sensors are boolean sensors detecting those parts. -The time is measured between the two sensors as well as the amount of complete transportation's is counted. -The measurement is initialized once the left sensor is true and stopped once the right sensor is true. -There can also be multiple parts on the conveyor as long as the individual parts do not change. +The Measure Time Between Two Sensors processor calculates the time difference between two boolean signals. It supports: +* Signal timing measurement +* Event sequence tracking +* Time difference calculation +* Signal counting +* Multiple time units (milliseconds, seconds, minutes) + +This processor is essential for: +* Measuring process times +* Tracking signal sequences +* Analyzing delays +* Monitoring performance +*** -<p align="center"> - <img src="time_measure_example.png" width="300px;" class="pe-image-documentation"/> -</p> +## Required input + +The processor requires a data stream containing: +* A left boolean field (start signal) +* A right boolean field (end signal) *** -## Required input -Requires two boolean fields in the datastream. +## Configuration ### Left Field -The left field starts the timer when value is true. + +Select the boolean field that starts the timer. This signal marks the beginning of the time measurement. ### Right Field -The right field stops the timer and emits the event when its value is true. -*** +Select the boolean field that ends the timer. This signal triggers the output event with the measured time. -## Configuration -No furhter configuration is required. +### Output Unit -## Output -Appends two fields to the input event. +Choose the time unit for the output: +* **Milliseconds**: Raw time difference +* **Seconds**: Time difference divided by 1000 +* **Minutes**: Time difference divided by 60000 -### Timer Field -The timer field is a numeric value representing the time between the two sensors. Runtime name: measured_time +## Output -### Counter -The counter indicated how many events where emitted by this component. Runtime name: counter \ No newline at end of file +The processor creates a new event containing: +* All original fields from the input event +* A "measured_time" field showing the time between signals in the selected unit +* A "counter" field showing the number of completed measurements + +### Example + +#### Input Event Stream +```json +[ + { + "deviceId": "machine01", + "timestamp": 1586380104915, + "startSignal": false, + "endSignal": false + }, + { + "deviceId": "machine01", + "timestamp": 1586380105015, + "startSignal": true, + "endSignal": false + }, + { + "deviceId": "machine01", + "timestamp": 1586380105115, + "startSignal": true, + "endSignal": true + } +] +``` + +#### Configuration +* Left Field: startSignal +* Right Field: endSignal +* Output Unit: Seconds + +#### Output Event +```json +{ + "deviceId": "machine01", + "timestamp": 1586380105115, + "startSignal": true, + "endSignal": true, + "measured_time": 1.0, + "counter": 1 +} +``` + +## Use Cases + +1. **Process Timing** + * Measure process cycle times + * Track signal sequences + * Analyze delays + * Monitor performance + +2. **Performance Analysis** + * Measure response times + * Track operation sequences + * Analyze system delays + * Monitor equipment performance + +## Notes + +* Only boolean fields can be monitored +* Processing is stateful +* Time measurement is based on system time +* Counter resets at Long.MAX_VALUE +* Multiple start signals are buffered +* Output is triggered by end signal +* Original event fields are preserved \ No newline at end of file diff --git a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.booloperator.timer/documentation.md b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.booloperator.timer/documentation.md index 30ee61eb0a..9e81772c0f 100644 --- a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.booloperator.timer/documentation.md +++ b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.booloperator.timer/documentation.md @@ -26,27 +26,131 @@ ## Description -This processor measures how long a boolean value does not change. -Once the value is changes the event with the measured time is emitted. +The Boolean Timer processor measures the duration that a boolean field maintains a specific state. It supports: +* Measuring TRUE state duration +* Measuring FALSE state duration +* Multiple time units (milliseconds, seconds, minutes) +* State change detection +* Duration tracking +This processor is essential for: +* Measuring state durations +* Monitoring active/inactive times +* Tracking operation periods +* Calculating cycle times *** ## Required input -A boolean value is required in the data stream. - -### Field - -The boolean field which is monitored for state changes. +The processor requires a data stream containing at least one boolean field to measure its state duration. *** ## Configuration -### Timer value -Define whether it should be measured how long the value is true or how long the value is false. +### Boolean Field + +Select the boolean field to monitor for state duration. This field will be used to measure how long it maintains a specific state. + +### Value to Observe + +Choose which state to measure the duration for: + +* **TRUE**: Measure how long the field stays true + * Use for: Measuring active periods + * Example: Machine runtime + +* **FALSE**: Measure how long the field stays false + * Use for: Measuring inactive periods + * Example: Downtime duration + +### Output Unit + +Select the time unit for the measured duration: + +* **Milliseconds**: Raw millisecond values + * Use for: Precise timing + * Example: Response time measurement + +* **Seconds**: Duration in seconds (milliseconds / 1000) + * Use for: General timing + * Example: Process duration + +* **Minutes**: Duration in minutes (milliseconds / 60000) + * Use for: Long periods + * Example: Operation time ## Output -Appends a field with the time how long the value did not change. -Is emitted on the change of the boolean value. Runtime name: measured_time + +The processor creates a new event containing: +* All original fields from the input event +* A new field named "measured_time" containing the duration in the selected unit + +### Example + +#### Input Event Stream +```json +[ + { + "deviceId": "machine01", + "isRunning": false, + "timestamp": 1586380104915 + }, + { + "deviceId": "machine01", + "isRunning": true, + "timestamp": 1586380105915 + }, + { + "deviceId": "machine01", + "isRunning": true, + "timestamp": 1586380106915 + }, + { + "deviceId": "machine01", + "isRunning": false, + "timestamp": 1586380107915 + } +] +``` + +#### Configuration 1: TRUE State in Seconds +* Boolean Field: isRunning +* Value to Observe: TRUE +* Output Unit: Seconds + +#### Output Event +```json +{ + "deviceId": "machine01", + "isRunning": false, + "timestamp": 1586380107915, + "measured_time": 2.0 +} +``` + +## Use Cases + +1. **Equipment Monitoring** + * Measure machine runtime + * Track downtime periods + * Monitor active states + * Calculate cycle times + +2. **Process Control** + * Measure process duration + * Track state changes + * Monitor operation times + * Calculate response times + +## Notes + +* Only boolean fields can be measured +* Processing is stateful +* Time measurement is based on system time +* Measurement starts when selected state is detected +* Measurement ends when opposite state is detected +* Output is only generated when measurement ends +* Time units are automatically converted +* Original event fields are preserved diff --git a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.changed-value/documentation.md b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.changed-value/documentation.md index 5a4b55ce34..b76ae34f5f 100644 --- a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.changed-value/documentation.md +++ b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.changed-value/documentation.md @@ -26,15 +26,96 @@ ## Description -This processor sends out an event everytime a specific object changes. -It also adds a timestamp in ms from the system time. +The Value Changed processor detects changes in field values and outputs events only when changes occur. It supports: +* Any data type comparison +* Dimension-based state tracking +* Change timestamp tracking +* State size management +* Multi-dimensional monitoring + +This processor is essential for: +* Detecting value changes +* Tracking state transitions +* Monitoring field updates +* Building change logs +* Implementing change triggers + +*** + +## Required input + +The processor requires a data stream containing at least one field to monitor for changes. *** ## Configuration -Select property to monitor for changes -Describe the configuration parameters here +### Keep Fields + +Select the field to monitor for changes. The processor will output an event only when the selected field's value changes. ## Output -Emit an event on change and append a timestamp when the change occured \ No newline at end of file + +The processor creates a new event containing: +* All original fields from the input event +* A new field named "change_detected" containing the timestamp when the change occurred +* Only when the monitored field changes value + +### Example + +#### Input Event Stream +```json +{ + "deviceId": "sensor01", + "location": "l1", + "value": 12, + "timestamp": 1586380104915 +} +``` +```json +{ + "deviceId": "sensor01", + "location": "l1", + "value": 15, + "timestamp": 1586380105015 +} +``` + +#### Configuration +* Keep Fields: value + +#### Output Event +```json +{ + "deviceId": "sensor01", + "location": "l1", + "value": 15, + "timestamp": 1586380105015, + "change_detected": 1586380105015 +} +``` + +## Use Cases + +1. **Change Detection** + * Monitor value changes + * Track state transitions + * Detect field updates + * Build change logs + +2. **State Monitoring** + * Track state changes + * Monitor transitions + * Detect updates + * Log changes + +## Notes + +* Supports any data type +* Processing is stateful +* State size limited to 5000 entries +* Events only emitted on value changes +* Original fields are preserved +* Change timestamps are added +* Dimension-based state tracking +* Multi-dimensional monitoring supported \ No newline at end of file diff --git a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.count-array/documentation.md b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.count-array/documentation.md index 1e006e60ac..2671afb072 100644 --- a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.count-array/documentation.md +++ b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.count-array/documentation.md @@ -26,24 +26,80 @@ ## Description -This processor takes a list field, computes the size of the list and appends the result to the event. +The Count Array processor counts the number of elements in array/list fields. It supports: +* Array size counting +* List length measurement +* Collection size tracking +* Dynamic array handling + +This processor is essential for: +* Measuring array sizes +* Tracking list lengths +* Monitoring collection growth +* Analyzing data volumes *** ## Required input -This processor works with any event that has a field of type ``list``. +The processor requires a data stream containing at least one array or list field to count its elements. *** ## Configuration -Describe the configuration parameters here - ### List Field -The field containing the list that should be used. +Select the array or list field to count its elements. This field will be used to determine the number of items it contains. ## Output -Outputs the incoming event while appending the list size (named ``countValue``) to the incoming event. \ No newline at end of file +The processor creates a new event containing: +* All original fields from the input event +* A new field named "countValue" containing the number of elements in the selected array/list + +### Example + +#### Input Event +```json +{ + "deviceId": "sensor01", + "measurements": [23.5, 24.1, 25.3, 24.8], + "timestamp": 1586380104915 +} +``` + +#### Configuration +* List Field: measurements + +#### Output Event +```json +{ + "deviceId": "sensor01", + "measurements": [23.5, 24.1, 25.3, 24.8], + "timestamp": 1586380104915, + "countValue": 4 +} +``` + +## Use Cases + +1. **Data Analysis** + * Count array elements + * Track list sizes + * Monitor collection growth + * Measure data volumes + +2. **Resource Management** + * Monitor buffer sizes + * Track queue lengths + * Measure storage usage + * Analyze capacity needs + +## Notes + +* Only array/list fields can be counted +* Count is calculated per event +* Processing is stateless +* Empty arrays return 0 +* Null arrays are not supported \ No newline at end of file diff --git a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.csvmetadata/documentation.md b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.csvmetadata/documentation.md index 9080e2d091..a624d088ec 100644 --- a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.csvmetadata/documentation.md +++ b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.csvmetadata/documentation.md @@ -16,56 +16,121 @@ ~ --> -## CSV Metadata Enrichment -Enrich a datastream with information provided in a CSV file. -The data of the CSV file is matched by an id column with a property value of a String in the data stream. +## CSV Metadata Enricher + +<p align="center"> + <img src="icon.png" width="150px;" class="pe-image-documentation"/> +</p> *** ## Description -Upload a CSV file with static meta information that will be appended to each event. -The file can contain different information for different keys in the stream. +The CSV Metadata Enricher processor adds additional fields to events by looking up values in a CSV file. It supports: +* CSV file integration +* Field mapping +* ID-based lookups +* Multiple field enrichment +* Static metadata + +This processor is essential for: +* Adding static metadata +* Enriching event data +* Mapping reference data +* Creating data relationships + +*** -### Structure of CSV file -The first row containes the runtime names for the properties to insert. -Once the file is uploaded the user can select which column to use for the matching property -and which values should be appended. -Delimiter: ';' +## Required input +The processor requires: +* A data stream containing an ID field for lookups +* A CSV file containing the metadata to be added +* A field in the CSV file that matches the ID field *** -## Example -Add the location of a production line to the event +## Configuration -### Input event -``` +### ID Field + +Select the field from the input event that will be used to look up matching records in the CSV file. + +### CSV File + +Upload the CSV file containing the metadata to be added to the events. The file should contain: +* A column that matches the ID field +* Additional columns with the metadata to be added + +### Field to Match + +Select the column name in the CSV file that corresponds to the ID field. This field will be used to find matching records. + +### Fields to Append + +Select one or more columns from the CSV file that should be added to the events. These fields will be appended to the output events. + +## Output + +The processor creates a new event containing: +* All original fields from the input event +* The selected fields from the CSV file for matching records + +### Example + +#### Input Event +```json { - 'line_id': 'line1', - 'timestamp': 1586378041 + "deviceId": "sensor01", + "temperature": 23.5, + "timestamp": 1586380104915 } ``` -### CSV File -``` -production_line;location -line1;germany -line2;uk -line3;usa +#### CSV File Content +```csv +device_id,location,manufacturer,model +sensor01,Building A,Acme Corp,TempPro 2000 +sensor02,Building B,Acme Corp,TempPro 2000 ``` -### Configuration -* The field that is used for the lookup (Example: line_id) -* The CSV file (Example: Upload the csv file) -* Field to match (Example: production_line) -* Fields to append (Example: location) +#### Configuration +* ID Field: deviceId +* Field to Match: device_id +* Fields to Append: location, manufacturer, model -### Output event -``` +#### Output Event +```json { - 'line_id': 'line1', - 'timestamp': 1586378041, - 'location': 'germany' + "deviceId": "sensor01", + "temperature": 23.5, + "timestamp": 1586380104915, + "location": "Building A", + "manufacturer": "Acme Corp", + "model": "TempPro 2000" } ``` + +## Use Cases + +1. **Device Management** + * Add device metadata + * Map locations + * Track equipment info + * Monitor assets + +2. **Data Enrichment** + * Add reference data + * Map relationships + * Create context + * Build hierarchies + +## Notes + +* CSV file must be uploaded +* ID field must exist in both event and CSV +* Field names are case-sensitive +* CSV must have a header row +* No matches return empty values +* Processing is stateless +* CSV is loaded at startup diff --git a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.datetime/documentation.md b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.datetime/documentation.md index 3dfcb6657f..b4b4804ed0 100644 --- a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.datetime/documentation.md +++ b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.datetime/documentation.md @@ -24,49 +24,89 @@ *** -## Overview +## Description -The "Datetime From String" processor is a handy tool that helps convert human-readable datetime information into a -format that machines can understand. This is particularly useful when dealing with data that includes dates and times. +The Datetime From String processor converts string timestamps into millisecond timestamps. It supports: +* ISO 8601 format parsing +* Timezone handling +* String to millisecond conversion +* Automatic timezone application -### Why Use This Processor? - -In the context of event streams, you may encounter dates and times formatted for human readability but not necessarily -optimized for computer processing. The "Datetime From String" processor addresses this by facilitating the conversion -of human-readable datetime information within your continuous stream of events. +This processor is essential for: +* Converting timestamps to milliseconds +* Standardizing date formats +* Handling timezone conversions +* Processing ISO 8601 dates *** -## How It Works - -When you input a data stream into this processor containing a datetime in a specific format (such as "2023-11-24 15:30: -00"), it -undergoes a transformation. The processor converts it into a computer-friendly format called a ZonedDateTime object. - -### Example +## Required input -Let's say you have an event stream with a property containing values like "2023-11-24 15:30:00" and you want to make -sure your computer understands it. You can use -this processor to convert it into a format that's machine-friendly. +The processor requires a data stream containing at least one string field with a timestamp in ISO 8601 format. *** -## Getting Started +## Configuration -To use this processor, you need one thing in your data: +### DateTime String -1. **Datetime String**: This is the name of the event property that contains the human-readable datetime string, like "2023-11-24 15:30:00". +Select the field containing the timestamp string. The string should be in ISO 8601 format (e.g., '2023-11-29T18:30:22' or '2023-11-29T18:30:22+01:00'). +### Time Zone -### Configuration - -The only thing you need to configure is the time zone. -1. **Time Zone**: Specify the time zone that applies to your datetime if it doesn't already have this information.This ensures that the processor understands the context of your -datetime. +Select the timezone for the input timestamp. This is used when the timestamp string doesn't include timezone information. If the input string already contains timezone information, this setting is ignored. ## Output -After the conversion happens, the processor adds a new piece of information to your data stream: +The processor creates a new event containing: +* All original fields from the input event +* A new field named "timestringInMillis" containing the timestamp in milliseconds since epoch +* A new field named "timeZone" containing the selected timezone + +### Example -* **timestringInMillis**: This is the transformed datetime in a format that computers can easily work with (UNIX timestamp in milliseconds). -* **timeZone**: The name of the timezone the `dateTime` value refers to. Can be used to reconstitute the actual time. \ No newline at end of file +#### Input Event +```json +{ + "deviceId": "sensor01", + "timestamp": "2023-11-29T18:30:22", + "value": 23.5 +} +``` + +#### Configuration +* DateTime String: timestamp +* Time Zone: UTC + +#### Output Event +```json +{ + "deviceId": "sensor01", + "timestamp": "2023-11-29T18:30:22", + "value": 23.5, + "timestringInMillis": 1701279022000, + "timeZone": "UTC" +} +``` + +## Use Cases + +1. **Data Standardization** + * Convert timestamps to milliseconds + * Standardize date formats + * Handle timezone conversions + * Process ISO 8601 dates + +2. **System Integration** + * Map timestamps to milliseconds + * Convert between timezones + * Standardize date formats + * Process time-based data + +## Notes + +* Input must be in ISO 8601 format +* Timezone in input string takes precedence over selected timezone +* Invalid formats will cause processing errors +* Processing is stateless +* Output is always in milliseconds since epoch \ No newline at end of file diff --git a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.duration-value/documentation.md b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.duration-value/documentation.md index 6a214ad016..8c40746a75 100644 --- a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.duration-value/documentation.md +++ b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.duration-value/documentation.md @@ -26,20 +26,96 @@ ## Description -This processor calculates the duration for a given stream with a start timestamp and an end timestamp. +The Calculate Duration processor computes the time difference between two timestamps. It supports: +* Time difference calculation +* Multiple time units +* Start/end timestamp selection +* Duration measurement + +This processor is essential for: +* Measuring time periods +* Calculating durations +* Analyzing intervals +* Tracking time spans *** ## Required input -Two timestamp fields + +The processor requires a data stream containing at least two timestamp fields to calculate the duration between them. *** ## Configuration -* Start Timestamp: The first timestamp (t1) -* End Timestamp: The second timestamp (t2) -* Time Unit of the result +### Start Timestamp + +Select the field containing the start timestamp. This timestamp marks the beginning of the duration period. + +### End Timestamp + +Select the field containing the end timestamp. This timestamp marks the end of the duration period. + +### Time Unit + +Select the unit for the calculated duration: +* Milliseconds (default) +* Seconds +* Minutes ## Output -Appends a new field with the difference of t2 and t1 \ No newline at end of file + +The processor creates a new event containing: +* All original fields from the input event +* A new field named "duration" containing the calculated time difference in the selected unit + +### Example + +#### Input Event +```json +{ + "deviceId": "machine01", + "startTime": 1586380104915, + "endTime": 1586380105915, + "operation": "process1" +} +``` + +#### Configuration +* Start Timestamp: startTime +* End Timestamp: endTime +* Time Unit: Seconds + +#### Output Event +```json +{ + "deviceId": "machine01", + "startTime": 1586380104915, + "endTime": 1586380105915, + "operation": "process1", + "duration": 1.0 +} +``` + +## Use Cases + +1. **Process Monitoring** + * Measure process duration + * Track operation times + * Monitor cycle times + * Calculate periods + +2. **Performance Analysis** + * Measure response times + * Track execution times + * Monitor durations + * Calculate intervals + +## Notes + +* Both timestamps must be present +* Timestamps must be valid +* End time must be after start time +* Processing is stateless +* Multiple durations require chaining +* Negative durations are not supported \ No newline at end of file diff --git a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.field-mapper/documentation.md b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.field-mapper/documentation.md index 2e86d36776..836c78ccbb 100644 --- a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.field-mapper/documentation.md +++ b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.field-mapper/documentation.md @@ -26,43 +26,93 @@ ## Description -Replaces one or more fields with a new field and computes a hash value of these fields +The Field Mapper processor combines multiple fields into a single field by computing an MD5 hash value of their combined contents. This processor is particularly useful for: + +* Creating unique identifiers from multiple fields +* Data anonymization and privacy protection +* Reducing data dimensionality +* Generating consistent keys for data grouping +* Combining related fields into a single reference *** -## Configuration +## Required input -* Fields: Fields that will be mapped into a property -* Name of the new field +The processor can work with any event that contains at least one field. The fields to be mapped can be of any data type, as they will be converted to their string representation before hashing. *** -## Example +## Configuration -Merge two fields into a hash value +### Fields -### Input event +Select one or more fields that should be combined into a single hash value. The order of field selection matters, as it affects the resulting hash value. -``` +### New Field Name + +Specify the name of the new field that will contain the MD5 hash value of the combined fields. + +## Output + +The processor creates a new event that: +* Retains all fields from the input event that were not selected for mapping +* Adds a new field with the specified name containing the MD5 hash of the combined selected fields +* Removes the original fields that were mapped + +### Example + +#### Input Event +```json { - "timestamp":1586380104915, - "mass_flow":4.3167, - "temperature":40.05, - "sensorId":"flowrate01" + "timestamp": 1586380104915, + "mass_flow": 4.3167, + "temperature": 40.05, + "pressure": 1013.25, + "sensorId": "flowrate01" } ``` -### Configuration - +#### Configuration * Fields: mass_flow, temperature -* Name of new field: demo +* New Field Name: combined_measurement -### Output event - -``` +#### Output Event +```json { - "timestamp":1586380104915, - "sensorId":"flowrate01" - "demo":"8ae11f5c83610104408d485b73120832", + "timestamp": 1586380104915, + "pressure": 1013.25, + "sensorId": "flowrate01", + "combined_measurement": "8ae11f5c83610104408d485b73120832" } -``` \ No newline at end of file +``` + +## Use Cases + +1. **Data Privacy** + * Combine personally identifiable information (PII) into anonymized identifiers + * Create privacy-preserving keys for data linkage + * Generate pseudonyms for sensitive data + +2. **Data Integration** + * Create composite keys for data joining + * Generate unique identifiers across systems + * Standardize multi-field references + +3. **Data Quality** + * Track changes across multiple fields + * Create checksums for data validation + * Monitor data integrity + +4. **Performance Optimization** + * Reduce storage requirements by combining fields + * Optimize indexing with combined keys + * Improve query performance with single-field lookups + +## Notes + +* The hash value is deterministic - the same input fields will always produce the same hash +* The hash is irreversible - you cannot recover the original field values from the hash +* Field order matters - changing the order of fields will produce a different hash +* All field values are converted to strings before hashing +* The output hash is always a 32-character hexadecimal string +* Null or empty field values are handled gracefully but will affect the resulting hash \ No newline at end of file diff --git a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.fieldhasher/documentation.md b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.fieldhasher/documentation.md index 30d72fa7c2..3fbbb8912c 100644 --- a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.fieldhasher/documentation.md +++ b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.fieldhasher/documentation.md @@ -26,24 +26,72 @@ ## Description -The Field Hasher uses an algorithm to encode values in a field. -The Field Hasher can use MD5, SHA1 or SHA2 to hash field values. +The Field Hasher processor is a data transformation component that applies cryptographic hash functions to string values in a data stream. It can be used to encode sensitive information, generate unique identifiers, or transform data for privacy and security purposes. + +The processor supports three widely-used hash algorithms: +* **MD5** - A 128-bit hash function +* **SHA1** - A 160-bit hash function +* **SHA2** - A 256-bit hash function (SHA-256 implementation) *** ## Required input -This processor requires at least one field of type string. + +This processor requires an event stream that contains at least one field of type string. The string field will be used as input for the hash function. *** ## Configuration ### Field -Specifies the string field that will be encoded. +Specifies the string field that will be encoded. This field must exist in the input event stream and must contain string values. ### Hash Algorithm -Specifies the algorithm used to encode the string field. The following algorithms -are available: SHA2, MD5 or SHA1. +Select the hash algorithm to use for encoding the string field. Available options are: +* **SHA1** - Produces a 40-character hexadecimal hash +* **SHA2** - Produces a 64-character hexadecimal hash +* **MD5** - Produces a 32-character hexadecimal hash ## Output -The encoded string field. \ No newline at end of file + +The processor modifies the input event by replacing the value of the selected field with its hashed version. All other fields in the event remain unchanged. + +### Example + +#### Input Event +```json +{ + "timestamp": 1617183834000, + "sensorId": "sensor123", + "value": 42.5, + "user": "[email protected]" +} +``` + +#### Configuration +* Field: user +* Hash Algorithm: MD5 + +#### Output Event +```json +{ + "timestamp": 1617183834000, + "sensorId": "sensor123", + "value": 42.5, + "user": "e87f955d3b3499b8b13e901fd61b6b64" +} +``` + +## Use Cases + +1. **Privacy Protection**: Hash personally identifiable information (PII) before storage or transmission +2. **Data Anonymization**: Create anonymous identifiers from user data +3. **Checksum Generation**: Generate checksums for data validation +4. **Unique ID Creation**: Create unique identifiers from combinations of fields + +## Notes + +* The hash functions are one-way transformations - the original value cannot be recovered from the hash +* The same input will always produce the same hash value +* Different hash algorithms provide different levels of collision resistance and output lengths +* For security-critical applications, consider using SHA2 as it provides stronger cryptographic properties \ No newline at end of file diff --git a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.fieldrename/documentation.md b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.fieldrename/documentation.md index 059b08ae8f..421859b1f5 100644 --- a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.fieldrename/documentation.md +++ b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.fieldrename/documentation.md @@ -18,36 +18,95 @@ ## Field Renamer +<p align="center"> + <img src="icon.png" width="150px;" class="pe-image-documentation"/> +</p> *** ## Description -Replaces the runtime name of an event property with a custom defined name. -Useful for data ingestion purposes where a specific event schema is required. +The Field Renamer processor changes the name of fields in events. It supports: +* Single field renaming +* Custom field names +* Runtime name modification +* Field name standardization +This processor is essential for: +* Standardizing field names +* Creating consistent naming +* Mapping field identifiers +* Improving readability *** -### OldFieldName -Specifies the field to rename. +## Required input -### NewFieldName -Specifies the new runtime name of the field. +The processor requires a data stream containing at least one field to rename. + +*** + +## Configuration + +### Field + +Select the field to rename. This field's name will be changed in the output event. + +### New Field Name + +Enter the new name for the selected field. This name will replace the original field name in the output event. ## Output -Example: -Old Output: -``` +The processor creates a new event containing: +* All original fields from the input event +* The selected field with its new name +* All field values remain unchanged + +### Example + +#### Input Event +```json { - 'timestamp': 16003000, + "deviceId": "sensor01", + "temp": 23.5, + "humidity": 45.2, + "timestamp": 1586380104915 } ``` -New Ouput: -``` +#### Configuration +* Field: temp +* New Field Name: temperature + +#### Output Event +```json { - 'time': 16003000, + "deviceId": "sensor01", + "temperature": 23.5, + "humidity": 45.2, + "timestamp": 1586380104915 } -``` \ No newline at end of file +``` + +## Use Cases + +1. **Data Standardization** + * Standardize field names + * Create naming conventions + * Map field identifiers + * Improve consistency + +2. **System Integration** + * Map field names + * Standardize interfaces + * Create consistency + * Build mappings + +## Notes + +* Only field names are changed +* Field values remain unchanged +* Original field order is preserved +* Processing is stateless +* Multiple renames require chaining \ No newline at end of file diff --git a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.measurementunitconverter/documentation.md b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.measurementunitconverter/documentation.md index 9d0b81b28c..73ad466d96 100644 --- a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.measurementunitconverter/documentation.md +++ b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.measurementunitconverter/documentation.md @@ -26,22 +26,114 @@ ## Description -Converts a unit of measurement to another one. +The Measurement Unit Converter processor automatically converts values between different units of measurement. It supports any unit type that is compatible with the input field's measurement unit, including: + +* Length (meters, feet, inches, etc.) +* Mass (kilograms, pounds, ounces, etc.) +* Temperature (Celsius, Fahrenheit, Kelvin) +* Volume (liters, gallons, cubic meters, etc.) +* Pressure (Pascal, bar, PSI, etc.) +* Speed (m/s, km/h, mph, etc.) +* Area (square meters, acres, etc.) +* Time (seconds, minutes, hours, etc.) +* Energy (joules, kilowatt-hours, etc.) +* Power (watts, horsepower, etc.) + +This processor is essential for: +* Standardizing measurements across systems +* Converting between international and US units +* Ensuring consistent data representation +* Supporting multi-regional deployments +* Enabling system interoperability *** ## Required input +The processor requires: +1. A numerical field that has a measurement unit defined +2. The field must be of type measurement property *** ## Configuration -Describe the configuration parameters here +### Field + +Select the numerical field containing the value to convert. The field must have a measurement unit defined. + +### Output Unit + +Select the desired unit of measurement for the output value. The available units are dynamically populated based on the input field's measurement unit type. + +## Output + +The processor creates a new event containing: +* All original fields from the input event +* The selected field is updated with the converted value + +### Example + +#### Input Event +```json +{ + "sensorId": "temp01", + "temperature": 20.0, + "humidity": 65, + "timestamp": 1586380104915 +} +``` + +#### Configuration +* Field: temperature (with unit Celsius) +* Output Unit: Fahrenheit + +#### Output Event +```json +{ + "sensorId": "temp01", + "temperature": 68.0, + "humidity": 65, + "timestamp": 1586380104915 +} +``` + +## Use Cases + +1. **International Operations** + * Convert between metric and imperial units + * Standardize measurements across regions + * Support global data analysis + * Enable multi-market compliance + +2. **System Integration** + * Convert between different system standards + * Normalize data from multiple sources + * Enable cross-platform compatibility + * Support legacy system integration -### 1st parameter +3. **Scientific Applications** + * Convert between scientific units + * Support experimental data analysis + * Enable cross-discipline collaboration + * Maintain measurement precision +4. **Industrial Processes** + * Convert process measurements + * Standardize control parameters + * Support equipment interoperability + * Enable cross-vendor integration -### 2nd parameter +## Notes -## Output \ No newline at end of file +* The input field must have a measurement unit defined +* Only compatible units are available for selection +* The conversion is performed in-place +* The original value is replaced with the converted value +* All conversions maintain numerical precision +* Conversion formulas follow international standards +* Invalid unit combinations are rejected +* Null values are preserved in output +* Conversion is stateless +* Conversion accuracy depends on input precision +* Temperature conversions handle both relative and absolute scales \ No newline at end of file diff --git a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.booloperator.edge/documentation.md b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.booloperator.edge/documentation.md index 776a949eca..854df1b649 100644 --- a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.booloperator.edge/documentation.md +++ b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.booloperator.edge/documentation.md @@ -16,7 +16,7 @@ ~ --> -## Signal Edge Filter +## Signal Edge Filter <p align="center"> <img src="icon.png" width="150px;" class="pe-image-documentation"/> @@ -26,27 +26,155 @@ ## Description -Observes a boolean value and forwards the event when a signal edge is detected +The Signal Edge Filter processor detects and forwards events when a boolean signal changes state (edge detection). It supports: +* Rising edge detection (FALSE -> TRUE) +* Falling edge detection (TRUE -> FALSE) +* Both edge detection +* Configurable delay +* Event selection options + +This processor is essential for: +* Detecting state transitions +* Monitoring signal changes +* Triggering actions on state changes +* Implementing edge-triggered logic *** ## Required input -### Boolean Field -Boolean field that is observed +The processor requires a data stream containing at least one boolean field to monitor for state changes. *** ## Configuration -### Kind of edge -* Detect rising edges -* Detect falling edges -* Detect both + +### Boolean Signal + +Select the boolean field to monitor for state changes. This field will be used to detect signal edges. + +### Signal Edge + +Choose the type of edge to detect: + +* **FALSE -> TRUE**: Rising edge detection + * Triggers when signal changes from false to true + * Use for: Detecting activation events + * Example: Device turned on + +* **TRUE -> FALSE**: Falling edge detection + * Triggers when signal changes from true to false + * Use for: Detecting deactivation events + * Example: Device turned off + +* **BOTH**: Both edge detection + * Triggers on any state change + * Use for: Monitoring all transitions + * Example: Any state change ### Delay -Defines for how many events the signal must be stable before result is emitted. -(E.g. if set to 2, the result is not emitted if value toggles between true and false, -it fires when two consecutive events are detected after the flank) + +Specify how many events to wait before forwarding the result after an edge is detected: +* Minimum: 0 (immediate forwarding) +* Use for: Debouncing signals +* Example: Wait 5 events to ensure stable state + +### Output Event Selection + +Choose which event(s) to forward after the delay: + +* **First**: Forward only the first event after edge detection + * Use for: Single trigger actions + * Example: Start a process once + +* **Last**: Forward only the last event after delay + * Use for: Final state capture + * Example: Capture stable state + +* **All**: Forward all events during delay + * Use for: Continuous monitoring + * Example: Track state changes ## Output -Emits input event, when the signal edge is detected + +The processor creates a new event containing: +* All original fields from the input event +* The event is forwarded based on the configured delay and event selection + +### Example + +#### Input Event Stream +```json +[ + { + "deviceId": "sensor01", + "isActive": false, + "timestamp": 1586380104915 + }, + { + "deviceId": "sensor01", + "isActive": true, + "timestamp": 1586380105015 + }, + { + "deviceId": "sensor01", + "isActive": true, + "timestamp": 1586380105115 + } +] +``` + +#### Configuration 1: Rising Edge with First Event +* Boolean Signal: isActive +* Signal Edge: FALSE -> TRUE +* Delay: 0 +* Output Event Selection: First + +#### Output Event +```json +{ + "deviceId": "sensor01", + "isActive": true, + "timestamp": 1586380105015 +} +``` + +#### Configuration 2: Both Edges with Last Event +* Boolean Signal: isActive +* Signal Edge: BOTH +* Delay: 1 +* Output Event Selection: Last + +#### Output Event +```json +{ + "deviceId": "sensor01", + "isActive": true, + "timestamp": 1586380105115 +} +``` + +## Use Cases + +1. **Device Monitoring** + * Detect device power state changes + * Monitor sensor activation + * Track equipment status + * Detect system transitions + +2. **Process Control** + * Trigger actions on state changes + * Monitor process transitions + * Detect phase changes + * Track state machines + +## Notes + +* Only boolean fields can be monitored +* Edge detection is stateful +* Delay is event-based, not time-based +* Event selection affects output behavior +* Processing is stateful +* Edge detection is immediate +* Delay starts after edge detection +* Event selection applies after delay diff --git a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.booloperator.state/documentation.md b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.booloperator.state/documentation.md index 34802cd97c..5998fe3830 100644 --- a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.booloperator.state/documentation.md +++ b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.booloperator.state/documentation.md @@ -16,7 +16,7 @@ ~ --> -## Boolean To State +## Boolean to State <p align="center"> <img src="icon.png" width="150px;" class="pe-image-documentation"/> @@ -26,32 +26,133 @@ ## Description -Converts boolean fields to a state string representing the current state of the system. -This processor requires one or multiple boolean values in the data stream. -For the selected value which is true, the runtime name is added as the state field. +The Boolean to State processor converts boolean fields into a descriptive state string. It supports: +* Multiple boolean inputs +* Custom state mapping +* Default state handling +* Runtime name mapping +* JSON configuration +* State prioritization + +This processor is essential for: +* Converting boolean states +* Creating descriptive states +* Mapping boolean values +* Handling multiple states +* Building state machines +* Creating human-readable states + *** ## Required input -### Boolean Fields +The processor requires a data stream containing at least one boolean field to convert into a state. -Boolean fields that are converted to the state when true +*** -### Default State +## Configuration -When all boolean values are false, a default state can be defined +### Current State -### Mapping Configuration +Select one or more boolean fields to monitor. When a field is true, its runtime name will be used as the state value. If multiple fields are true, the first one in the list will be used. -Configuration to provide a string mapping for each possible value. -On the left ist the value of the runtime name and on the right the new value (e.g. {"runtimeName": "newValue"}). +### Default State -*** +Define a default state string to use when all boolean fields are false. This ensures a state is always present in the output. -## Configuration +### Mapping Configuration + +Define custom mappings to replace the runtime names with your own state strings. Use JSON format: -No further configuration required +```json +{ + "runtimeName1": "Custom State 1", + "runtimeName2": "Custom State 2" +} +``` ## Output -The output contains a new value with the string values of the state +The processor creates a new event containing: +* All original fields from the input event +* A new field named "current_state" containing the state string + +### Example + +#### Input Event +```json +{ + "deviceId": "machine01", + "isRunning": true, + "isError": false, + "isMaintenance": false, + "timestamp": 1586380104915 +} +``` + +#### Configuration +* Current State: isRunning, isError, isMaintenance +* Default State: "IDLE" +* Mapping Configuration: +```json +{ + "isRunning": "OPERATIONAL", + "isError": "ERROR", + "isMaintenance": "MAINTENANCE" +} +``` + +#### Output Event +```json +{ + "deviceId": "machine01", + "isRunning": true, + "isError": false, + "isMaintenance": false, + "timestamp": 1586380104915, + "current_state": "OPERATIONAL" +} +``` + +## Use Cases + +1. **Equipment Monitoring** + * Convert device states + * Map status flags + * Create state machines + * Monitor conditions + * Track operations + +2. **Process Control** + * Map process states + * Convert conditions + * Create workflows + * Monitor phases + * Track progress + +3. **System Integration** + * Map system states + * Convert signals + * Create interfaces + * Monitor status + * Track conditions + +4. **Quality Control** + * Map quality states + * Convert checks + * Create reports + * Monitor tests + * Track results + +## Notes + +* Only boolean fields can be converted +* State mapping is case-sensitive +* Default state is required +* JSON configuration is optional +* Processing is stateless +* Multiple states require prioritization +* Consider state naming +* Mapping is immediate +* No delay in processing +* State is always present diff --git a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.state.buffer/documentation.md b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.state.buffer/documentation.md index e1c87bcbc4..f0311fe452 100644 --- a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.state.buffer/documentation.md +++ b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.state.buffer/documentation.md @@ -26,29 +26,94 @@ ## Description -Buffers values of a sensor, while state does not change. -Select a state field in the event. Events are buffered as long as state field does not change. -When it changes result -event is emitted. +The State Buffer processor caches sensor values during specific states. It supports: +* State-based value buffering +* Timestamp tracking +* Sensor value caching +* State monitoring + +This processor is essential for: +* Caching sensor values +* Tracking state changes +* Monitoring conditions +* Storing measurements *** ## Required input -Define the state and sensor value field +The processor requires a data stream containing: +* A timestamp field +* A state field +* At least one sensor value field to cache + +*** + +## Configuration ### Timestamp -A mapping property for a timestamp field +Select the field containing the event timestamp. This is used to track when values are buffered. ### State -Select the field representing the state +Select the field containing the state information. This determines when values are cached. -### Sensor value to cache +### Sensor Value to Cache -Select the field with the numerical values to buffer +Select the sensor value field that should be cached while the state is active. ## Output -Emits a new event on state change, with the fields `timestamp`, `state`, and a list containing all `sensor values`. +The processor creates a new event containing: +* A timestamp field +* A list of buffered values +* A list of states + +### Example + +#### Input Event +```json +{ + "deviceId": "sensor01", + "timestamp": 1586380104915, + "state": ["active"], + "temperature": 23.5 +} +``` + +#### Configuration +* Timestamp: timestamp +* State: state +* Sensor Value to Cache: temperature + +#### Output Event (when state changes from "active" to "inactive") +```json +{ + "timestamp": 1586380105915, + "values": [23.5, 24.1, 24.3], + "state": ["active"] +} +``` + +## Use Cases + +1. **State Monitoring** + * Cache sensor values + * Track state changes + * Monitor conditions + * Store measurements + +2. **Data Analysis** + * Analyze state patterns + * Track value changes + * Monitor conditions + * Store measurements + +## Notes + +* Values are cached during active states +* Timestamps are preserved +* State changes trigger updates +* Processing is stateful +* Multiple values can be buffered diff --git a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.state.labeler.buffer/documentation.md b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.state.labeler.buffer/documentation.md index 4b73d3e7b8..a39fd749a5 100644 --- a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.state.labeler.buffer/documentation.md +++ b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.state.labeler.buffer/documentation.md @@ -26,40 +26,114 @@ ## Description -Apply a rule to a time-series recorded during a state of a machine. -(E.g. when minimum value is lower then 10, add label `not ok` else add label `ok`) - +The State Buffer Labeler processor adds labels to sensor time-series data based on statistical operations and user-defined rules. It supports: +* State-based labeling +* Statistical operations (min, max, average) +* Custom rule definition +* Multiple conditions +* Default labels + +This processor is essential for: +* Adding context to data +* Classifying measurements +* Identifying patterns +* Marking conditions *** ## Required input -Requires a list with sensor values and a field defining the state +The processor requires a data stream containing: +* A state field (array of strings) +* A sensor value field (array of numbers) -### Sensor values +*** -An array representing sensor values recorded during the state. +## Configuration -### State field +### State Field -A field representing the state when the sensor values where recorded. +Select the field containing the state information. This determines when rules are applied. -*** +### Select a specific state -## Configuration +Add a filter to define which states to evaluate. Use '*' to select all states. -### Select a specific state -When you are interested in the values of a specific state add it here. -All other states will be ignored. To get results of all states enter `*` +### Sensor values + +Select the array containing the sensor values to evaluate against the rules. ### Operation -Operation that will be performed on the sensor values (calculate `maximim`, or `average`, or `minimum`) + +Define the statistical operation to apply to the sensor values: +* Minimum: Get the lowest value +* Maximum: Get the highest value +* Average: Calculate the mean value ### Condition -Define a rule which label to add. Example: `<;5;nok` means when the calculated value is smaller then 5 add label ok. -The default label can be defined with `*;nok`. -The first rule that is true defines the label. Rules are applied in the same order as defined here. +Add conditions in the format: +* `<;5;ok` - Label as "ok" if value is less than 5 +* `<;10;ok` - Label as "ok" if value is less than 10 +* `*;nok` - Default label "nok" for all other cases ## Output -Appends a new field with the label defined in the Condition Configuration + +The processor creates a new event containing: +* All original fields from the input event +* A new label field based on the conditions + +### Example + +#### Input Event +```json +{ + "deviceId": "sensor01", + "timestamp": 1586380104915, + "state": ["active"], + "values": [23.5, 24.1, 24.3] +} +``` + +#### Configuration +* State Field: state +* Select a specific state: active +* Sensor values: values +* Operation: Average +* Condition: "<;20;cold", "<;30;warm", "*;hot" + +#### Output Event +```json +{ + "deviceId": "sensor01", + "timestamp": 1586380104915, + "state": ["active"], + "values": [23.5, 24.1, 24.3], + "label": "warm" +} +``` + +## Use Cases + +1. **Data Classification** + * Add context to data + * Classify measurements + * Identify patterns + * Mark conditions + +2. **Quality Control** + * Label quality levels + * Mark thresholds + * Identify issues + * Track conditions + +## Notes + +* Conditions are evaluated in order +* Default label is required +* State filtering is optional +* Processing is stateless +* Multiple conditions supported +* Statistical operation is applied before condition evaluation +* Input arrays must contain numeric values +* State field must be an array of strings diff --git a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.state.labeler.number/documentation.md b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.state.labeler.number/documentation.md index 8c34b55db3..2a9d78435a 100644 --- a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.state.labeler.number/documentation.md +++ b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.state.labeler.number/documentation.md @@ -26,27 +26,95 @@ ## Description -Apply a rule to a value of a field. (E.g. when minimum value is lower then 10, add label `not ok` else add label `ok`) +The Number Labeler processor adds labels to numerical values based on user-defined rules. It supports: +* Value-based labeling +* Custom rule definition +* Multiple conditions +* Default labels +* Value comparison + +This processor is essential for: +* Classifying measurements +* Adding context to data +* Identifying patterns +* Marking conditions *** ## Required input -Requires a sensor value - -### Sensor value - -A number representing the current sensor value. +The processor requires a data stream containing: +* A numerical value field to evaluate +* Timestamp information *** ## Configuration +### Sensor Value + +Select the numerical field to evaluate against the rules. + +### Label Name + +Specify the name of the label field in the output event. + ### Condition -Define a rule which label to add. Example: `<;5;nok` means when the calculated value is smaller then 5 add label ok. -The default label can be defined with `*;nok`. -The first rule that is true defines the label. Rules are applied in the same order as defined here. +Add conditions in the format: +* `<;5;low` - Label as "low" if value is less than 5 +* `<;10;medium` - Label as "medium" if value is less than 10 +* `*;high` - Default label "high" for all other cases ## Output -Appends a new field with the label defined in the Condition Configuration + +The processor creates a new event containing: +* All original fields from the input event +* A new label field based on the conditions + +### Example + +#### Input Event +```json +{ + "deviceId": "sensor01", + "timestamp": 1586380104915, + "temperature": 23.5 +} +``` + +#### Configuration +* Sensor Value: temperature +* Label Name: temperature_status +* Condition: "<;20;cold", "<;30;warm", "*;hot" + +#### Output Event +```json +{ + "deviceId": "sensor01", + "timestamp": 1586380104915, + "temperature": 23.5, + "temperature_status": "warm" +} +``` + +## Use Cases + +1. **Data Classification** + * Classify measurements + * Add context to data + * Identify patterns + * Mark conditions + +2. **Quality Control** + * Label quality levels + * Mark thresholds + * Identify issues + * Track conditions + +## Notes + +* Conditions are evaluated in order +* Default label is required +* Processing is stateless +* Multiple conditions supported diff --git a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.staticmetadata/documentation.md b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.staticmetadata/documentation.md index ebfc4a7092..9c687c7863 100644 --- a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.staticmetadata/documentation.md +++ b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.staticmetadata/documentation.md @@ -17,52 +17,121 @@ --> ## Static Metadata Enricher -Enrich a data stream by dynamically adding fields based on user-provided static metadata configuration. +<p align="center"> + <img src="icon.png" width="150px;" class="pe-image-documentation"/> +</p> ---- +*** ## Description -The Static Metadata Enricher is designed to enrich a data stream by dynamically adding fields based on user-provided -metadata configuration. Users can specify static properties, and the processor will process each event, adding fields -according to the provided key-value pairs. The output strategy is determined dynamically based on the provided metadata. -For added convenience, users also have the option of uploading a CSV file with metadata information. +The Static Metadata Enricher processor adds predefined metadata fields to events. It supports: +* Custom field addition +* Multiple data types (String, Boolean, Float, Integer) +* Field labeling +* Field descriptions +* Runtime naming +* Metadata enrichment -### Configuration +This processor is essential for: +* Adding context to data +* Enriching events +* Creating metadata +* Building annotations +* Standardizing fields +* Documenting data -For each metadata entry, configure the following three options: +*** -- **Runtime Name:** A unique identifier for the property during runtime. -- **Value:** The value associated with the property. -- **Data Type:** The data type of the property value. +## Required input -#### Using CSV Option +The processor requires a data stream to enrich with additional metadata fields. -Alternatively, you can utilize the CSV upload feature by creating a CSV file with the following format: +*** -``` -Runtime Name,Runtime Value,Data Type -sensorType,Temperature,String -maxSensorValue,100.0,Float -minSensorValue,0,Float -``` +## Configuration + +### Metadata Input + +Configure the metadata fields to add to each event: + +#### Runtime Name +Enter the name that will be used for the field in the output event. + +#### Runtime Value +Enter the value that will be assigned to the field. + +#### Data Type +Select the data type of the value: +* String: Text values +* Boolean: true/false values +* Float: Decimal numbers +* Integer: Whole numbers + +#### Label (Optional) +Provide a short label describing the field. -## Example -### Input Event +#### Description (Optional) +Provide a detailed description of the field. +## Output + +The processor creates a new event containing: +* All original fields from the input event +* The configured metadata fields with their values + +### Example + +#### Input Event ```json { - "reading": 25.5 + "deviceId": "sensor01", + "temperature": 23.5, + "humidity": 45.2 } ``` -### Output Event +#### Configuration +* Runtime Name: location +* Runtime Value: Building A +* Data Type: String +* Label: Sensor Location +* Description: Physical location of the sensor +#### Output Event ```json { - "reading": 25.5, - "sensorType": "Temperature", - "maxSensorValue": 100.0, - "minSensorValue": 0.0 + "deviceId": "sensor01", + "temperature": 23.5, + "humidity": 45.2, + "location": "Building A" } ``` + +## Use Cases + +1. **Data Enrichment** + * Add context to data + * Enrich events + * Create metadata + * Build annotations + * Standardize fields + +2. **Documentation** + * Document data + * Add descriptions + * Create labels + * Build metadata + * Standardize fields + +## Notes + +* Multiple fields can be added +* Data types must match values +* Labels are optional +* Descriptions are optional +* Processing is stateless +* Field names must be unique +* Values are automatically cast to the selected data type +* Original event fields are preserved +* Metadata is consistent across all events diff --git a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.stringoperator.state/documentation.md b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.stringoperator.state/documentation.md index c58304e58b..f38402ab5e 100644 --- a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.stringoperator.state/documentation.md +++ b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.stringoperator.state/documentation.md @@ -26,20 +26,81 @@ ## Description -Convert string fields to a state representing the current state of the system. -This processor requires one or multiple string values in the data stream. -For each of the selected values is added to the states field. +The String To State processor transforms string properties into a list of state values. It supports: +* Multiple string inputs +* List-based state output +* Field value preservation +* State collection + +This processor is essential for: +* Converting strings to states +* Collecting multiple states +* Preserving field values +* Creating state lists + *** ## Required input -### String Fields -String fields that are added to the state array +The processor requires a data stream containing at least one string field to convert into a state. *** ## Configuration -No further configuration required + +### State Field + +Select one or more string fields to convert into states. The values of these fields will be collected into a list of states. ## Output -The output contains a new value with the string values of the state + +The processor creates a new event containing: +* All original fields from the input event +* A new field named "current_state" containing a list of the selected string field values + +### Example + +#### Input Event +```json +{ + "deviceId": "sensor01", + "status": "running", + "mode": "normal" +} +``` + +#### Configuration +* State Fields: status, mode + +#### Output Event +```json +{ + "deviceId": "sensor01", + "status": "running", + "mode": "normal", + "current_state": ["running", "normal"] +} +``` + +## Use Cases + +1. **State Collection** + * Gather multiple states + * Track field values + * Monitor statuses + * Collect modes + +2. **State Analysis** + * Analyze state combinations + * Track value patterns + * Monitor field changes + * Process state lists + +## Notes + +* Multiple fields can be selected +* Output is always a list +* Original fields are preserved +* Processing is stateless +* Empty selections result in empty list +* Field values are preserved as-is diff --git a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.timestampextractor/documentation.md b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.timestampextractor/documentation.md index c3e64d8684..0331d4e3a9 100644 --- a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.timestampextractor/documentation.md +++ b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.processor.timestampextractor/documentation.md @@ -26,14 +26,30 @@ ## Description -This processor extracts a timestamp into the individual time fields (e.g. day field, hour field, ....) +The Timestamp Extractor processor breaks down a timestamp into its individual time components. It supports: +* Year extraction +* Month extraction +* Day extraction +* Hour extraction +* Minute extraction +* Second extraction +* Weekday extraction +* Custom field selection +* Component isolation + +This processor is essential for: +* Time analysis +* Date processing +* Time component extraction +* Temporal analysis +* Time-based grouping +* Time series analysis *** ## Required input -This processor requires an event that provides a timestamp value (a field that is marked to be of type ``http://schema -.org/DateTime``. +The processor requires a data stream containing at least one timestamp field to extract components from. *** @@ -41,12 +57,96 @@ This processor requires an event that provides a timestamp value (a field that i ### Timestamp Field -The field of the event containing the timestamp to parse. +Select the field containing the timestamp to extract components from. This should be a valid timestamp value. ### Extract Fields -Select the individual parts of the timestamp that should be extracted, e.g., Year, Minute and Day. +Select which time components to extract: +* Year (numeric) +* Month (numeric, 1-12) +* Day (numeric, 1-31) +* Hour (numeric, 0-23) +* Minute (numeric, 0-59) +* Second (numeric, 0-59) +* Weekday (string: Monday-Sunday) ## Output -The output of this processor is a new event that contains the fields selected by the ``Extract Fields`` parameter. \ No newline at end of file +The processor creates a new event containing: +* All original fields from the input event +* New fields for each extracted time component with prefix "timestamp" +* Original timestamp preserved + +### Example + +#### Input Event +```json +{ + "deviceId": "sensor01", + "timestamp": 1586380104915, + "value": 23.5 +} +``` + +#### Configuration +* Timestamp Field: timestamp +* Extract Fields: year, month, day, hour, minute, weekday + +#### Output Event +```json +{ + "deviceId": "sensor01", + "timestamp": 1586380104915, + "value": 23.5, + "timestampYear": 2020, + "timestampMonth": 4, + "timestampDay": 8, + "timestampHour": 15, + "timestampMinute": 35, + "timestampWeekday": "Wednesday" +} +``` + +## Use Cases + +1. **Time Analysis** + * Extract time components + * Analyze patterns + * Group by time + * Track changes + * Build time series + +2. **Data Processing** + * Process timestamps + * Extract components + * Analyze patterns + * Group data + * Build metrics + +3. **Reporting** + * Generate time reports + * Extract components + * Analyze patterns + * Group data + * Build summaries + +4. **Monitoring** + * Monitor time patterns + * Extract components + * Analyze trends + * Track changes + * Build alerts + +## Notes + +* Timestamp must be valid +* Components are optional +* Processing is stateless +* Multiple components supported +* Extraction is immediate +* No delay in processing +* Original timestamp preserved +* Components are standardized +* Weekday is returned as string +* Month is 1-based (1-12) +* Hour is 24-hour format (0-23) \ No newline at end of file diff --git a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.round/documentation.md b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.round/documentation.md index 90f67db771..c8b05a51cc 100644 --- a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.round/documentation.md +++ b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.round/documentation.md @@ -26,14 +26,18 @@ ## Description -This processor rounds numeric values to the given decimal places. -It supports multiple rounding strategies. +The Number Rounder processor provides precise control over the decimal places in numerical data by applying various rounding strategies. This is essential for: +* Ensuring consistent precision across numerical data +* Reducing noise in measurements +* Improving data readability +* Meeting specific business or technical requirements for numerical precision +* Standardizing numerical outputs for downstream processing *** ## Required input -This processor requires an event that provides numerical properties. +This processor requires an event that contains one or more numerical properties (integers or floating-point numbers). *** @@ -41,26 +45,95 @@ This processor requires an event that provides numerical properties. ### Fields to Be Rounded -Select which fields of the event should be rounded. +Select which numerical fields in the event should be rounded. Multiple fields can be selected, and each will be rounded according to the same configuration. ### Number of Digits -Specify the number of digits after the decimal point to round/keep, e.g., if number is 2.8935 and 'digits' is 3, -the result will be 2.894. +Specify the number of decimal places to keep after rounding. This determines the precision of the output: +* Positive values (e.g., 2): Keep that many decimal places +* Zero (0): Round to whole numbers +* Negative values (e.g., -2): Round to tens, hundreds, etc. + +Examples: +* Input: 2.8935, Digits: 3 → Output: 2.894 +* Input: 2.8935, Digits: 2 → Output: 2.89 +* Input: 2.8935, Digits: 0 → Output: 3 +* Input: 285.8935, Digits: -2 → Output: 300 ### Mode of Rounding -Specify the mode of rounding. -Supported rounding modes: -* `UP`: Rounding mode to round away from zero. Always increments the digit prior to a non-zero discarded fraction. Note that this rounding mode never decreases the magnitude of the calculated value. -* `DOWN`: Rounding mode to round towards zero. Never increments the digit prior to a discarded fraction (i.e., truncates). Note that this rounding mode never increases the magnitude of the calculated value. -* `CEILING`: Rounding mode to round towards positive infinity. If the result is positive, behaves as for `UP`; if negative, behaves as for `DOWN`. Note that this rounding mode never decreases the calculated value -* `FLOOR`: Rounding mode to round towards negative infinity. If the result is positive, behave as for `DOWN`; if negative, behave as for `UP`. Note that this rounding mode never increases the calculated value. -* `HALF_UP`: Rounding mode to round towards "nearest neighbor" unless both neighbors are equidistant, in which case round up. Behaves as for `UP` if the discarded fraction is ≥ 0.5; otherwise, behaves as for `DOWN`. -* `HALF_DOWN`: Rounding mode to round towards "nearest neighbor" unless both neighbors are equidistant, in which case round down. Behaves as for `UP` if the discarded fraction is > 0.5; otherwise, behaves as for `DOWN`. -* `HALF_EVEN`: Rounding mode to round towards the "nearest neighbor" unless both neighbors are equidistant, in which case, round towards the even neighbor. Behaves as for `HALF_UP` if the digit to the left of the discarded fraction is odd; behaves as for `HALF_DOWN` if it's even. Note that this is the rounding mode that statistically minimizes cumulative error when applied repeatedly over a sequence of calculations. +Select the rounding strategy to use. Each mode handles rounding differently, especially around the midpoint between two numbers: + +* **UP** + * Always rounds away from zero + * 3.1 → 4, -3.1 → -4 + * Use when you need to ensure the result is never smaller in magnitude + +* **DOWN** + * Always rounds toward zero (truncates) + * 3.7 → 3, -3.7 → -3 + * Use when you need to ensure the result is never larger in magnitude + +* **CEILING** + * Always rounds toward positive infinity + * 3.1 → 4, -3.7 → -3 + * Use when you need to ensure the result never decreases + +* **FLOOR** + * Always rounds toward negative infinity + * 3.7 → 3, -3.1 → -4 + * Use when you need to ensure the result never increases + +* **HALF_UP** (Most common) + * Rounds to nearest number, ties round up + * 3.5 → 4, 3.4 → 3, -3.5 → -4 + * Use for standard mathematical rounding + +* **HALF_DOWN** + * Rounds to nearest number, ties round down + * 3.5 → 3, 3.6 → 4, -3.5 → -3 + * Use when ties should be rounded down + +* **HALF_EVEN** (Banker's Rounding) + * Rounds to nearest number, ties round to even neighbor + * 3.5 → 4, 4.5 → 4, -3.5 → -4 + * Use to minimize cumulative rounding errors in large datasets ## Output -The output of this processor is the same event with the fields selected by the ``Fiels to Be Rounded`` parameter rounded -to ``Number of Digits`` digits. \ No newline at end of file +The processor outputs an event with the same structure as the input, but with the selected numerical fields rounded according to the configuration. All other fields remain unchanged. + +### Example + +#### Input Event +```json +{ + "sensorId": "temp01", + "temperature": 23.4567, + "pressure": 1013.8935, + "humidity": 45.5000 +} +``` + +#### Configuration +* Fields to Be Rounded: temperature, pressure +* Number of Digits: 2 +* Mode of Rounding: HALF_UP + +#### Output Event +```json +{ + "sensorId": "temp01", + "temperature": 23.46, + "pressure": 1013.89, + "humidity": 45.5000 +} +``` + +## Use Cases + +1. **Measurement Data**: Standardize precision of sensor readings +2. **Financial Calculations**: Ensure proper decimal handling in monetary values +3. **Scientific Analysis**: Control significant figures in experimental data +4. **Display Formatting**: Prepare numbers for user interfaces +5. **Data Storage**: Optimize numerical precision for storage efficiency \ No newline at end of file diff --git a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.split-array/documentation.md b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.split-array/documentation.md index 665c6092b9..5af7643708 100644 --- a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.split-array/documentation.md +++ b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.split-array/documentation.md @@ -26,29 +26,117 @@ ## Description -This processor takes an array of event properties and creates an event for each of them. -Further property of the events can be added to each element. +The Split Array processor transforms array fields into multiple individual events, with each array element becoming a separate event. It supports: +* Array element extraction +* Context preservation +* Nested field handling +* Custom field naming + +This processor is essential for: +* Converting batch data into individual events +* Processing array elements independently +* Distributing array data across streams +* Enabling element-wise analysis *** ## Required input -This processor works with any event that has a field of type ``list``. +The processor requires a data stream containing at least one array field. The array can contain elements of any supported data type: +* Numbers (integer or float) +* Strings +* Booleans +* Objects +* Nested arrays *** ## Configuration -### Keep Fields - -Fields of the event that should be kept in each resulting event. +### Array Field Selection -### List field +Select the array field to split into individual events. The field must be an array type. -The name of the field that contains the list values that should be split. +### Keep Fields +Select one or more fields from the input event that should be preserved in each output event. These fields will be copied to each output event. ## Output -This data processor produces an event with all fields selected by the ``Keep Fields`` parameter and all fields of the - selected list field. \ No newline at end of file +For each element in the input array, the processor creates a new event containing: +* The array element as a single value in a field named "array_value" +* All selected fields from the original event + +### Example + +#### Input Event +```json +{ + "deviceId": "sensor123", + "timestamp": 1586380104915, + "measurements": [22.5, 23.1, 22.8, 23.4], + "status": "active" +} +``` + +#### Configuration +* Array Field: measurements +* Keep Fields: deviceId, timestamp, status + +#### Output Events +```json +// First element +{ + "deviceId": "sensor123", + "timestamp": 1586380104915, + "status": "active", + "array_value": 22.5 +} + +// Second element +{ + "deviceId": "sensor123", + "timestamp": 1586380104915, + "status": "active", + "array_value": 23.1 +} + +// Third element +{ + "deviceId": "sensor123", + "timestamp": 1586380104915, + "status": "active", + "array_value": 22.8 +} + +// Fourth element +{ + "deviceId": "sensor123", + "timestamp": 1586380104915, + "status": "active", + "array_value": 23.4 +} +``` + +## Use Cases + +1. **Batch Processing** + * Split batch sensor readings + * Process multi-measurement data + * Handle grouped observations + * Transform batch uploads + +2. **Data Distribution** + * Distribute workload across processors + * Enable parallel processing + * Balance processing load + * Scale data processing + +## Notes + +* Output events maintain original event order +* Empty arrays produce no output events +* Null array elements are preserved +* Processing is stateless +* Memory usage scales with array size +* Nested fields are handled automatically \ No newline at end of file diff --git a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.stringoperator.counter/documentation.md b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.stringoperator.counter/documentation.md index 57c778cb89..a8e9d8f411 100644 --- a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.stringoperator.counter/documentation.md +++ b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.stringoperator.counter/documentation.md @@ -26,34 +26,107 @@ ## Description -This processor monitors a string field and counts how often the value of the string changes. -Hereby, a change is characterized by -the value of the field before and the value after the change, combined forming a pair. -The processor keeps track of the counter for each pair. +The String Counter processor counts changes in string field values. It supports: +* Value change detection +* Change pair tracking +* Incremental counting +* State transition monitoring + +This processor is essential for: +* Counting value changes +* Tracking state transitions +* Monitoring string patterns +* Measuring change frequency *** ## Required input -A string field is required in the data stream and can be selected with the field mapping. - -### String Field - -The string field to be monitored. +The processor requires a data stream containing at least one string field to monitor for value changes. *** ## Configuration -(no further configuration required) - -## Output +### String Field -The following three fields are appended to the event: +Select the string field to monitor for value changes. This field will be used to detect changes and increment the counter. -* [counter] numerical field with the current count value for the given value pair -* [change_from] the value of the string before the change -* [change_to] the value of the string after the change +## Output -The event is emitted whenever the value of the string field changes. +The processor creates a new event containing: +* All original fields from the input event +* A new field named "counter" containing the current count of value changes +* A new field named "change_from" containing the previous value +* A new field named "change_to" containing the new value + +### Example + +#### Input Event Stream +```json +{ + "deviceId": "sensor01", + "status": "idle" +} +``` +```json +{ + "deviceId": "sensor01", + "status": "running" +} +``` +```json +{ + "deviceId": "sensor01", + "status": "idle" +} +``` + +#### Configuration +* String Field: status + +#### Output Event +```json +{ + "deviceId": "sensor01", + "status": "running", + "change_from": "idle", + "change_to": "running", + "counter": 1 +} +``` +```json +{ + "deviceId": "sensor01", + "status": "idle", + "change_from": "running", + "change_to": "idle", + "counter": 2 +} +``` + +## Use Cases + +1. **State Monitoring** + * Track state changes + * Count transitions + * Monitor patterns + * Measure frequency + +2. **Process Analysis** + * Analyze workflows + * Track sequences + * Count cycles + * Monitor operations + +## Notes + +* Only counts actual value changes +* Ignores consecutive identical values +* Processing is stateful +* Counter is incremental +* Events are only emitted on value changes +* Original event fields are preserved +* Change pairs are tracked (from -> to) +* Counter starts at 1 for first change diff --git a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.stringoperator.timer/documentation.md b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.stringoperator.timer/documentation.md index c6dd5a08ae..60c3d367ea 100644 --- a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.stringoperator.timer/documentation.md +++ b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.stringoperator.timer/documentation.md @@ -26,35 +26,98 @@ ## Description -This processor measures how long a value of a string field does not change. -Once the value is changes the event with the measured time and the corresponding string value is emitted. - +The String Timer processor measures how long a string field maintains a specific value. It supports: +* String value monitoring +* Duration measurement +* Multiple time units +* Configurable output frequency + +This processor is essential for: +* Measuring state durations +* Tracking value persistence +* Monitoring string states +* Calculating time periods *** ## Required input -A string field is required in the data stream. +The processor requires a data stream containing at least one string field to monitor for value changes. -### Field +*** -The string field which is monitored for any value changes. +## Configuration +### String Field -*** +Select the string field to monitor for value changes. This field will be used to measure how long it maintains a specific value. -## Configuration +### Output Unit + +Choose the time unit for the measured duration: +* Milliseconds (default) +* Seconds +* Minutes ### Output Frequency -Define when an event should be emitted, either on each input event or just when the string value changes. +Define when the processor should emit an output event: +* On Input Event: Emit for every input event +* When String Value Changes: Emit only when the string value changes ## Output -The following two fields are appended to the event: - -* [measured_time] the measured time for the string value to not change -* [field_value] the corresponding string value - -The event is emitted whenever the value of the string field changes. +The processor creates a new event containing: +* All original fields from the input event +* A measured_time field showing the duration in the selected unit +* A field_value field showing the previous string value + +### Example + +#### Input Event +```json +{ + "deviceId": "machine01", + "status": "running", + "timestamp": 1586380104915 +} +``` + +#### Configuration +* String Field: status +* Output Unit: Seconds +* Output Frequency: When String Value Changes + +#### Output Event (when status changes from "running" to "stopped") +```json +{ + "deviceId": "machine01", + "status": "stopped", + "timestamp": 1586380106915, + "measured_time": 2.0, + "field_value": "running" +} +``` + +## Use Cases + +1. **State Monitoring** + * Measure state durations + * Track value persistence + * Monitor status changes + * Calculate time periods + +2. **Process Control** + * Measure process durations + * Track state changes + * Monitor operations + * Calculate times + +## Notes + +* Only string fields can be monitored +* Time measurement is stateful +* Measurement starts on value change +* Measurement ends on next change +* Output depends on frequency setting diff --git a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.taskduration/documentation.md b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.taskduration/documentation.md index 5cbe48e563..9e262e1693 100644 --- a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.taskduration/documentation.md +++ b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.taskduration/documentation.md @@ -18,27 +18,123 @@ ## Task Duration +<p align="center"> + <img src="icon.png" width="150px;" class="pe-image-documentation"/> +</p> *** ## Description -This processors computes the duration of a task, i.e., a field containing a task description. It outputs an event - every time this task value changes and computes the duration between the first occurrence of this task and the - current event. For instance, you can use this event to calculate the time a specific process step requires. +The Task Duration processor calculates the time duration between state changes in a task field. It supports: +* Task state tracking +* Duration calculation +* Multiple time units +* Process identification +* State transition timing +* Performance measurement + +This processor is essential for: +* Measuring task durations +* Tracking process times +* Analyzing state transitions +* Monitoring performance +* Building metrics +* Optimizing workflows + *** ## Required input -A timestamp value is required and a field containing a task value. +The processor requires a data stream containing: +* A task field that changes to indicate different states +* A timestamp field for duration calculation *** ## Configuration -(no further configuration required) +### Task Field +Select the field that contains the task state. The processor will calculate the duration between changes in this field. + +### Timestamp Field +Select the field containing the timestamp for duration calculation. + +### Output Unit +Choose the time unit for the duration output: +* Milliseconds (default) +* Seconds +* Minutes ## Output -Emits an event that contains the process step, built from the names of the first task identifier and the identifier - of the subsequent task. In addition, the duration is part of the output event, provided in milliseconds. \ No newline at end of file +The processor creates a new event containing: +* A processId field showing the state transition (format: "previousState-newState") +* A duration field showing the time between state changes in the selected unit + +### Example + +#### Input Event +```json +{ + "deviceId": "machine01", + "timestamp": 1586380104915, + "taskState": "running" +} +``` + +#### Configuration +* Task Field: taskState +* Timestamp Field: timestamp +* Output Unit: Seconds + +#### Output Event (when taskState changes from "running" to "completed") +```json +{ + "processId": "running-completed", + "duration": 120.5 +} +``` + +## Use Cases + +1. **Process Timing** + * Measure task durations + * Track state transitions + * Analyze process times + * Monitor performance + * Build metrics + +2. **Workflow Analysis** + * Measure step durations + * Track transitions + * Analyze workflows + * Monitor efficiency + * Build analytics + +3. **Performance Monitoring** + * Measure state durations + * Track changes + * Analyze timing + * Monitor efficiency + * Build reports + +4. **Quality Control** + * Measure cycle times + * Track states + * Analyze durations + * Monitor quality + * Build controls + +## Notes + +* Task field must change to trigger output +* Duration is calculated between state changes +* Multiple time units supported +* Processing is stateful +* ProcessId shows state transition +* Consider time unit selection +* No delay in processing +* Original values not preserved +* Only outputs on state change +* Duration is always positive \ No newline at end of file diff --git a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.transform-to-boolean/documentation.md b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.transform-to-boolean/documentation.md index a889be6ce5..81d698bbce 100644 --- a/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.transform-to-boolean/documentation.md +++ b/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/resources/org.apache.streampipes.processors.transformation.jvm.transform-to-boolean/documentation.md @@ -26,22 +26,83 @@ ## Description -This processors transforms numbers and strings to boolean values. +The Transform to Boolean processor converts string or number fields into boolean values. It supports: +* String to boolean conversion +* Number to boolean conversion +* Multiple field transformation +* In-place value modification +This processor is essential for: +* Converting data types +* Creating boolean flags +* Transforming values +* Standardizing data *** ## Required input -A string with the values "true", "True", "false", "False" or a number with value 1.0, 1, 0, or 0.0 +The processor requires a data stream containing at least one string or number field to transform into a boolean. *** ## Configuration -Select fields that should be converted to boolean. +### Transform Fields + +Select one or more string or number fields to transform into boolean values. The transformation rules are: +* Strings: "true" or "1" becomes true, "false" or "0" becomes false +* Numbers: 1 or 1.0 becomes true, 0 or 0.0 becomes false ## Output -Selected properties of input events are transformed to booleans. -When the value is not valid an error message is logged and the event is discarde. \ No newline at end of file +The processor creates a new event containing: +* All original fields from the input event +* The selected fields with their values transformed to boolean + +### Example + +#### Input Event +```json +{ + "deviceId": "sensor01", + "status": "true", + "value": 1, + "timestamp": 1586380104915 +} +``` + +#### Configuration +* Transform Fields: status, value + +#### Output Event +```json +{ + "deviceId": "sensor01", + "status": true, + "value": true, + "timestamp": 1586380104915 +} +``` + +## Use Cases + +1. **Data Standardization** + * Convert string states + * Transform numeric flags + * Standardize values + * Create boolean flags + +2. **Condition Creation** + * Create boolean conditions + * Transform thresholds + * Convert states + * Build flags + +## Notes + +* Only string and number fields can be transformed +* String comparison is case-insensitive +* Numbers use 1/0 logic +* Processing is stateless +* Multiple fields can be transformed \ No newline at end of file
