This is an automated email from the ASF dual-hosted git repository. zehnder pushed a commit to branch blog-rel-0_97_0 in repository https://gitbox.apache.org/repos/asf/streampipes-website.git
commit 619f7cc56dd19f2f86f186fc39c24d58b459f2c4 Author: Philipp Zehnder <[email protected]> AuthorDate: Mon Feb 3 21:04:47 2025 +0100 [hotfix] Add first draft for release blog post --- website-v2/blog/2025-02-15-release-097.md | 176 +++++++++++++++++++++ .../static/img/blog/2025-02-15/asset_filter.png | Bin 0 -> 270753 bytes .../2025-02-15/data_explorer_configuration.png | Bin 0 -> 410031 bytes .../img/blog/2025-02-15/deprecated_dashboards.png | Bin 0 -> 152894 bytes .../static/img/blog/2025-02-15/file_upload.png | Bin 0 -> 151427 bytes .../img/blog/2025-02-15/kafka_configuration.png | Bin 0 -> 290008 bytes .../img/blog/2025-02-15/opc_ua_configuration.png | Bin 0 -> 250592 bytes .../blog/2025-02-15/start_stop_all_adapters.png | Bin 0 -> 280522 bytes 8 files changed, 176 insertions(+) diff --git a/website-v2/blog/2025-02-15-release-097.md b/website-v2/blog/2025-02-15-release-097.md new file mode 100644 index 000000000..5a531b938 --- /dev/null +++ b/website-v2/blog/2025-02-15-release-097.md @@ -0,0 +1,176 @@ +--- +title: Apache StreamPipes release 0.97.0 +author: Philipp Zehnder +authorURL: "https://github.com/tenthe"; +authorImageURL: "/img/zehnder.png" +--- + +# Apache StreamPipes 0.97.0 Release + +## Introduction +Apache StreamPipes 0.97.0 brings exciting new features, enhanced security, and significant UX improvements to streamline your data processing workflows. +This release introduces **Adapters & Pipelines as Code**, enabling users to define and deploy StreamPipes resources programmatically using YAML or JSON. +Security enhancements, such as **OPC UA certificate support** and **Kafka SSL/SASL authentication**, improve connectivity and data integrity. +Additionally, we have made various **UX improvements**, including an **enhanced Data Explorer**, **asset-based filtering**, and **centralized adapter control**. + +Read on to discover what’s new in StreamPipes 0.97.0! + +## Adapters & Pipeline Elements +### Adapter & Pipeline as Code +StreamPipes now supports managing adapters and pipelines programmatically using compact Json or YAML definitions. +These features are ideal for automating deployments, integrating with CI/CD pipelines, and applying infrastructure-as-code practices. + +#### Adapters as Code + +With Adapters as Code, you can define and deploy adapters via a compact description, streamlining the creation of multiple similar adapters and simplifying automation. +For example, here's a sample YAML snippet to define an OPC-UA adapter: + +```yml +name: My OPC Adapter +id: testadapter +description: Test +appId: org.apache.streampipes.connect.iiot.adapters.opcua +configuration: + - opc_host_or_url: OPC_URL + opc_server_url: opc.tcp://localhost:62541/milo + - adapter_type: PULL_MODE + pulling_interval: 1000 + - access_mode: UNAUTHENTICATED + - available_nodes: + - "ns=2;s=Leakage Test Station/temperature" + - "ns=2;s=Leakage Test Station/pressure" + +schema: + temperature: + propertyScope: measurement + label: Temp + description: Temperature value + semanticType: http://schema.org/temperature + +enrich: + timestamp: timestamp + +transform: + rename: + pressure: pressure3 + +createOptions: + persist: true + start: true +``` + +This approach considerably simplifies the programmatic creation of adapters [Learn more about Adapters as Code](https://streampipes.apache.org/docs/use-programmatically-create-adapters/) + +#### Pipelines as Code + +Pipelines as Code lets you define end-to-end data flows—from data sources through processors to sinks. +This function is ideal for the standardization and automation of pipeline deployments. +For example, here’s a sample YAML snippet for a simple pipeline: + +```yaml +id: my-pipeline +name: "" +description: "" +pipelineElements: # Define pipeline components here + - type: stream # Data source + ref: <reference> # Unique reference ID + id: <data-stream-id> # ID of the stream + + - type: processor # Data transformation + ref: <reference> # Unique reference ID + id: <processor-id> # ID of the processor + connectedTo: # Previous pipeline element reference(s) + - <reference> + configuration: # Processor-specific configurations + - <configuration-option> + + - type: sink # Data sink (output) + ref: <reference> # Unique reference ID + id: <sink-id> # ID of the sink + connectedTo: # Previous pipeline element reference(s) + - <reference> + configuration: # Sink-specific configurations + - <configuration-option> + +createOptions: + start: <true|false> # Whether to start the pipeline immediately +``` + +This method provides a programmatic interface to deploy and manage pipelines quickly and easily. +[Learn more about Pipelines as Code](https://streampipes.apache.org/docs/use-programmatically-create-pipelines/) + + + +### OPC UA Certificates +The OPC UA adapter now supports both signed/encrypted and unencrypted communication. +Certificate provisioning is handled externally via environment variables, following a dedicated security directory layout. Refer to the [OPC UA adapter documentation](https://streampipes.apache.org/docs/pe/org.apache.streampipes.connect.iiot.adapters.opcua/) for further details. + +https://github.com/apache/streampipes/pull/3354 + +<img src="/img/blog/2025-02-15/opc_ua_configuration.png" alt="OPC UA adapter configuration"/> + + +### Kafka Avro & Certificates +The Kafka adapter and sink now support SSL and SASL for enhanced security. +Additionally, users can provide custom producer/consumer configurations. +A new Avro parser has been added along with an input field for specifying the consumer group in the Kafka adapter. + +<img src="/img/blog/2025-02-15/kafka_configuration.png" alt="Kafka adapter configuration"/> + + +## New Features & Deprecations + +### Browse Asset Functionality +StreamPipes now allows users to model real-world assets and associate them with StreamPipes resources such as adapters, pipelines, and dashboards. +These asset mappings enable intuitive filtering across different modules, making it easier to navigate and manage related resources. + +The screenshot below illustrates an example where a StreamPipes instance is configured with two machines and various stations. +Clicking on a machine or station dynamically filters the associated resources in the list on the right. +This asset-based filtering is available across all main modules of StreamPipes. + +<img src="/img/blog/2025-02-15/asset_filter.png" alt="Asset Filtering in StreamPipes"/> + +### Deprecation Notice: Dashboard +The current live dashboard feature in StreamPipes is being deprecated and will be removed in a future release. +As a replacement, the Data Explorer dashboards will gain auto-refresh functionality, ensuring a seamless transition for users who rely on real-time data visualization. + +We encourage all users to migrate their existing dashboards to the Data Explorer to continue leveraging powerful visualization and analysis capabilities. + +<img src="/img/blog/2025-02-15/deprecated_dashboards.png" alt="Dashboard Deprecation Notice"/> + + +## UX improvements +### File Relocation +The file upload configuration has been moved from the top-level navigation to the settings menu, improving the overall structure and clarity of the main navigation. + +This change does not affect file uploads required by adapters or pipeline elements—these configurations remain unchanged. +Further, users can still upload files as before via the files configuration section. + +<img src="/img/blog/2025-02-15/file_upload.png" alt="File upload configuration"/> + +### Adapter Control Enhancements +Managing multiple adapters is now more convenient with the addition of a **Start all adapters** and **Stop all adapters** functionality. +This allows users to quickly control all adapters at once, streamlining workflow and improving efficiency. + +<img src="/img/blog/2025-02-15/start_stop_all_adapters.png" alt="Add buttons to start stop all adapters"/> + +### Data Explorer Configuration Improvements +Configuring data views in the **Data Explorer** is now more intuitive and user-friendly. +We have enhanced validation, making it easier to select data sources and specify queries while reducing potential misconfigurations. + +Key improvements include: +- **Default Table Visualization**: New data views now default to a table view, minimizing setup effort. +- **Improved Time Range Picker**: More flexibility in selecting time ranges, including support for different time ranges per widget in Data Lake dashboards. +- **Auto-Refresh for Dashboards**: Live updates are now possible with the new auto-refresh functionality. +- **Better Error Messages**: Enhanced feedback when configurations are incorrect to improve usability. + +<img src="/img/blog/2025-02-15/data_explorer_configuration.png" alt="Improve configuration for data explorer views"/> + + +## Conclusion +We appreciate the ongoing support and contributions from our community in making StreamPipes better with each release. +Version 0.97.0 brings improvements in usability, security, and automation, helping users to analyze their IIoT data sources. + +As always, we welcome your feedback and suggestions to continue refining and enhancing StreamPipes. +Feel free to share your thoughts or get involved through **GitHub** or **mailing list**. + diff --git a/website-v2/static/img/blog/2025-02-15/asset_filter.png b/website-v2/static/img/blog/2025-02-15/asset_filter.png new file mode 100644 index 000000000..0ef53a76c Binary files /dev/null and b/website-v2/static/img/blog/2025-02-15/asset_filter.png differ diff --git a/website-v2/static/img/blog/2025-02-15/data_explorer_configuration.png b/website-v2/static/img/blog/2025-02-15/data_explorer_configuration.png new file mode 100644 index 000000000..35bda975b Binary files /dev/null and b/website-v2/static/img/blog/2025-02-15/data_explorer_configuration.png differ diff --git a/website-v2/static/img/blog/2025-02-15/deprecated_dashboards.png b/website-v2/static/img/blog/2025-02-15/deprecated_dashboards.png new file mode 100644 index 000000000..90497968d Binary files /dev/null and b/website-v2/static/img/blog/2025-02-15/deprecated_dashboards.png differ diff --git a/website-v2/static/img/blog/2025-02-15/file_upload.png b/website-v2/static/img/blog/2025-02-15/file_upload.png new file mode 100644 index 000000000..bb3636225 Binary files /dev/null and b/website-v2/static/img/blog/2025-02-15/file_upload.png differ diff --git a/website-v2/static/img/blog/2025-02-15/kafka_configuration.png b/website-v2/static/img/blog/2025-02-15/kafka_configuration.png new file mode 100644 index 000000000..b81034a85 Binary files /dev/null and b/website-v2/static/img/blog/2025-02-15/kafka_configuration.png differ diff --git a/website-v2/static/img/blog/2025-02-15/opc_ua_configuration.png b/website-v2/static/img/blog/2025-02-15/opc_ua_configuration.png new file mode 100644 index 000000000..5e07073af Binary files /dev/null and b/website-v2/static/img/blog/2025-02-15/opc_ua_configuration.png differ diff --git a/website-v2/static/img/blog/2025-02-15/start_stop_all_adapters.png b/website-v2/static/img/blog/2025-02-15/start_stop_all_adapters.png new file mode 100644 index 000000000..d47314973 Binary files /dev/null and b/website-v2/static/img/blog/2025-02-15/start_stop_all_adapters.png differ
