Re: [DISCUSS] MiNiFi C++ 1.0.0-M1 release
As there seems to be no objections, I will move forward with the release with the new 0.99.0 version proposal. BR, Gabor On Thu, 25 Apr 2024 at 15:41, Marton Szasz wrote: > There is a limitation of CMake that it doesn't allow anything other than > numbers in the version. MiNiFi C++ uses CMake as its build system > generator, so this affects us. The initial idea was to go with 1.0.0 > internally, and only specify the -M1 suffix in the package name, git > tag, etc. > > We had a conversation with the c++ agent contributors, where I proposed > to go with 0.99.0 instead, which still signifies that this is "close to > 1.0", but it may avoid confusion when someone looks at the internal > version metadata. Not sure if it shows up anywhere on Linux, but on > Windows, this version is visible in 'Programs and Features', and in the > PE metadata of our binaries (minifi.exe, extension DLLs), where users > would only see 1.0.0, even for the milestone releases, if we stayed with > 1.0.0 as the CMake-internal version. > > I want to propose to the wider community as well to go with 0.99.0 as > the first MiNiFi C++ milestone release version for 1.0, both internally > in CMake and externally in the package name, git tag, and anywhere else > where we show version. We can continue with 0.99.1 and so on for future > pre-1.0 milestone releases. We already agreed to this change in a > private conversation with the MiNiFi C++ core contributors, so I'll go > ahead and change the version in Jira, etc, unless anyone objects. > > Thanks, > Marton > > On 4/18/24 1:14 PM, Arpad Boda wrote: > > +1 and thanks for RMing! > > > > The python support in this new AI era is great! > > > > On Thu, Apr 18, 2024 at 1:01 PM Marton Szasz wrote: > > > >> +1, thanks for RMing. > >> > >> I think we should follow the progression of the new NiFi Python API with > >> the MiNiFi C++ 1.0 milestone releases, and when NiFi 2.0 is out, we > >> could release MiNiFi C++ 1.0, and commit to backwards-compatibility to > >> this API. > >> > >> I'd exclude the C++ API, and the old MiNiFi C++-style Python and Lua > >> scripting APIs out of the backwards compatibility commitment. > >> > >> Thanks, > >> Marton > >> > >> On 4/16/24 6:41 PM, Pierre Villard wrote: > >>> I'm a +1 with this approach. Being able to use the Python extensions in > >>> MiNiFi cpp is great! > >>> > >>> Thanks, > >>> Pierre > >>> > >>> Le mar. 16 avr. 2024 à 18:39, Gábor Gyimesi a > >> écrit : > Hi community, > > I'd like to initiate a discussion about the next release of MiNiFi > C++. > >> The > last release was more than seven months ago, and there have been many > >> new > features, bug fixes and stability improvements committed to the > >> development > branch since then: 107 tickets closed, over 122 commits as of today. > > I would be happy to take on RM duties for this release. > > Notable features and improvements since the 0.15.0 release: > > New notable features: > - Added support for using NiFi 2.0 Python processors in MiNiFi C++ > - This also includes several improvements to the previous MiNiFi > >> style > python processors, like additional property options, custom > >> relationships > and virtualenv support > - Added new python based multiplatform bootstrap script > - Added encryption support for sensitive properties in flow > >> configuration > - Releasing Windows installer now can be done (and will be done) under > >> the > Apache license > - Added support for service installation on MacOS > - Added C2 debug command to MiNiFi Controller > - Added support for setting MiNiFi properties from command line > - Added system load average field to C2 and Prometheus metrics > - Added support for manually configuring RocksDB options > - Added custom delimiter property for ListenTCP processor > - Added bandwidth limit properties to InvokeHTTP processor > - Added JSON flow configuration examples > > New processors: > - Added PutSmb, FetchSmb and ListSmb processor for SMB networking > >> protocol > support > - Added PushGrafanaLokiGrpc and PushGrafanaLokiREST processors for > >> pushing > logs to Grafana Loki > - Added JoltTransform to use Jolt JSON transformations > - Added SplitText processor > - Added AttributeRollingWindow processor > > Changes and improvements: > - Dropped support for disabling peer verification in InvokeHTTP > - Corrupt flow files are now filtered to avoid errors in the flow > - Using administrative yield duration instead of onschedule retry > >> interval > in scheduling adjusting to NiFi's functionality > - Fixed high disk IO usage issue with MergeContent > - Fixed the site-to-site transfer or large files > - Fixed memory leak caused by unused loggers > - Fixed yielding processors to still respect scheduling period > >
Re: [DISCUSS] MiNiFi C++ 1.0.0-M1 release
There is a limitation of CMake that it doesn't allow anything other than numbers in the version. MiNiFi C++ uses CMake as its build system generator, so this affects us. The initial idea was to go with 1.0.0 internally, and only specify the -M1 suffix in the package name, git tag, etc. We had a conversation with the c++ agent contributors, where I proposed to go with 0.99.0 instead, which still signifies that this is "close to 1.0", but it may avoid confusion when someone looks at the internal version metadata. Not sure if it shows up anywhere on Linux, but on Windows, this version is visible in 'Programs and Features', and in the PE metadata of our binaries (minifi.exe, extension DLLs), where users would only see 1.0.0, even for the milestone releases, if we stayed with 1.0.0 as the CMake-internal version. I want to propose to the wider community as well to go with 0.99.0 as the first MiNiFi C++ milestone release version for 1.0, both internally in CMake and externally in the package name, git tag, and anywhere else where we show version. We can continue with 0.99.1 and so on for future pre-1.0 milestone releases. We already agreed to this change in a private conversation with the MiNiFi C++ core contributors, so I'll go ahead and change the version in Jira, etc, unless anyone objects. Thanks, Marton On 4/18/24 1:14 PM, Arpad Boda wrote: +1 and thanks for RMing! The python support in this new AI era is great! On Thu, Apr 18, 2024 at 1:01 PM Marton Szasz wrote: +1, thanks for RMing. I think we should follow the progression of the new NiFi Python API with the MiNiFi C++ 1.0 milestone releases, and when NiFi 2.0 is out, we could release MiNiFi C++ 1.0, and commit to backwards-compatibility to this API. I'd exclude the C++ API, and the old MiNiFi C++-style Python and Lua scripting APIs out of the backwards compatibility commitment. Thanks, Marton On 4/16/24 6:41 PM, Pierre Villard wrote: I'm a +1 with this approach. Being able to use the Python extensions in MiNiFi cpp is great! Thanks, Pierre Le mar. 16 avr. 2024 à 18:39, Gábor Gyimesi a écrit : Hi community, I'd like to initiate a discussion about the next release of MiNiFi C++. The last release was more than seven months ago, and there have been many new features, bug fixes and stability improvements committed to the development branch since then: 107 tickets closed, over 122 commits as of today. I would be happy to take on RM duties for this release. Notable features and improvements since the 0.15.0 release: New notable features: - Added support for using NiFi 2.0 Python processors in MiNiFi C++ - This also includes several improvements to the previous MiNiFi style python processors, like additional property options, custom relationships and virtualenv support - Added new python based multiplatform bootstrap script - Added encryption support for sensitive properties in flow configuration - Releasing Windows installer now can be done (and will be done) under the Apache license - Added support for service installation on MacOS - Added C2 debug command to MiNiFi Controller - Added support for setting MiNiFi properties from command line - Added system load average field to C2 and Prometheus metrics - Added support for manually configuring RocksDB options - Added custom delimiter property for ListenTCP processor - Added bandwidth limit properties to InvokeHTTP processor - Added JSON flow configuration examples New processors: - Added PutSmb, FetchSmb and ListSmb processor for SMB networking protocol support - Added PushGrafanaLokiGrpc and PushGrafanaLokiREST processors for pushing logs to Grafana Loki - Added JoltTransform to use Jolt JSON transformations - Added SplitText processor - Added AttributeRollingWindow processor Changes and improvements: - Dropped support for disabling peer verification in InvokeHTTP - Corrupt flow files are now filtered to avoid errors in the flow - Using administrative yield duration instead of onschedule retry interval in scheduling adjusting to NiFi's functionality - Fixed high disk IO usage issue with MergeContent - Fixed the site-to-site transfer or large files - Fixed memory leak caused by unused loggers - Fixed yielding processors to still respect scheduling period Upgraded dependencies: - Upgraded OpenSSL to version 3.3.0 - Upgraded AWS SDK to version 1.11.219 with support for new AWS regions - Upgraded libuvc to version 0.0.7 - Upgraded docker base image to alpine:3.18 - Upgraded Sol2 to version 3.3.0 With the current maturity level of the project and with the support for NiFi 2.0 style python processors and json flow configuration, I suggest releasing it as version 1.0.0-M1 milestone release. Do you agree it is time for a new release? Do you agree with the suggested version? Are there any blockers that we should definitely include in this release? Thanks, Gabor
Re: [DISCUSS] MiNiFi C++ 1.0.0-M1 release
+1 and thanks for RMing! The python support in this new AI era is great! On Thu, Apr 18, 2024 at 1:01 PM Marton Szasz wrote: > +1, thanks for RMing. > > I think we should follow the progression of the new NiFi Python API with > the MiNiFi C++ 1.0 milestone releases, and when NiFi 2.0 is out, we > could release MiNiFi C++ 1.0, and commit to backwards-compatibility to > this API. > > I'd exclude the C++ API, and the old MiNiFi C++-style Python and Lua > scripting APIs out of the backwards compatibility commitment. > > Thanks, > Marton > > On 4/16/24 6:41 PM, Pierre Villard wrote: > > I'm a +1 with this approach. Being able to use the Python extensions in > > MiNiFi cpp is great! > > > > Thanks, > > Pierre > > > > Le mar. 16 avr. 2024 à 18:39, Gábor Gyimesi a > écrit : > > > >> Hi community, > >> > >> I'd like to initiate a discussion about the next release of MiNiFi C++. > The > >> last release was more than seven months ago, and there have been many > new > >> features, bug fixes and stability improvements committed to the > development > >> branch since then: 107 tickets closed, over 122 commits as of today. > >> > >> I would be happy to take on RM duties for this release. > >> > >> Notable features and improvements since the 0.15.0 release: > >> > >> New notable features: > >> - Added support for using NiFi 2.0 Python processors in MiNiFi C++ > >>- This also includes several improvements to the previous MiNiFi > style > >> python processors, like additional property options, custom > relationships > >> and virtualenv support > >> - Added new python based multiplatform bootstrap script > >> - Added encryption support for sensitive properties in flow > configuration > >> - Releasing Windows installer now can be done (and will be done) under > the > >> Apache license > >> - Added support for service installation on MacOS > >> - Added C2 debug command to MiNiFi Controller > >> - Added support for setting MiNiFi properties from command line > >> - Added system load average field to C2 and Prometheus metrics > >> - Added support for manually configuring RocksDB options > >> - Added custom delimiter property for ListenTCP processor > >> - Added bandwidth limit properties to InvokeHTTP processor > >> - Added JSON flow configuration examples > >> > >> New processors: > >> - Added PutSmb, FetchSmb and ListSmb processor for SMB networking > protocol > >> support > >> - Added PushGrafanaLokiGrpc and PushGrafanaLokiREST processors for > pushing > >> logs to Grafana Loki > >> - Added JoltTransform to use Jolt JSON transformations > >> - Added SplitText processor > >> - Added AttributeRollingWindow processor > >> > >> Changes and improvements: > >> - Dropped support for disabling peer verification in InvokeHTTP > >> - Corrupt flow files are now filtered to avoid errors in the flow > >> - Using administrative yield duration instead of onschedule retry > interval > >> in scheduling adjusting to NiFi's functionality > >> - Fixed high disk IO usage issue with MergeContent > >> - Fixed the site-to-site transfer or large files > >> - Fixed memory leak caused by unused loggers > >> - Fixed yielding processors to still respect scheduling period > >> > >> Upgraded dependencies: > >> - Upgraded OpenSSL to version 3.3.0 > >> - Upgraded AWS SDK to version 1.11.219 with support for new AWS regions > >> - Upgraded libuvc to version 0.0.7 > >> - Upgraded docker base image to alpine:3.18 > >> - Upgraded Sol2 to version 3.3.0 > >> > >> With the current maturity level of the project and with the support for > >> NiFi 2.0 style python processors and json flow configuration, I suggest > >> releasing it as version 1.0.0-M1 milestone release. > >> > >> Do you agree it is time for a new release? Do you agree with the > suggested > >> version? Are there any blockers that we should definitely include in > this > >> release? > >> > >> Thanks, > >> Gabor > >>
Re: [DISCUSS] MiNiFi C++ 1.0.0-M1 release
+1, thanks for RMing. I think we should follow the progression of the new NiFi Python API with the MiNiFi C++ 1.0 milestone releases, and when NiFi 2.0 is out, we could release MiNiFi C++ 1.0, and commit to backwards-compatibility to this API. I'd exclude the C++ API, and the old MiNiFi C++-style Python and Lua scripting APIs out of the backwards compatibility commitment. Thanks, Marton On 4/16/24 6:41 PM, Pierre Villard wrote: I'm a +1 with this approach. Being able to use the Python extensions in MiNiFi cpp is great! Thanks, Pierre Le mar. 16 avr. 2024 à 18:39, Gábor Gyimesi a écrit : Hi community, I'd like to initiate a discussion about the next release of MiNiFi C++. The last release was more than seven months ago, and there have been many new features, bug fixes and stability improvements committed to the development branch since then: 107 tickets closed, over 122 commits as of today. I would be happy to take on RM duties for this release. Notable features and improvements since the 0.15.0 release: New notable features: - Added support for using NiFi 2.0 Python processors in MiNiFi C++ - This also includes several improvements to the previous MiNiFi style python processors, like additional property options, custom relationships and virtualenv support - Added new python based multiplatform bootstrap script - Added encryption support for sensitive properties in flow configuration - Releasing Windows installer now can be done (and will be done) under the Apache license - Added support for service installation on MacOS - Added C2 debug command to MiNiFi Controller - Added support for setting MiNiFi properties from command line - Added system load average field to C2 and Prometheus metrics - Added support for manually configuring RocksDB options - Added custom delimiter property for ListenTCP processor - Added bandwidth limit properties to InvokeHTTP processor - Added JSON flow configuration examples New processors: - Added PutSmb, FetchSmb and ListSmb processor for SMB networking protocol support - Added PushGrafanaLokiGrpc and PushGrafanaLokiREST processors for pushing logs to Grafana Loki - Added JoltTransform to use Jolt JSON transformations - Added SplitText processor - Added AttributeRollingWindow processor Changes and improvements: - Dropped support for disabling peer verification in InvokeHTTP - Corrupt flow files are now filtered to avoid errors in the flow - Using administrative yield duration instead of onschedule retry interval in scheduling adjusting to NiFi's functionality - Fixed high disk IO usage issue with MergeContent - Fixed the site-to-site transfer or large files - Fixed memory leak caused by unused loggers - Fixed yielding processors to still respect scheduling period Upgraded dependencies: - Upgraded OpenSSL to version 3.3.0 - Upgraded AWS SDK to version 1.11.219 with support for new AWS regions - Upgraded libuvc to version 0.0.7 - Upgraded docker base image to alpine:3.18 - Upgraded Sol2 to version 3.3.0 With the current maturity level of the project and with the support for NiFi 2.0 style python processors and json flow configuration, I suggest releasing it as version 1.0.0-M1 milestone release. Do you agree it is time for a new release? Do you agree with the suggested version? Are there any blockers that we should definitely include in this release? Thanks, Gabor
Re: [DISCUSS] MiNiFi C++ 1.0.0-M1 release
I'm a +1 with this approach. Being able to use the Python extensions in MiNiFi cpp is great! Thanks, Pierre Le mar. 16 avr. 2024 à 18:39, Gábor Gyimesi a écrit : > Hi community, > > I'd like to initiate a discussion about the next release of MiNiFi C++. The > last release was more than seven months ago, and there have been many new > features, bug fixes and stability improvements committed to the development > branch since then: 107 tickets closed, over 122 commits as of today. > > I would be happy to take on RM duties for this release. > > Notable features and improvements since the 0.15.0 release: > > New notable features: > - Added support for using NiFi 2.0 Python processors in MiNiFi C++ > - This also includes several improvements to the previous MiNiFi style > python processors, like additional property options, custom relationships > and virtualenv support > - Added new python based multiplatform bootstrap script > - Added encryption support for sensitive properties in flow configuration > - Releasing Windows installer now can be done (and will be done) under the > Apache license > - Added support for service installation on MacOS > - Added C2 debug command to MiNiFi Controller > - Added support for setting MiNiFi properties from command line > - Added system load average field to C2 and Prometheus metrics > - Added support for manually configuring RocksDB options > - Added custom delimiter property for ListenTCP processor > - Added bandwidth limit properties to InvokeHTTP processor > - Added JSON flow configuration examples > > New processors: > - Added PutSmb, FetchSmb and ListSmb processor for SMB networking protocol > support > - Added PushGrafanaLokiGrpc and PushGrafanaLokiREST processors for pushing > logs to Grafana Loki > - Added JoltTransform to use Jolt JSON transformations > - Added SplitText processor > - Added AttributeRollingWindow processor > > Changes and improvements: > - Dropped support for disabling peer verification in InvokeHTTP > - Corrupt flow files are now filtered to avoid errors in the flow > - Using administrative yield duration instead of onschedule retry interval > in scheduling adjusting to NiFi's functionality > - Fixed high disk IO usage issue with MergeContent > - Fixed the site-to-site transfer or large files > - Fixed memory leak caused by unused loggers > - Fixed yielding processors to still respect scheduling period > > Upgraded dependencies: > - Upgraded OpenSSL to version 3.3.0 > - Upgraded AWS SDK to version 1.11.219 with support for new AWS regions > - Upgraded libuvc to version 0.0.7 > - Upgraded docker base image to alpine:3.18 > - Upgraded Sol2 to version 3.3.0 > > With the current maturity level of the project and with the support for > NiFi 2.0 style python processors and json flow configuration, I suggest > releasing it as version 1.0.0-M1 milestone release. > > Do you agree it is time for a new release? Do you agree with the suggested > version? Are there any blockers that we should definitely include in this > release? > > Thanks, > Gabor >
[DISCUSS] MiNiFi C++ 1.0.0-M1 release
Hi community, I'd like to initiate a discussion about the next release of MiNiFi C++. The last release was more than seven months ago, and there have been many new features, bug fixes and stability improvements committed to the development branch since then: 107 tickets closed, over 122 commits as of today. I would be happy to take on RM duties for this release. Notable features and improvements since the 0.15.0 release: New notable features: - Added support for using NiFi 2.0 Python processors in MiNiFi C++ - This also includes several improvements to the previous MiNiFi style python processors, like additional property options, custom relationships and virtualenv support - Added new python based multiplatform bootstrap script - Added encryption support for sensitive properties in flow configuration - Releasing Windows installer now can be done (and will be done) under the Apache license - Added support for service installation on MacOS - Added C2 debug command to MiNiFi Controller - Added support for setting MiNiFi properties from command line - Added system load average field to C2 and Prometheus metrics - Added support for manually configuring RocksDB options - Added custom delimiter property for ListenTCP processor - Added bandwidth limit properties to InvokeHTTP processor - Added JSON flow configuration examples New processors: - Added PutSmb, FetchSmb and ListSmb processor for SMB networking protocol support - Added PushGrafanaLokiGrpc and PushGrafanaLokiREST processors for pushing logs to Grafana Loki - Added JoltTransform to use Jolt JSON transformations - Added SplitText processor - Added AttributeRollingWindow processor Changes and improvements: - Dropped support for disabling peer verification in InvokeHTTP - Corrupt flow files are now filtered to avoid errors in the flow - Using administrative yield duration instead of onschedule retry interval in scheduling adjusting to NiFi's functionality - Fixed high disk IO usage issue with MergeContent - Fixed the site-to-site transfer or large files - Fixed memory leak caused by unused loggers - Fixed yielding processors to still respect scheduling period Upgraded dependencies: - Upgraded OpenSSL to version 3.3.0 - Upgraded AWS SDK to version 1.11.219 with support for new AWS regions - Upgraded libuvc to version 0.0.7 - Upgraded docker base image to alpine:3.18 - Upgraded Sol2 to version 3.3.0 With the current maturity level of the project and with the support for NiFi 2.0 style python processors and json flow configuration, I suggest releasing it as version 1.0.0-M1 milestone release. Do you agree it is time for a new release? Do you agree with the suggested version? Are there any blockers that we should definitely include in this release? Thanks, Gabor
