This is an automated email from the ASF dual-hosted git repository.
aichrist pushed a change to branch analytics-framework
in repository https://gitbox.apache.org/repos/asf/nifi.git.
omit e9986c2 NIFI-6585 - Refactored tests to use mocked models and extract
functions. Added check in ConnectionStatusAnalytics to confirm expected model
by type
omit 7d6e1b5 NIFI-6566 - Refactor to decouple model instance from status
analytics object. Also allow configurable model from nifi.properties
omit 714b3f6 NIFI-6510 Added property to nifi.properties - Prediction
Interval for connection status analytics (#11)
omit 586185e NIFI-6510 Adjusted interval and incorporated R-squared check
omit 0c9e8ed NIFI-6510 Fixed checkstyle issue in
TestConnectionStatusAnalytics
omit 9cc3404 NIFI-6150 Fixed NaN check and refactored time prediction.
Switched to use non caching engine for testing
omit 34e6191 NIFI-6150 Make checkstyle happy
omit ecce279 NIFI-6150 Added tests for connection status analytics class,
corrected variable names
omit 0bdedd5 NIFI-6510 Added tests for engine
omit 6c5eda1 NIFI-6510 Changes to inject flowManager instead of flow
controller, also changes to properly reflect when predictions can be made vs
not.
omit 8a48c77 NIFI-6510 Add prediction percent values and predicted
interval seconds
omit f3824a0 NIFI-6510 Analytics Framework Introduction (#10)
omit 6a45d01 NIFI-6510 Added prediction fields for use by UI, still need
to be populated
omit 7adaa24 NIFI-6510 Split StatusAnalytics interface into Engine and
per-Connection versions
omit d1a9a92 NIFI-6510 Revert "DFA-9 Remove redundant connection
prediction interfaces as we can just use ConnectionStatusAnalytics directly"
omit af7becb NIFI-6510 Remove redundant connection prediction interfaces
as we can just use ConnectionStatusAnalytics directly
omit 2bcf3db NIFI-6510 adjustments for interface updates, added call to
StandardEventAccess, updated interface to use connection id
omit 7252413 NIFI-6510 Updated objects and interfaces to reflect 4
prediction metrics
omit bd011f3 NIFI-6510 Added poc engine with prediction model caching
omit be0993c NIFI-6510 Connect the dots for StatusAnalytics -> API
omit 42b50e8 NIFI-6510 Initial analytics REST endpoint and supporting
objects
omit cfc3fdc NIFI-6510 Implemented basic linear regression model for queue
counts
omit c4cc6c9 NIFI-6510 Implement initial analytic engine
add 16f3dbd NIFI-6364 - Add CLI commands for parameter contexts
add e32689a NIFI-6436 Fix NPE at StandardPublicPort Removing unnecessary
ProcessGroup from PublicPort constructor.
add 326bc91 NIFI-5839: - Introducing case insensitive group membership. -
Ensuring user identity and group name mapping is applied when the user identity
or group name is inferred through group membership decisions.
add 81136bf NIFI-6502 RecordPath padding functions: - PadLeft(label,
desiredLength, paddingChar) prepends the paddingChar (or the default value '_'
if a paddingChar is not provided) to the label string until desiredLength is
reached. - PadRight(label, desiredLength, paddingChar) appends the paddingChar
(or the default value '_' if a paddingChar is not provided) to the label string
until desiredLength is reached. Added Apache license disclaimers checkstyle
fixes Replaced functional i [...]
add e822a16 NIFI-6572 Added a workaround for GetMongoRecordIT that fixes
a weird issue with Groovy where the Groovy addons don't appear to be getting
recognized for java.util.Date.
add 1cb1b00 NIFI-6477 Delete operate policy when its component is deleted
add 00f0f84 NIFI-6508 - Fix test failure caused by timezone and/or DST
difference
add 0b74795 NIFI-6562 Expose ExecutionNode in ProcessContext
add bb9758b NIFI-6500 (Original commit messages left to preserve
authorship credit for multiple contributors)
add cc88dd4 NIFI-6158 Fix conversion of Avro fixed type with logicalType
decimal
add 3de81d6 NIFI-6509 - Date related issue in unit test
VolatileComponentStatusRepositoryTest
new 4fbdb48 NIFI-6510 Implement initial analytic engine
new 2d8c70f NIFI-6510 Implemented basic linear regression model for queue
counts
new e1f5296 NIFI-6510 Initial analytics REST endpoint and supporting
objects
new a4cab9c NIFI-6510 Connect the dots for StatusAnalytics -> API
new 5ef715e NIFI-6510 Added poc engine with prediction model caching
new b90430a NIFI-6510 Updated objects and interfaces to reflect 4
prediction metrics
new 0d07835 NIFI-6510 adjustments for interface updates, added call to
StandardEventAccess, updated interface to use connection id
new bc996e3 NIFI-6510 Remove redundant connection prediction interfaces
as we can just use ConnectionStatusAnalytics directly
new 9761ee1 NIFI-6510 Revert "DFA-9 Remove redundant connection
prediction interfaces as we can just use ConnectionStatusAnalytics directly"
new 7a8d13b NIFI-6510 Split StatusAnalytics interface into Engine and
per-Connection versions
new 84d981d NIFI-6510 Added prediction fields for use by UI, still need
to be populated
new e46dd37 NIFI-6510 Analytics Framework Introduction (#10)
new bad5c11 NIFI-6510 Add prediction percent values and predicted
interval seconds
new c98ae35 NIFI-6510 Changes to inject flowManager instead of flow
controller, also changes to properly reflect when predictions can be made vs
not.
new 59147a3 NIFI-6510 Added tests for engine
new 6ea3312 NIFI-6150 Added tests for connection status analytics class,
corrected variable names
new d658441 NIFI-6150 Make checkstyle happy
new b12f9d1 NIFI-6150 Fixed NaN check and refactored time prediction.
Switched to use non caching engine for testing
new 94b120d NIFI-6510 Fixed checkstyle issue in
TestConnectionStatusAnalytics
new f093f48 NIFI-6510 Adjusted interval and incorporated R-squared check
new 6736574 NIFI-6510 Added property to nifi.properties - Prediction
Interval for connection status analytics (#11)
new 8810f9c NIFI-6566 - Refactor to decouple model instance from status
analytics object. Also allow configurable model from nifi.properties
new dbbd41b NIFI-6585 - Refactored tests to use mocked models and extract
functions. Added check in ConnectionStatusAnalytics to confirm expected model
by type
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (e9986c2)
\
N -- N -- N refs/heads/analytics-framework (dbbd41b)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 23 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
NOTICE | 5 +
.../org/apache/nifi/processor/ProcessContext.java | 6 +
nifi-assembly/NOTICE | 5 +
.../language/antlr/AttributeExpressionLexer.g | 2 +
.../language/antlr/AttributeExpressionParser.g | 2 +-
.../language/compile/ExpressionCompiler.java | 26 ++
.../evaluation/functions/PadLeftEvaluator.java | 37 ++
.../evaluation/functions/PadRightEvaluator.java | 36 ++
.../evaluation/functions/PaddingEvaluator.java | 68 ++++
.../attribute/expression/language/TestQuery.java | 42 +++
.../java/org/apache/nifi/util/StringUtils.java | 398 ++++++++++++++++++++-
.../java/org/apache/nifi/util/StringUtilsTest.java | 29 +-
.../apache/nifi/record/path/functions/PadLeft.java | 42 +++
.../nifi/record/path/functions/PadRight.java | 42 +++
.../apache/nifi/record/path/functions/Padding.java | 106 ++++++
.../nifi/record/path/paths/RecordPathCompiler.java | 24 ++
.../apache/nifi/record/path/TestRecordPath.java | 51 +++
.../src/main/asciidoc/administration-guide.adoc | 6 +-
.../main/asciidoc/expression-language-guide.adoc | 70 ++++
nifi-docs/src/main/asciidoc/record-path-guide.adoc | 61 ++++
.../org/apache/nifi/util/MockProcessContext.java | 6 +
.../java/org/apache/nifi/avro/AvroTypeUtil.java | 5 +
.../org/apache/nifi/avro/TestAvroTypeUtil.java | 34 ++
.../org/apache/nifi/controller/AbstractPort.java | 5 +-
.../org/apache/nifi/remote/RemoteGroupPort.java | 5 +-
.../org/apache/nifi/connectable/LocalPort.java | 2 +-
.../nifi/controller/flow/StandardFlowManager.java | 4 +-
.../scheduling/ConnectableProcessContext.java | 6 +
.../nifi/processor/StandardProcessContext.java | 7 +
.../nifi/remote/StandardRemoteProcessGroup.java | 6 +-
.../VolatileComponentStatusRepositoryTest.java | 149 ++++----
.../org/apache/nifi/mock/MockProcessContext.java | 6 +
.../nifi-framework/nifi-properties-loader/pom.xml | 4 -
.../src/main/resources/conf/authorizers.xml | 4 +
.../org/apache/nifi/remote/StandardPublicPort.java | 13 +-
.../nifi/remote/StandardRemoteGroupPort.java | 5 +-
.../apache/nifi/remote/TestStandardPublicPort.java | 8 +-
.../nifi/remote/TestStandardRemoteGroupPort.java | 2 +-
.../apache/nifi/web/StandardNiFiServiceFacade.java | 1 +
.../nifi/ldap/tenants/LdapUserGroupProvider.java | 45 ++-
.../ldap/tenants/LdapUserGroupProviderTest.java | 119 +++++-
.../src/test/resources/nifi-example.ldif | 8 +
.../processors/mongodb/GetMongoRecordIT.groovy | 4 +-
.../nifi-parquet-processors/pom.xml | 7 +
.../nifi/processors/parquet/FetchParquetTest.java | 108 ++++--
.../resources/avro/user-with-fixed-decimal.avsc | 8 +
.../stateless/core/StatelessProcessContext.java | 6 +
.../toolkit/cli/impl/client/NiFiClientFactory.java | 16 +
.../toolkit/cli/impl/client/nifi/NiFiClient.java | 8 +
.../cli/impl/client/nifi/ParamContextClient.java | 41 +++
.../impl/client/nifi/impl/JerseyNiFiClient.java | 18 +
.../client/nifi/impl/JerseyParamContextClient.java | 157 ++++++++
.../toolkit/cli/impl/command/AbstractCommand.java | 37 ++
.../toolkit/cli/impl/command/CommandOption.java | 13 +
.../toolkit/cli/impl/command/CommandProcessor.java | 1 +
.../cli/impl/command/nifi/NiFiCommandGroup.java | 22 ++
.../params/AbstractUpdateParamContextCommand.java | 84 +++++
.../command/nifi/params/CreateParamContext.java | 78 ++++
.../cli/impl/command/nifi/params/DeleteParam.java | 106 ++++++
.../command/nifi/params/DeleteParamContext.java | 63 ++++
.../command/nifi/params/ExportParamContext.java | 130 +++++++
.../impl/command/nifi/params/GetParamContext.java | 57 +++
.../command/nifi/params/ImportParamContext.java | 93 +++++
.../command/nifi/params/ListParamContexts.java | 50 +++
.../command/nifi/params/MergeParamContext.java | 136 +++++++
.../cli/impl/command/nifi/params/SetParam.java | 129 +++++++
.../impl/command/nifi/pg/PGGetParamContext.java | 81 +++++
.../impl/command/nifi/pg/PGSetParamContext.java | 79 ++++
.../command/registry/flow/ImportFlowVersion.java | 17 +-
.../cli/impl/result/nifi/ParamContextResult.java | 82 +++++
.../cli/impl/result/nifi/ParamContextsResult.java | 118 ++++++
pom.xml | 1 +
72 files changed, 3049 insertions(+), 203 deletions(-)
create mode 100644
nifi-commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/PadLeftEvaluator.java
create mode 100644
nifi-commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/PadRightEvaluator.java
create mode 100644
nifi-commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/PaddingEvaluator.java
create mode 100644
nifi-commons/nifi-record-path/src/main/java/org/apache/nifi/record/path/functions/PadLeft.java
create mode 100644
nifi-commons/nifi-record-path/src/main/java/org/apache/nifi/record/path/functions/PadRight.java
create mode 100644
nifi-commons/nifi-record-path/src/main/java/org/apache/nifi/record/path/functions/Padding.java
create mode 100644
nifi-nar-bundles/nifi-parquet-bundle/nifi-parquet-processors/src/test/resources/avro/user-with-fixed-decimal.avsc
create mode 100644
nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/ParamContextClient.java
create mode 100644
nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyParamContextClient.java
create mode 100644
nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/command/nifi/params/AbstractUpdateParamContextCommand.java
create mode 100644
nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/command/nifi/params/CreateParamContext.java
create mode 100644
nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/command/nifi/params/DeleteParam.java
create mode 100644
nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/command/nifi/params/DeleteParamContext.java
create mode 100644
nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/command/nifi/params/ExportParamContext.java
create mode 100644
nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/command/nifi/params/GetParamContext.java
create mode 100644
nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/command/nifi/params/ImportParamContext.java
create mode 100644
nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/command/nifi/params/ListParamContexts.java
create mode 100644
nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/command/nifi/params/MergeParamContext.java
create mode 100644
nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/command/nifi/params/SetParam.java
create mode 100644
nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/command/nifi/pg/PGGetParamContext.java
create mode 100644
nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/command/nifi/pg/PGSetParamContext.java
create mode 100644
nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/result/nifi/ParamContextResult.java
create mode 100644
nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/result/nifi/ParamContextsResult.java
