[jira] [Closed] (NIFI-9287) nifi.content.repository.directory.default* has (potentially) incorrect formatting in admin guide
[ https://issues.apache.org/jira/browse/NIFI-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alasdair Brown closed NIFI-9287. > nifi.content.repository.directory.default* has (potentially) incorrect > formatting in admin guide > > > Key: NIFI-9287 > URL: https://issues.apache.org/jira/browse/NIFI-9287 > Project: Apache NiFi > Issue Type: Bug > Components: Documentation Website >Affects Versions: 1.14.0 >Reporter: Alasdair Brown >Assignee: Alasdair Brown >Priority: Trivial > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (NIFI-9287) nifi.content.repository.directory.default* has (potentially) incorrect formatting in admin guide
[ https://issues.apache.org/jira/browse/NIFI-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alasdair Brown resolved NIFI-9287. -- Resolution: Invalid Issue only apparent in Asciidoctor preview, not in the rendered html guide > nifi.content.repository.directory.default* has (potentially) incorrect > formatting in admin guide > > > Key: NIFI-9287 > URL: https://issues.apache.org/jira/browse/NIFI-9287 > Project: Apache NiFi > Issue Type: Bug > Components: Documentation Website >Affects Versions: 1.14.0 >Reporter: Alasdair Brown >Assignee: Alasdair Brown >Priority: Trivial > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-9287) nifi.content.repository.directory.default* has (potentially) incorrect formatting in admin guide
[
https://issues.apache.org/jira/browse/NIFI-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alasdair Brown updated NIFI-9287:
-
Description: (was: Some lines contain just a `+` which is used for new
lines, but without a space infront of the +, this isn't always interpreted as a
new line -
{code:java}
|`nifi.content.repository.directory.default`*|The location of the Content
Repository. The default value is `./content_repository`. ++*NOTE*: Multiple
content repositories can be specified by using the
`nifi.content.repository.directory.` prefix with unique suffixes and separate
paths as values. ++For example, to provide two additional locations to act as
part of the content repository, a user could also specify additional properties
with keys of: ++`nifi.content.repository.directory.content1=/repos/content1`
+`nifi.content.repository.directory.content2=/repos/content2` ++Providing three
total locations, including `nifi.content.repository.directory.default`.{code})
> nifi.content.repository.directory.default* has (potentially) incorrect
> formatting in admin guide
>
>
> Key: NIFI-9287
> URL: https://issues.apache.org/jira/browse/NIFI-9287
> Project: Apache NiFi
> Issue Type: Bug
> Components: Documentation Website
>Affects Versions: 1.14.0
>Reporter: Alasdair Brown
>Assignee: Alasdair Brown
>Priority: Trivial
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-9287) nifi.content.repository.directory.default* has (potentially) incorrect formatting in admin guide
[
https://issues.apache.org/jira/browse/NIFI-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alasdair Brown updated NIFI-9287:
-
Summary: nifi.content.repository.directory.default* has (potentially)
incorrect formatting in admin guide (was:
nifi.content.repository.directory.default* has incorrect formatting in admin
guide)
> nifi.content.repository.directory.default* has (potentially) incorrect
> formatting in admin guide
>
>
> Key: NIFI-9287
> URL: https://issues.apache.org/jira/browse/NIFI-9287
> Project: Apache NiFi
> Issue Type: Bug
> Components: Documentation Website
>Affects Versions: 1.14.0
>Reporter: Alasdair Brown
>Assignee: Alasdair Brown
>Priority: Trivial
>
> Some lines contain just a `+` which is used for new lines, but without a
> space infront of the +, this isn't always interpreted as a new line -
>
> {code:java}
> |`nifi.content.repository.directory.default`*|The location of the Content
> Repository. The default value is `./content_repository`. ++*NOTE*: Multiple
> content repositories can be specified by using the
> `nifi.content.repository.directory.` prefix with unique suffixes and separate
> paths as values. ++For example, to provide two additional locations to act as
> part of the content repository, a user could also specify additional
> properties with keys of:
> ++`nifi.content.repository.directory.content1=/repos/content1`
> +`nifi.content.repository.directory.content2=/repos/content2` ++Providing
> three total locations, including
> `nifi.content.repository.directory.default`.{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-9287) nifi.content.repository.directory.default* has incorrect formatting in admin guide
[
https://issues.apache.org/jira/browse/NIFI-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alasdair Brown updated NIFI-9287:
-
Description:
Some lines contain just a `+` which is used for new lines, but without a space
infront of the +, this isn't always interpreted as a new line -
{code:java}
|`nifi.content.repository.directory.default`*|The location of the Content
Repository. The default value is `./content_repository`. ++*NOTE*: Multiple
content repositories can be specified by using the
`nifi.content.repository.directory.` prefix with unique suffixes and separate
paths as values. ++For example, to provide two additional locations to act as
part of the content repository, a user could also specify additional properties
with keys of: ++`nifi.content.repository.directory.content1=/repos/content1`
+`nifi.content.repository.directory.content2=/repos/content2` ++Providing three
total locations, including `nifi.content.repository.directory.default`.{code}
was:
Some lines contain just a `+` which is used for new lines, but without a space
infront of the +, this isn't interpreted as a new line - instead the plus is
rendered as text.
{code:java}
|`nifi.content.repository.directory.default`*|The location of the Content
Repository. The default value is `./content_repository`. ++*NOTE*: Multiple
content repositories can be specified by using the
`nifi.content.repository.directory.` prefix with unique suffixes and separate
paths as values. ++For example, to provide two additional locations to act as
part of the content repository, a user could also specify additional properties
with keys of: ++`nifi.content.repository.directory.content1=/repos/content1`
+`nifi.content.repository.directory.content2=/repos/content2` ++Providing three
total locations, including `nifi.content.repository.directory.default`.{code}
> nifi.content.repository.directory.default* has incorrect formatting in admin
> guide
> --
>
> Key: NIFI-9287
> URL: https://issues.apache.org/jira/browse/NIFI-9287
> Project: Apache NiFi
> Issue Type: Bug
> Components: Documentation Website
>Affects Versions: 1.14.0
>Reporter: Alasdair Brown
>Assignee: Alasdair Brown
>Priority: Trivial
>
> Some lines contain just a `+` which is used for new lines, but without a
> space infront of the +, this isn't always interpreted as a new line -
>
> {code:java}
> |`nifi.content.repository.directory.default`*|The location of the Content
> Repository. The default value is `./content_repository`. ++*NOTE*: Multiple
> content repositories can be specified by using the
> `nifi.content.repository.directory.` prefix with unique suffixes and separate
> paths as values. ++For example, to provide two additional locations to act as
> part of the content repository, a user could also specify additional
> properties with keys of:
> ++`nifi.content.repository.directory.content1=/repos/content1`
> +`nifi.content.repository.directory.content2=/repos/content2` ++Providing
> three total locations, including
> `nifi.content.repository.directory.default`.{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (NIFI-9287) nifi.content.repository.directory.default* has incorrect formatting in admin guide
Alasdair Brown created NIFI-9287:
Summary: nifi.content.repository.directory.default* has incorrect
formatting in admin guide
Key: NIFI-9287
URL: https://issues.apache.org/jira/browse/NIFI-9287
Project: Apache NiFi
Issue Type: Bug
Components: Documentation Website
Affects Versions: 1.14.0
Reporter: Alasdair Brown
Assignee: Alasdair Brown
Some lines contain just a `+` which is used for new lines, but without a space
infront of the +, this isn't interpreted as a new line - instead the plus is
rendered as text.
{code:java}
|`nifi.content.repository.directory.default`*|The location of the Content
Repository. The default value is `./content_repository`. ++*NOTE*: Multiple
content repositories can be specified by using the
`nifi.content.repository.directory.` prefix with unique suffixes and separate
paths as values. ++For example, to provide two additional locations to act as
part of the content repository, a user could also specify additional properties
with keys of: ++`nifi.content.repository.directory.content1=/repos/content1`
+`nifi.content.repository.directory.content2=/repos/content2` ++Providing three
total locations, including `nifi.content.repository.directory.default`.{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (NIFI-9029) Document Missing Properties in the Sys Admin Guide
[ https://issues.apache.org/jira/browse/NIFI-9029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alasdair Brown reassigned NIFI-9029: Assignee: Alasdair Brown > Document Missing Properties in the Sys Admin Guide > -- > > Key: NIFI-9029 > URL: https://issues.apache.org/jira/browse/NIFI-9029 > Project: Apache NiFi > Issue Type: Task > Components: Documentation Website >Affects Versions: 1.13.2 >Reporter: Ryan Hendrickson >Assignee: Alasdair Brown >Priority: Critical > Fix For: 1.15.0 > > > Our team recently upgraded to 1.13.2 from 1.11.4. The performance > improvements caused a number of servers which see quite a bit of data to > perform better. However, we began experiencing "frozen NiFi's". NiFi's that > appear fine, but will not process any data. We determined the file I/O > improvements from the 1.11.4 to 1.13.2 release were significant enough for > us, that we increased the amount of data passing through them to go from the > 50% archive to exceeding the 52% archive backpressure percentage prior to the > archive job completing. > This locked-up all our NiFi's causing us to revert. > How did we figure this out? > We came across a mailing list email outlining the _*undocumented*_ NiFi > Property *nifi.content.repository.archive.backpressure.percentage* causing > the issue. > This is a repeatable issue. Something more should be done to warn the user > on the UI that this has happened. Until then. Here is a ticket outlining > all of the missing properties in the [Sys Admin > Guide|https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html] > as compared to the real properties defined in > [NiFiProperties.java|https://github.com/apache/nifi/blob/main/nifi-commons/nifi-properties/src/main/java/org/apache/nifi/util/NiFiProperties.java]. > > | |*Default*| > |*Missing Core Properties*| | > |RESTORE_DIRECTORY = "nifi.restore.directory";|??| > | | | > |*Missing Content Repository Properties*| | > |MAX_FLOWFILES_PER_CLAIM = "nifi.content.claim.max.flow.files";|100| > |CONTENT_ARCHIVE_BACK_PRESSURE_PERCENTAGE = > "nifi.content.repository.archive.backpressure.percentage";|2%| > |CONTENT_ARCHIVE_CLEANUP_FREQUENCY = > "nifi.content.repository.archive.cleanup.frequency";|1 Second| > | | | > |*Missing Provenance Properties*| | > |PROVENANCE_ROLLOVER_EVENT_COUNT = > "nifi.provenance.repository.rollover.events";|Integer.MAX_VALUE| > |PROVENANCE_REPO_DEBUG_FREQUENCY = > "nifi.provenance.repository.debug.frequency";|1_000_000| > | | | > |*Missing Web Properties*| | > |WEB_MAX_ACCESS_TOKEN_REQUESTS_PER_SECOND = > "nifi.web.max.access.token.requests.per.second";|25| > |WEB_SHOULD_SEND_SERVER_VERSION = "nifi.web.should.send.server.version";|TRUE| -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-9199) Pasting from clipboard should select text/plain where possible
Alasdair Brown created NIFI-9199: Summary: Pasting from clipboard should select text/plain where possible Key: NIFI-9199 URL: https://issues.apache.org/jira/browse/NIFI-9199 Project: Apache NiFi Issue Type: Improvement Affects Versions: 1.14.0 Reporter: Alasdair Brown When copying a cell from Excel, multiple copies of the data are put in the clipboard with different formats. Pasting in to a NiFi param does not necessarily paste plain text, which can leave invisible characters in the param value and cause failures that are very hard to diagnose. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8087) Amazon SNS FIFO Support in PutSNS Processor
[ https://issues.apache.org/jira/browse/NIFI-8087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409964#comment-17409964 ] Alasdair Brown commented on NIFI-8087: -- Raised PR to add MessageGroupID and MessageDuplicationID for initial support of SNS FIFO topics. Tested it against a fresh SNS FIFO topic. > Amazon SNS FIFO Support in PutSNS Processor > --- > > Key: NIFI-8087 > URL: https://issues.apache.org/jira/browse/NIFI-8087 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.12.1 >Reporter: Tim Casada >Assignee: Alasdair Brown >Priority: Minor > Labels: AWS, newbie > Time Spent: 10m > Remaining Estimate: 0h > > Please consider an improvement request to add functionality to the PutSNS > processor to support the October 2020 release of FIFO messaging in AWS SNS. > This would include the capabilities to have attributes for Message Group ID, > Content-Based Deduplication and a Deduplication ID. > This would be similar to NIFI-4540 that was recently resolved for PutSQS FIFO. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (NIFI-8087) Amazon SNS FIFO Support in PutSNS Processor
[ https://issues.apache.org/jira/browse/NIFI-8087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alasdair Brown reassigned NIFI-8087: Assignee: Alasdair Brown > Amazon SNS FIFO Support in PutSNS Processor > --- > > Key: NIFI-8087 > URL: https://issues.apache.org/jira/browse/NIFI-8087 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.12.1 >Reporter: Tim Casada >Assignee: Alasdair Brown >Priority: Minor > Labels: AWS, newbie > Time Spent: 10m > Remaining Estimate: 0h > > Please consider an improvement request to add functionality to the PutSNS > processor to support the October 2020 release of FIFO messaging in AWS SNS. > This would include the capabilities to have attributes for Message Group ID, > Content-Based Deduplication and a Deduplication ID. > This would be similar to NIFI-4540 that was recently resolved for PutSQS FIFO. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8011) Add "Move to group" function
[ https://issues.apache.org/jira/browse/NIFI-8011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17232332#comment-17232332 ] Alasdair Brown commented on NIFI-8011: -- "Suggestion 1" is already a feature in NiFi - select multiple processors and drag them on top of a PG and they will be moved in to the PG. > Add "Move to group" function > > > Key: NIFI-8011 > URL: https://issues.apache.org/jira/browse/NIFI-8011 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Affects Versions: 1.12.1 >Reporter: Svyatoslav >Priority: Major > > The only way to move several existing processors into a group is copy-paste. > Suggestions: > # Select processors with shift-click or shift-drag, drag them onto existing > group. Result: selected processors are moved into the group. > # Select processors with shift-click or shift-drag, open popup menu, select > "Create group" item. Result: new group is created and selected processors are > moved into it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (NIFI-7376) Avro Single-object encoding Support
[ https://issues.apache.org/jira/browse/NIFI-7376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17229094#comment-17229094 ] Alasdair Brown edited comment on NIFI-7376 at 11/10/20, 9:53 AM: - FYI see NIFI-7917 which is tracking Avro upgrade to 1.10 was (Author: sdairs): FYI see NIFI-7917 > Avro Single-object encoding Support > --- > > Key: NIFI-7376 > URL: https://issues.apache.org/jira/browse/NIFI-7376 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.9.2 >Reporter: Nathan English >Priority: Minor > > For my Flows I consume Avro Binary Encoded Messages from Kafka which is > currently working great! However going forward one of our inputs is looking > to provide [Avro Single-object > Encoding|[https://avro.apache.org/docs/1.8.2/spec.html#single_object_encoding]]. > > Avro Single Object Encoding provides us a way to where we don't have to have > the overhead and message size increase of embedding the schema in the > message. Single Object Encoding achieves this with a couple of extra fields > to firstly confirm the message is an Avro message and secondly a fingerprint > of the schema used to encode the message. > This is a massive benefit for us, because we have multiple of the same device > producing messages into one Kafka Topic. This is fine until we start > upgrading these devices where schema changes may occur, this is when schema > fingerprinting comes into its element. > From all the information I have found, it looks to me as if Avro Single > Object Encoding was added in version 1.8.2 of the Avro Specification. NiFi is > currently using version 1.8.1 of the Avro Specification based on the > [pom|[https://github.com/apache/nifi/blob/rel/nifi-1.11.4/nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/pom.xml]] > in NiFi Record Serialization Services section of the github project. > I'm sure there are a million ways to tackle this issue and I'm personally > haven't done enough research on NiFi or Avro on it to suggest a way to > resolve this, but I can tell it's not as simple as just upgrading the Avro > version used. > My thoughts were to upgrade the Library version, then modify the avro schema > registry to either take in a fingerprint value or have it calculated on > enablement of the registry. I'm sure it's probably not as simple as I have > just made it!. > More than happy to help where I can. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7376) Avro Single-object encoding Support
[ https://issues.apache.org/jira/browse/NIFI-7376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17229094#comment-17229094 ] Alasdair Brown commented on NIFI-7376: -- FYI see NIFI-7917 > Avro Single-object encoding Support > --- > > Key: NIFI-7376 > URL: https://issues.apache.org/jira/browse/NIFI-7376 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.9.2 >Reporter: Nathan English >Priority: Minor > > For my Flows I consume Avro Binary Encoded Messages from Kafka which is > currently working great! However going forward one of our inputs is looking > to provide [Avro Single-object > Encoding|[https://avro.apache.org/docs/1.8.2/spec.html#single_object_encoding]]. > > Avro Single Object Encoding provides us a way to where we don't have to have > the overhead and message size increase of embedding the schema in the > message. Single Object Encoding achieves this with a couple of extra fields > to firstly confirm the message is an Avro message and secondly a fingerprint > of the schema used to encode the message. > This is a massive benefit for us, because we have multiple of the same device > producing messages into one Kafka Topic. This is fine until we start > upgrading these devices where schema changes may occur, this is when schema > fingerprinting comes into its element. > From all the information I have found, it looks to me as if Avro Single > Object Encoding was added in version 1.8.2 of the Avro Specification. NiFi is > currently using version 1.8.1 of the Avro Specification based on the > [pom|[https://github.com/apache/nifi/blob/rel/nifi-1.11.4/nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/pom.xml]] > in NiFi Record Serialization Services section of the github project. > I'm sure there are a million ways to tackle this issue and I'm personally > haven't done enough research on NiFi or Avro on it to suggest a way to > resolve this, but I can tell it's not as simple as just upgrading the Avro > version used. > My thoughts were to upgrade the Library version, then modify the avro schema > registry to either take in a fingerprint value or have it calculated on > enablement of the registry. I'm sure it's probably not as simple as I have > just made it!. > More than happy to help where I can. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7891) Allow for Defaults to be set by Avro writers
[
https://issues.apache.org/jira/browse/NIFI-7891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1728#comment-1728
]
Alasdair Brown commented on NIFI-7891:
--
Initial commit...welcome feedback on quality & approach
> Allow for Defaults to be set by Avro writers
>
>
> Key: NIFI-7891
> URL: https://issues.apache.org/jira/browse/NIFI-7891
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Reporter: Alasdair Brown
>Assignee: Alasdair Brown
>Priority: Minor
> Time Spent: 20m
> Remaining Estimate: 0h
>
> This would be an improvement to Avro writer services. Allowing a set of
> default values to be given based on the data type when infering the schema.
>
> For example, you could have an additional property per type (e.g. Int,
> String, etc.) where the value of the property would be used as the default
> value. However, this would add a lot of properties Possibly you could
> provide a JSON set of types -> defaults, e.g.
> {code:java}
> [
>{ "type":"string", "default":"test"},
>{ "type":"int","default":1},
> ]{code}
> Thus, for any field inferred as type String, it's embedded Avro schema
> contains a default value entry for that field.
> This would then provide flexibility where NULL values are present - the
> current behaviour only supports adding NULL to the allowable types - this
> would be a very useful alternative.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (NIFI-7891) Allow for Defaults to be set by Avro writers
[
https://issues.apache.org/jira/browse/NIFI-7891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alasdair Brown reassigned NIFI-7891:
Assignee: Alasdair Brown
> Allow for Defaults to be set by Avro writers
>
>
> Key: NIFI-7891
> URL: https://issues.apache.org/jira/browse/NIFI-7891
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Reporter: Alasdair Brown
>Assignee: Alasdair Brown
>Priority: Minor
> Time Spent: 20m
> Remaining Estimate: 0h
>
> This would be an improvement to Avro writer services. Allowing a set of
> default values to be given based on the data type when infering the schema.
>
> For example, you could have an additional property per type (e.g. Int,
> String, etc.) where the value of the property would be used as the default
> value. However, this would add a lot of properties Possibly you could
> provide a JSON set of types -> defaults, e.g.
> {code:java}
> [
>{ "type":"string", "default":"test"},
>{ "type":"int","default":1},
> ]{code}
> Thus, for any field inferred as type String, it's embedded Avro schema
> contains a default value entry for that field.
> This would then provide flexibility where NULL values are present - the
> current behaviour only supports adding NULL to the allowable types - this
> would be a very useful alternative.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (NIFI-7901) Support an 'initial offset' on stateful SQL fetch processors where a max-value column is set
[ https://issues.apache.org/jira/browse/NIFI-7901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alasdair Brown resolved NIFI-7901. -- Resolution: Duplicate > Support an 'initial offset' on stateful SQL fetch processors where a > max-value column is set > > > Key: NIFI-7901 > URL: https://issues.apache.org/jira/browse/NIFI-7901 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Alasdair Brown >Priority: Minor > > Where a max-value column is set, it would be beneficial to be able to set an > 'initial offset' value, such that when there is no state to begin from, it > uses the value of the initial offset. > > The use case I am seeing for this is the desire to re-ingest the past X days > only, without building a new flow/adding additional temporary wheres. They > could simply stop the processor, update the initial offset, clear the state, > and resume as normal. > > E.g. on GenerateTableFetch -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7901) Support an 'initial offset' on stateful SQL fetch processors where a max-value column is set
[ https://issues.apache.org/jira/browse/NIFI-7901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17210390#comment-17210390 ] Alasdair Brown commented on NIFI-7901: -- I've just noticed this is already a feature...so uhexcuse me :D > Support an 'initial offset' on stateful SQL fetch processors where a > max-value column is set > > > Key: NIFI-7901 > URL: https://issues.apache.org/jira/browse/NIFI-7901 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Alasdair Brown >Priority: Minor > > Where a max-value column is set, it would be beneficial to be able to set an > 'initial offset' value, such that when there is no state to begin from, it > uses the value of the initial offset. > > The use case I am seeing for this is the desire to re-ingest the past X days > only, without building a new flow/adding additional temporary wheres. They > could simply stop the processor, update the initial offset, clear the state, > and resume as normal. > > E.g. on GenerateTableFetch -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-7901) Support an 'initial offset' on stateful SQL fetch processors where a max-value column is set
Alasdair Brown created NIFI-7901: Summary: Support an 'initial offset' on stateful SQL fetch processors where a max-value column is set Key: NIFI-7901 URL: https://issues.apache.org/jira/browse/NIFI-7901 Project: Apache NiFi Issue Type: New Feature Reporter: Alasdair Brown Where a max-value column is set, it would be beneficial to be able to set an 'initial offset' value, such that when there is no state to begin from, it uses the value of the initial offset. The use case I am seeing for this is the desire to re-ingest the past X days only, without building a new flow/adding additional temporary wheres. They could simply stop the processor, update the initial offset, clear the state, and resume as normal. E.g. on GenerateTableFetch -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-7891) Allow for Defaults to be set by Avro writers
Alasdair Brown created NIFI-7891:
Summary: Allow for Defaults to be set by Avro writers
Key: NIFI-7891
URL: https://issues.apache.org/jira/browse/NIFI-7891
Project: Apache NiFi
Issue Type: New Feature
Components: Extensions
Reporter: Alasdair Brown
This would be an improvement to Avro writer services. Allowing a set of default
values to be given based on the data type when infering the schema.
For example, you could have an additional property per type (e.g. Int, String,
etc.) where the value of the property would be used as the default value.
However, this would add a lot of properties Possibly you could provide a
JSON set of types -> defaults, e.g.
{code:java}
[
{ "type":"string", "default":"test"},
{ "type":"int","default":1},
]{code}
Thus, for any field inferred as type String, it's embedded Avro schema contains
a default value entry for that field.
This would then provide flexibility where NULL values are present - the
current behaviour only supports adding NULL to the allowable types - this would
be a very useful alternative.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-4897) Add documentation to user guide for terminating a processor
[ https://issues.apache.org/jira/browse/NIFI-4897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206843#comment-17206843 ] Alasdair Brown commented on NIFI-4897: -- Submitted a new PR for this [https://github.com/apache/nifi/pull/4568] > Add documentation to user guide for terminating a processor > --- > > Key: NIFI-4897 > URL: https://issues.apache.org/jira/browse/NIFI-4897 > Project: Apache NiFi > Issue Type: Sub-task > Components: Documentation Website >Reporter: Mark Payne >Assignee: Alasdair Brown >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (NIFI-4897) Add documentation to user guide for terminating a processor
[ https://issues.apache.org/jira/browse/NIFI-4897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alasdair Brown reassigned NIFI-4897: Assignee: Alasdair Brown > Add documentation to user guide for terminating a processor > --- > > Key: NIFI-4897 > URL: https://issues.apache.org/jira/browse/NIFI-4897 > Project: Apache NiFi > Issue Type: Sub-task > Components: Documentation Website >Reporter: Mark Payne >Assignee: Alasdair Brown >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-7877) Clicking a bulletin opens the Bulletin Board with a filter applied to the processor of the clicked bulletin
Alasdair Brown created NIFI-7877: Summary: Clicking a bulletin opens the Bulletin Board with a filter applied to the processor of the clicked bulletin Key: NIFI-7877 URL: https://issues.apache.org/jira/browse/NIFI-7877 Project: Apache NiFi Issue Type: Wish Components: Core UI Reporter: Alasdair Brown Bulletins can contain entire stack traces, which makes the bulletin diaglogue too large for the screen, thus cuts off useful information. It would be very handy to be able to interact with the bulletin messages that are attached to the processors. For example, in this case, it would be useful if you could click the bulletin icon on the processor, which would then open up the Bulletin Board and automatically apply the necessary filter to show you the errors from the processor/group on which the bulletin was clicked. Additionally, this would mean that you could cut off error messages that were too long, to prevent them going out of the visible work space, and still be able to efficiently navigate to the full text (without resorting to the log files). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7864) No Cancel button when adding Variables or editing their values
[ https://issues.apache.org/jira/browse/NIFI-7864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204673#comment-17204673 ] Alasdair Brown commented on NIFI-7864: -- It is indeed, apologies > No Cancel button when adding Variables or editing their values > -- > > Key: NIFI-7864 > URL: https://issues.apache.org/jira/browse/NIFI-7864 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Affects Versions: 1.11.4 >Reporter: Alasdair Brown >Priority: Minor > Fix For: 1.12.0 > > > There is no Cancel button when you add a new variable, nor when you edit an > existing Variables value. You have to use the Escape key to cancel out of the > dialogs. This is inconsistent with the UX elsewhere, e.g. when configuring > processor properties or working with Parameters -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-7864) No Cancel button when adding Variables or editing their values
Alasdair Brown created NIFI-7864: Summary: No Cancel button when adding Variables or editing their values Key: NIFI-7864 URL: https://issues.apache.org/jira/browse/NIFI-7864 Project: Apache NiFi Issue Type: Improvement Components: Core UI Affects Versions: 1.11.4 Reporter: Alasdair Brown There is no Cancel button when you add a new variable, nor when you edit an existing Variables value. You have to use the Escape key to cancel out of the dialogs. This is inconsistent with the UX elsewhere, e.g. when configuring processor properties or working with Parameters -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-5061) NiFi documentation incomplete/wrong for EL hierarchy.
[ https://issues.apache.org/jira/browse/NIFI-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17196295#comment-17196295 ] Alasdair Brown commented on NIFI-5061: -- [https://github.com/apache/nifi/pull/4530] Very simple PR for this submitted - just tidies up the example and splits the hierarchy you provided in to a new subsection. > NiFi documentation incomplete/wrong for EL hierarchy. > - > > Key: NIFI-5061 > URL: https://issues.apache.org/jira/browse/NIFI-5061 > Project: Apache NiFi > Issue Type: Bug > Components: Documentation Website >Affects Versions: 1.5.0 > Environment: N/A >Reporter: Matthew Clarke >Assignee: Alasdair Brown >Priority: Major > Labels: newbie > Time Spent: 10m > Remaining Estimate: 0h > > Within the NiFi Expression Language Guide under the "Structure of a NiFi > Expression" section, the hierarchy of how NiFi searches for a Attribute key > is wrong (still based off NiFi 0.x hierarchy) and incomplete: > Current text: > In this example, the value to be returned is the value of the "my attribute" > value, if it exists. If that attribute does not exist, the Expression > Language will then look for a System Environment Variable named "my > attribute." If unable to find this, it will look for a JVM System Property > named "my attribute." Finally, if none of these exists, the Expression > Language will return a null value. > The current hierarchy in NiFi is as follows: > 1. Search FlowFile for Attribute/key > 2. Search Process Group for attribute/key > 3. Search File Registry File for attribute/key > 4. Search NiFi JVM Properties for attribute/key > 5. Search System Environment Variables for attribute/key > We should not only fix the above, but in doing so make the above hierarchy > more prominent in the documentation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (NIFI-5061) NiFi documentation incomplete/wrong for EL hierarchy.
[ https://issues.apache.org/jira/browse/NIFI-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alasdair Brown reassigned NIFI-5061: Assignee: Alasdair Brown > NiFi documentation incomplete/wrong for EL hierarchy. > - > > Key: NIFI-5061 > URL: https://issues.apache.org/jira/browse/NIFI-5061 > Project: Apache NiFi > Issue Type: Bug > Components: Documentation Website >Affects Versions: 1.5.0 > Environment: N/A >Reporter: Matthew Clarke >Assignee: Alasdair Brown >Priority: Major > Labels: newbie > > Within the NiFi Expression Language Guide under the "Structure of a NiFi > Expression" section, the hierarchy of how NiFi searches for a Attribute key > is wrong (still based off NiFi 0.x hierarchy) and incomplete: > Current text: > In this example, the value to be returned is the value of the "my attribute" > value, if it exists. If that attribute does not exist, the Expression > Language will then look for a System Environment Variable named "my > attribute." If unable to find this, it will look for a JVM System Property > named "my attribute." Finally, if none of these exists, the Expression > Language will return a null value. > The current hierarchy in NiFi is as follows: > 1. Search FlowFile for Attribute/key > 2. Search Process Group for attribute/key > 3. Search File Registry File for attribute/key > 4. Search NiFi JVM Properties for attribute/key > 5. Search System Environment Variables for attribute/key > We should not only fix the above, but in doing so make the above hierarchy > more prominent in the documentation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7580) Add documentation around autoloading NARs
[
https://issues.apache.org/jira/browse/NIFI-7580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17196061#comment-17196061
]
Alasdair Brown commented on NIFI-7580:
--
PR for updates to the admin guide for adding custom processors and using the
autoload feature.
I made the assumption that deploying custom processors would generally fall
under an administrative task rather than a developers task, so didn't touch the
dev guide.
I didn't mention versioning considerations, but perhaps this might need
expanding upon.
> Add documentation around autoloading NARs
> -
>
> Key: NIFI-7580
> URL: https://issues.apache.org/jira/browse/NIFI-7580
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Documentation Website
>Affects Versions: 1.9.0
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Major
> Labels: autoload, documentation, hot-swap, nar
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The autoloading NAR feature was added in 1.9.0, but the current documentation
> does not cover this feature.
> * Admin Guide - describe behavior and {{nifi.nar.library.autoload.directory}}
> property value
> * Developer Guide - describe deployment of custom processors to
> {{extensions/}} directory (caution that versioning is important for loading
> and SNAPSHOTS without timestamp in filename will not be detected and reloaded)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-7807) PutKudu documentation clarity & examples
[ https://issues.apache.org/jira/browse/NIFI-7807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17195445#comment-17195445 ] Alasdair Brown commented on NIFI-7807: -- PR Submitted [https://github.com/apache/nifi/pull/4526] > PutKudu documentation clarity & examples > > > Key: NIFI-7807 > URL: https://issues.apache.org/jira/browse/NIFI-7807 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation Website >Affects Versions: 1.12.1 >Reporter: Alasdair Brown >Assignee: Alasdair Brown >Priority: Trivial > Time Spent: 10m > Remaining Estimate: 0h > > The PutKudu processor could use some changes to documentation. > > Issue created to track some changes, namely: > * Updating the processor description for more clarity about how the Record > -> Column schema works (i.e. inferred from the record reader) > * Adding an additional details page for more clarity on the above and some > examples to demonstrate > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-7807) PutKudu documentation clarity & examples
Alasdair Brown created NIFI-7807: Summary: PutKudu documentation clarity & examples Key: NIFI-7807 URL: https://issues.apache.org/jira/browse/NIFI-7807 Project: Apache NiFi Issue Type: Improvement Components: Documentation Website Affects Versions: 1.12.1 Reporter: Alasdair Brown The PutKudu processor could use some changes to documentation. Issue created to track some changes, namely: * Updating the processor description for more clarity about how the Record -> Column schema works (i.e. inferred from the record reader) * Adding an additional details page for more clarity on the above and some examples to demonstrate -- This message was sent by Atlassian Jira (v8.3.4#803005)
