[jira] [Updated] (NIFI-4951) convertToAvroObject does not utilize AvroTypeUtil when trying to convert Date, Time, or Timestamps
[ https://issues.apache.org/jira/browse/NIFI-4951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Derek Straka updated NIFI-4951: --- Assignee: (was: Derek Straka) Status: Patch Available (was: In Progress) > convertToAvroObject does not utilize AvroTypeUtil when trying to convert > Date, Time, or Timestamps > -- > > Key: NIFI-4951 > URL: https://issues.apache.org/jira/browse/NIFI-4951 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Derek Straka >Priority: Major > > Currently, AvroTypeUtil::convertToAvroObject assumes that the logical types > have already been converted into their internal representation. While this > may sometimes be the case, it would be useful to allow String->Logical type > conversion. The feature can be added by using the appropriate > DataTypeUtils::to method rather than blindly attempting to use > the underlying type. The DataTypeUtils already performs the integral type > parsing, so the existing functionality remains intact. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2527: FetchHBaseRow - log level and displayName
Github user bdesert commented on the issue: https://github.com/apache/nifi/pull/2527 @pvillard31 , @bbende In case we want to have bulletin to be generated, we can change "Bulletin Level" of the component to "DEBUG". But then bulletin will be generated even if rowkey is found. I would rather remove lines 276-278, to have clean debug level bulletin for "not found" only. Your thoughts? ---
[jira] [Commented] (NIFI-4936) NiFi parent pom dependency management forcing versions to align defeating classloader isolation
[ https://issues.apache.org/jira/browse/NIFI-4936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394567#comment-16394567 ] Joseph Percivall commented on NIFI-4936: [~joewitt] is there anything we'll need to add to the Migration Guidance for this for those that are utilizing the parent pom for their nar bundles? > NiFi parent pom dependency management forcing versions to align defeating > classloader isolation > --- > > Key: NIFI-4936 > URL: https://issues.apache.org/jira/browse/NIFI-4936 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework, Extensions, Tools and Build >Affects Versions: 1.1.0, 1.2.0, 1.0.1, 1.3.0, 1.4.0, 1.5.0 >Reporter: Joseph Witt >Assignee: Joseph Witt >Priority: Major > Fix For: 1.6.0 > > Attachments: NIFI-4936.patch, build-fixing, > old-vs-new-dependencies.txt > > > the top level pom in nifi has a massive dependency management section. this > was used initially to help enforce consistent usage of dependencies across > nifi but this also can defeat the purpose of the classloader isolation > offered by nars. We need to push down dependency version declarations to the > nar levels where appropriate. > there have been reported issues of defects happening due to us using much > newer versions (or sometimes older) of dependencies due to this dependency > management model. By pushing declarations down to the proper scope each > nar/etc.. can use the specific versions of components it needs and we'll stop > introducing issues by forcing a different version. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4838) Make GetMongo support multiple commits and give some progress indication
[ https://issues.apache.org/jira/browse/NIFI-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394534#comment-16394534 ] ASF GitHub Bot commented on NIFI-4838: -- Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2448 @pvillard31 I found that a catch block was suppressing a critical problem with how I was handling the input flowfile. When I'd call `session.commit()` after a block of results were written to a flowfile, it would throw an exception because the input flowfile had not been transferred yet. So what I am going to do is redo this to make it configurable again. With the default behavior being only one single commit. Then users can choose a "streaming mode" where at the the first sign of success it transfers the input flowfile to REL_ORIGINAL. Then if an error happens, it would write a new flowfile with the same attributes and content as the original and send that cloned copy to the failure relationship. It may not be perfect, but I don't think NiFi has a way to do a partial commit which is what I'd need to make it work as-is. > Make GetMongo support multiple commits and give some progress indication > > > Key: NIFI-4838 > URL: https://issues.apache.org/jira/browse/NIFI-4838 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > > It shouldn't wait until the end to do a commit() call because the effect is > that GetMongo looks like it has hung to a user who is pulling a very large > data set. > It should also have an option for running a count query to get the current > approximate count of documents that would match the query and append an > attribute that indicates where a flowfile stands in the total result count. > Ex: > query.progress.point.start = 2500 > query.progress.point.end = 5000 > query.count.estimate = 17,568,231 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2448: NIFI-4838 Added configurable progressive commits to GetMon...
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2448 @pvillard31 I found that a catch block was suppressing a critical problem with how I was handling the input flowfile. When I'd call `session.commit()` after a block of results were written to a flowfile, it would throw an exception because the input flowfile had not been transferred yet. So what I am going to do is redo this to make it configurable again. With the default behavior being only one single commit. Then users can choose a "streaming mode" where at the the first sign of success it transfers the input flowfile to REL_ORIGINAL. Then if an error happens, it would write a new flowfile with the same attributes and content as the original and send that cloned copy to the failure relationship. It may not be perfect, but I don't think NiFi has a way to do a partial commit which is what I'd need to make it work as-is. ---
[jira] [Commented] (NIFI-4800) Expose the flattenMode as property in FlattenJSON processor
[ https://issues.apache.org/jira/browse/NIFI-4800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394530#comment-16394530 ] ASF GitHub Bot commented on NIFI-4800: -- GitHub user yjhyjhyjh0 opened a pull request: https://github.com/apache/nifi/pull/2530 NIFI-4800 Expose the flattenMode as property in FlattenJSON processor Expose all 3 kinds of flatten mode. Documentation along with unit tests. Finish local nifi instance integration test. ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/yjhyjhyjh0/nifi NIFI-4800 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2530.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2530 commit d78cf4e088b075feef17c515cb3577b3198b1eaf Author: Deon Huang Date: 2018-03-11T15:01:53Z NIFI-4800 Expose the flattenMode as property in FlattenJSON processor > Expose the flattenMode as property in FlattenJSON processor > --- > > Key: NIFI-4800 > URL: https://issues.apache.org/jira/browse/NIFI-4800 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.5.0 >Reporter: Alois Gruber >Assignee: Deon Huang >Priority: Trivial > > the flattening class supports 3 different modes, which cannot be selected in > the processor. Especially the flattening of arrays would be helpful -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2530: NIFI-4800 Expose the flattenMode as property in Fla...
GitHub user yjhyjhyjh0 opened a pull request: https://github.com/apache/nifi/pull/2530 NIFI-4800 Expose the flattenMode as property in FlattenJSON processor Expose all 3 kinds of flatten mode. Documentation along with unit tests. Finish local nifi instance integration test. ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/yjhyjhyjh0/nifi NIFI-4800 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2530.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2530 commit d78cf4e088b075feef17c515cb3577b3198b1eaf Author: Deon Huang Date: 2018-03-11T15:01:53Z NIFI-4800 Expose the flattenMode as property in FlattenJSON processor ---
[jira] [Commented] (NIFI-2853) Improve ListHDFS state tracking
[ https://issues.apache.org/jira/browse/NIFI-2853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394497#comment-16394497 ] Sivaprasanna Sethuraman commented on NIFI-2853: --- Although not a critical one, I believe this is an important feature that is needed. And I also think it is better to have not just the root level directory name appended to the "listing.timestamp" and "emitted.timestamp" but also include the sub directories, like "listing.timestamp.dir1.subdir2", "listing.timestamp.dir1.subdir3.subdir3_1" to avoid edgecase scenarios. The reason is, if we don't do that, files might not get picked up in some scenario. Ex: # Create a directory "/tmp/sub-dir1" # Create a file "file1.txt" under "/tmp/sub-dir1" # Create a couple of files under "/tmp" # Create another file "file2.txt" under "/tmp/sub-dir1" Now set ListHDFS as "Directory" : /tmp/sub-dir1. Run the flow. It will set the timestamp to the last accessed file which is "/tmp/sub-dir1/file2.txt". Now change the directory of ListHDFS to "/tmp", it won't pull in the files that were created in step 3 because those files modified time would be lesser than the timestamp stored as part of the processor's state. It will not happen with the said approach. Thoughts? > Improve ListHDFS state tracking > --- > > Key: NIFI-2853 > URL: https://issues.apache.org/jira/browse/NIFI-2853 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.0.0 >Reporter: Bryan Bende >Priority: Minor > > Currently ListHDFS tracks two properties in state management, > "listing.timestamp" and "emitted.timestamp". In the 1.0.0 release, the > directory property now supports expression language which means the directory > being listed could dynamically change on any execution of the processor. > The processor should be changed to store state specific to the directory that > was listed, for example "listing.timestamp.dir1" and "emitted.timestamp.dir1". > This would also help in a clustered scenario... currently ListHDFS has to be > run on primary node only, otherwise each node will be overwriting each others > state and producing unexpected results. With the above improvement, if the > directory evaluated to a unique path for each node, it would store the state > of each of those path separately. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (NIFI-4185) Add XML record reader & writer services
[
https://issues.apache.org/jira/browse/NIFI-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Johannes Peter reassigned NIFI-4185:
Assignee: Johannes Peter
> Add XML record reader & writer services
> ---
>
> Key: NIFI-4185
> URL: https://issues.apache.org/jira/browse/NIFI-4185
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Andy LoPresto
>Assignee: Johannes Peter
>Priority: Major
> Labels: json, records, xml
>
> With the addition of the {{RecordReader}} and {{RecordSetWriter}} paradigm,
> XML conversion has not yet been targeted. This will replace the previous
> ticket for XML to JSON conversion.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4185) Add XML record reader & writer services
[
https://issues.apache.org/jira/browse/NIFI-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394487#comment-16394487
]
Johannes Peter edited comment on NIFI-4185 at 3/11/18 12:47 PM:
[~alopresto]:
Started implementing an XML Record Reader. Shall I create a separate ticket
for this?
Similar to the JSON readers, the XML reader will expect either a single record,
e. g.
{code}
content
...
{code}
or an array of records, e. g.
{code}
content
...
...
{code}
The reader will be aligned with common transformators. "Normal" fields (e. g.
String, Integer) can be described by simple key-value pairs:
XML definition
{code}
content
123
{code}
Schema definition
{code}
{ "name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{ "name": "field1", "type": "string" },
{ "name": "field2", "type": "int" }
]
}
{code}
Parsing of attributes or nested fields require the definition of nested records
and a field name for the content (optional, a prefix for attributes can be
defined):
Property: CONTENT_FIELD=content_field
Property: ATTRIBUTE_PREFIX=attr.
XML definition
{code}
some text
some nested text
some other nested text
{code}
Schema definition
{code}
{
"name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{
"name": "field1",
"type": {
"name": "NestedRecord",
"type": "record",
"fields" : [
{"name": "attr.attribute", "type": "string"},
{"name": "content_field", "type": "string"}
]
}
},
{
"name": "field2",
"type": {
"name": "NestedRecord",
"type": "record",
"fields" : [
{"name": "attr.attribute", "type": "string"},
{"name": "nested1", "type": "string"},
{"name": "nested2", "type": "string"}
]
}
}
]
}
{code}
What do you say?
was (Author: jope):
[~alopresto]:
Started implementing an XML Record Reader. Shall I create a separate ticket
for this?
Similar to the JSON readers, the XML reader will expect either a single record
(e. g. content ... ) or an array of
records (e. g. content ...
... )
The reader will be aligned with common transformators. "Normal" fields (e. g.
String, Integer) can be described by simple key-value pairs:
XML definition
{code}
content
123
{code}
Schema definition
{code}
{ "name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{ "name": "field1", "type": "string" },
{ "name": "field2", "type": "int" }
]
}
{code}
Parsing of attributes or nested fields require the definition of nested records
and a field name for the content (optional, a prefix for attributes can be
defined):
Property: CONTENT_FIELD=content_field
Property: ATTRIBUTE_PREFIX=attr.
XML definition
{code}
some text
some nested text
some other nested text
{code}
Schema definition
{code}
{
"name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{
"name": "field1",
"type": {
"name": "NestedRecord",
"type": "record",
"fields" : [
{"name": "attr.attribute", "type": "string"},
{"name": "content_field", "type": "string"}
]
}
},
{
"name": "field2",
"type": {
"name": "NestedRecord",
"type": "record",
"fields" : [
{"name": "attr.attribute", "type": "string"},
{"name": "nested1", "type": "string"},
{"name": "nested2", "type": "string"}
]
}
}
]
}
{code}
What do you say?
> Add XML record reader & writer services
> ---
>
> Key: NIFI-4185
> URL: https://issues.apache.org/jira/browse/NIFI-4185
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Andy LoPresto
>Priority: Major
> Labels: json, records, xml
>
> With the addition of the {{RecordReader}} and {{RecordSetWriter}} paradigm,
> XML conversion has not yet been targeted. This will replace the previous
> ticket for XML to JSON conversion.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4185) Add XML record reader & writer services
[
https://issues.apache.org/jira/browse/NIFI-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394487#comment-16394487
]
Johannes Peter edited comment on NIFI-4185 at 3/11/18 12:45 PM:
[~alopresto]:
Started implementing an XML Record Reader. Shall I create a separate ticket
for this?
Similar to the JSON readers, the XML reader will expect either a single record
(e. g. content ... ) or an array of
records (e. g. content ...
... )
The reader will be aligned with common transformators. "Normal" fields (e. g.
String, Integer) can be described by simple key-value pairs:
XML definition
{code}
content
123
{code}
Schema definition
{code}
{ "name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{ "name": "field1", "type": "string" },
{ "name": "field2", "type": "int" }
]
}
{code}
Parsing of attributes or nested fields require the definition of nested records
and a field name for the content (optional, a prefix for attributes can be
defined):
Property: CONTENT_FIELD=content_field
Property: ATTRIBUTE_PREFIX=attr.
XML definition
{code}
some text
some nested text
some other nested text
{code}
Schema definition
{code}
{
"name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{
"name": "field1",
"type": {
"name": "NestedRecord",
"type": "record",
"fields" : [
{"name": "attr.attribute", "type": "string"},
{"name": "content_field", "type": "string"}
]
}
},
{
"name": "field2",
"type": {
"name": "NestedRecord",
"type": "record",
"fields" : [
{"name": "attr.attribute", "type": "string"},
{"name": "nested1", "type": "string"},
{"name": "nested2", "type": "string"}
]
}
}
]
}
{code}
What do you say?
was (Author: jope):
[~alopresto]:
Started implementing an XML Record Reader. Shall I create a separate ticket
for this?
Similar to the JSON readers, the XML reader will expect either a single record
(e. g. content ... ) or an array of
records (e. g. content ...
... )
The reader will be aligned with common transformators. "Normal" fields (e. g.
String, Integer) can be described by simple key-value pairs:
XML definition
{code}
content
123
{code}
Schema definition
{code}
{ "name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{ "name": "field1", "type": "string" },
{ "name": "field2", "type": "int" }
]
}
{code}
Parsing of attributes or nested fields require the definition of nested records
and a field name for the content (optional, a prefix for attributes can be
defined):
Property: CONTENT_FIELD=content_field
Property: ATTRIBUTE_PREFIX=attr.
XML definition
{code}
some text
some nested text
some other nested text
{code}
Schema definition
{code}
{
"name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{
"name": "field1",
"type": {
"name": "NestedRecord",
"type": "record",
"fields" : [
{"name": "attr.attribute", "type":
"string"},
{"name": "content_field", "type":
"string"}
]
}
},
{
"name": "field2",
"type": {
"name": "NestedRecord",
"type": "record",
"fields" : [
{"name": "attr.attribute", "type":
"string"},
{"name": "nested1", "type": "string"},
{"name": "nested2", "type": "string"}
]
}
}
]
}
{code}
What do you say?
> Add XML record reader & writer services
> ---
>
> Key: NIFI-4185
> URL: https://issues.apache.org/jira/browse/NIFI-4185
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Andy LoPresto
>Priority: Major
> Labels: json, records, xml
>
> With the addition of the {{RecordReader}} and {{RecordSetWriter}} paradigm,
> XML conversion has not yet been targeted. This will replace the previous
> ticket for XML to JSON conversion.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4185) Add XML record reader & writer services
[
https://issues.apache.org/jira/browse/NIFI-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394487#comment-16394487
]
Johannes Peter edited comment on NIFI-4185 at 3/11/18 12:43 PM:
[~alopresto]:
Started implementing an XML Record Reader. Shall I create a separate ticket
for this?
Similar to the JSON readers, the XML reader will expect either a single record
(e. g. content ... ) or an array of
records (e. g. content ...
... )
The reader will be aligned with common transformators. "Normal" fields (e. g.
String, Integer) can be described by simple key-value pairs:
XML definition
{code}
content
123
{code}
Schema definition
{code}
{ "name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{ "name": "field1", "type": "string" },
{ "name": "field2", "type": "int" }
]
}
{code}
Parsing of attributes or nested fields require the definition of nested records
and a field name for the content (optional, a prefix for attributes can be
defined):
Property: CONTENT_FIELD=content_field
Property: ATTRIBUTE_PREFIX=attr.
XML definition
{code}
some text
some nested text
some other nested text
{code}
Schema definition
{code}
{
"name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{
"name": "field1",
"type": {
"name": "NestedRecord",
"type": "record",
"fields" : [
{"name": "attr.attribute", "type":
"string"},
{"name": "content_field", "type":
"string"}
]
}
},
{
"name": "field2",
"type": {
"name": "NestedRecord",
"type": "record",
"fields" : [
{"name": "attr.attribute", "type":
"string"},
{"name": "nested1", "type": "string"},
{"name": "nested2", "type": "string"}
]
}
}
]
}
{code}
What do you say?
was (Author: jope):
[~alopresto]:
Started implementing an XML Record Reader. Shall I create a separate ticket
for this?
Similar to the JSON readers, the XML reader will expect either a single record
(e. g. content ... ) or an array of
records (e. g. content ...
... )
The reader will be aligned with common transformators. "Normal" fields (e. g.
String, Integer) can be described by simple key-value pairs:
XML definition
{code}
content
123
{code}
Schema definition
{code}
{ "name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{ "name": "field1", "type": "string" },
{ "name": "field2", "type": "int" }
]
}
{code}
Parsing of attributes or nested fields require the definition of nested records
and a field name for the content (optional, a prefix for attributes can be
defined):
Property: CONTENT_FIELD=content_field
Property: ATTRIBUTE_PREFIX=attr.
XML definition
{code}
some text
some nested text
some other nested text
{code}
Schema definition
{code}
{
"name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{
"name": "field1",
"type": {
"name": "NestedRecord",
"type": "record",
"fields" : [
{ "name": "attr.attribute", "type": "string" },
{ "name": "content_field", "type": "string" }
]
}
},
{
"name": "field2",
"type": {
"name": "NestedRecord",
"type": "record",
"fields" : [
{ "name": "attr.attribute", "type": "string" },
{ "name": "nested1", "type": "string" },
{ "name": "nested2", "type": "string" }
]
}
}
]
}
{code}
What do you say?
> Add XML record reader & writer services
> ---
>
> Key: NIFI-4185
> URL: https://issues.apache.org/jira/browse/NIFI-4185
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Andy LoPresto
>Priority: Major
> Labels: json, records, xml
>
> With the addition of the {{RecordReader}} and {{RecordSetWriter}} paradigm,
> XML conversion has not yet been targeted. This will replace the previous
> ticket for XML to JSON conversion.
--
T
[jira] [Comment Edited] (NIFI-4185) Add XML record reader & writer services
[
https://issues.apache.org/jira/browse/NIFI-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394487#comment-16394487
]
Johannes Peter edited comment on NIFI-4185 at 3/11/18 12:42 PM:
[~alopresto]:
Started implementing an XML Record Reader. Shall I create a separate ticket
for this?
Similar to the JSON readers, the XML reader will expect either a single record
(e. g. content ... ) or an array of
records (e. g. content ...
... )
The reader will be aligned with common transformators. "Normal" fields (e. g.
String, Integer) can be described by simple key-value pairs:
XML definition
{code}
content
123
{code}
Schema definition
{code}
{ "name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{ "name": "field1", "type": "string" },
{ "name": "field2", "type": "int" }
]
}
{code}
Parsing of attributes or nested fields require the definition of nested records
and a field name for the content (optional, a prefix for attributes can be
defined):
Property: CONTENT_FIELD=content_field
Property: ATTRIBUTE_PREFIX=attr.
XML definition
{code}
some text
some nested text
some other nested text
{code}
Schema definition
{code}
{
"name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{
"name": "field1",
"type": {
"name": "NestedRecord",
"type": "record",
"fields" : [
{ "name": "attr.attribute", "type": "string" },
{ "name": "content_field", "type": "string" }
]
}
},
{
"name": "field2",
"type": {
"name": "NestedRecord",
"type": "record",
"fields" : [
{ "name": "attr.attribute", "type": "string" },
{ "name": "nested1", "type": "string" },
{ "name": "nested2", "type": "string" }
]
}
}
]
}
{code}
What do you say?
was (Author: jope):
[~alopresto]:
Started implementing an XML Record Reader. Shall I create a separate ticket
for this?
Similar to the JSON readers, the XML reader will expect either a single record
(e. g. content ... ) or an array of
records (e. g. content ...
... )
The reader will be aligned with common transformators. "Normal" fields (e. g.
String, Integer) can be described by simple key-value pairs:
XML definition
{code}
content
123
{code}
Schema definition
{code}
{ "name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{ "name": "field1", "type": "string" },
{ "name": "field2", "type": "int" }
]
}
{code}
Parsing of attributes or nested fields require the definition of nested records
and a field name for the content (optional, a prefix for attributes can be
defined):
Property: CONTENT_FIELD=content_field
Property: ATTRIBUTE_PREFIX=attr.
XML definition
some text
some nested text
some other nested text
Schema definition
{
"name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{
"name": "field1",
"type":
{ "name": "NestedRecord", "type": "record", "fields"
: [ \\{"name": "attr.attribute", "type": "string"}
,
\{"name": "content_field", "type": "string"}
]
}
},
{
"name": "field2",
"type":
{ "name": "NestedRecord", "type": "record", "fields" : [
\\{"name": "attr.attribute", "type": "string"}
,
\{"name": "nested1", "type": "string"},
\{"name": "nested2", "type": "string"}
]
}
}
]
}
What do you say?
> Add XML record reader & writer services
> ---
>
> Key: NIFI-4185
> URL: https://issues.apache.org/jira/browse/NIFI-4185
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Andy LoPresto
>Priority: Major
> Labels: json, records, xml
>
> With the addition of the {{RecordReader}} and {{RecordSetWriter}} paradigm,
> XML conversion has not yet been targeted. This will replace the previous
> ticket for XML to JSON conversion.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4185) Add XML record reader & writer services
[
https://issues.apache.org/jira/browse/NIFI-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394487#comment-16394487
]
Johannes Peter edited comment on NIFI-4185 at 3/11/18 12:39 PM:
[~alopresto]:
Started implementing an XML Record Reader. Shall I create a separate ticket
for this?
Similar to the JSON readers, the XML reader will expect either a single record
(e. g. content ... ) or an array of
records (e. g. content ...
... )
The reader will be aligned with common transformators. "Normal" fields (e. g.
String, Integer) can be described by simple key-value pairs:
XML definition
{code}
content
123
{code}
Schema definition
{code}
{ "name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{ "name": "field1", "type": "string" },
{ "name": "field2", "type": "int" }
]
}
{code}
Parsing of attributes or nested fields require the definition of nested records
and a field name for the content (optional, a prefix for attributes can be
defined):
Property: CONTENT_FIELD=content_field
Property: ATTRIBUTE_PREFIX=attr.
XML definition
some text
some nested text
some other nested text
Schema definition
{
"name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{
"name": "field1",
"type":
{ "name": "NestedRecord", "type": "record", "fields"
: [ \\{"name": "attr.attribute", "type": "string"}
,
\{"name": "content_field", "type": "string"}
]
}
},
{
"name": "field2",
"type":
{ "name": "NestedRecord", "type": "record", "fields" : [
\\{"name": "attr.attribute", "type": "string"}
,
\{"name": "nested1", "type": "string"},
\{"name": "nested2", "type": "string"}
]
}
}
]
}
What do you say?
was (Author: jope):
[~alopresto]:
Started implementing an XML Record Reader. Shall I create a separate ticket
for this?
Similar to the JSON readers, the XML reader will expect either a single record
(e. g. content ... ) or an array of
records (e. g. content ...
... )
The reader will be aligned with common transformators. "Normal" fields (e. g.
String, Integer) can be described by simple key-value pairs:
XML definition
{code}
content
123
{code}
Schema definition
{code}
{ "name": "testschema", "namespace": "nifi", "type": "record",
"fields": [ { "name": "field1", "type": "string" }, { "name": "field2", "type":
"int" } ] }
{code}
Parsing of attributes or nested fields require the definition of nested records
and a field name for the content (optional, a prefix for attributes can be
defined):
Property: CONTENT_FIELD=content_field
Property: ATTRIBUTE_PREFIX=attr.
XML definition
some text
some nested text
some other nested text
Schema definition
{
"name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{
"name": "field1",
"type":
{ "name": "NestedRecord", "type": "record", "fields"
: [ \\{"name": "attr.attribute", "type": "string"}
,
\{"name": "content_field", "type": "string"}
]
}
},
{
"name": "field2",
"type":
{ "name": "NestedRecord", "type": "record", "fields" : [
\\{"name": "attr.attribute", "type": "string"}
,
\{"name": "nested1", "type": "string"},
\{"name": "nested2", "type": "string"}
]
}
}
]
}
What do you say?
> Add XML record reader & writer services
> ---
>
> Key: NIFI-4185
> URL: https://issues.apache.org/jira/browse/NIFI-4185
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Andy LoPresto
>Priority: Major
> Labels: json, records, xml
>
> With the addition of the {{RecordReader}} and {{RecordSetWriter}} paradigm,
> XML conversion has not yet been targeted. This will replace the previous
> ticket for XML to JSON conversion.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4185) Add XML record reader & writer services
[
https://issues.apache.org/jira/browse/NIFI-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394487#comment-16394487
]
Johannes Peter edited comment on NIFI-4185 at 3/11/18 12:38 PM:
[~alopresto]:
Started implementing an XML Record Reader. Shall I create a separate ticket
for this?
Similar to the JSON readers, the XML reader will expect either a single record
(e. g. content ... ) or an array of
records (e. g. content ...
... )
The reader will be aligned with common transformators. "Normal" fields (e. g.
String, Integer) can be described by simple key-value pairs:
XML definition
{code}
content
123
{code}
Schema definition
{code}
{ "name": "testschema", "namespace": "nifi", "type": "record",
"fields": [ { "name": "field1", "type": "string" }, { "name": "field2", "type":
"int" } ] }
{code}
Parsing of attributes or nested fields require the definition of nested records
and a field name for the content (optional, a prefix for attributes can be
defined):
Property: CONTENT_FIELD=content_field
Property: ATTRIBUTE_PREFIX=attr.
XML definition
some text
some nested text
some other nested text
Schema definition
{
"name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{
"name": "field1",
"type":
{ "name": "NestedRecord", "type": "record", "fields"
: [ \\{"name": "attr.attribute", "type": "string"}
,
\{"name": "content_field", "type": "string"}
]
}
},
{
"name": "field2",
"type":
{ "name": "NestedRecord", "type": "record", "fields" : [
\\{"name": "attr.attribute", "type": "string"}
,
\{"name": "nested1", "type": "string"},
\{"name": "nested2", "type": "string"}
]
}
}
]
}
What do you say?
was (Author: jope):
[~alopresto]:
Started implementing an XML Record Reader. Shall I create a separate ticket
for this?
Similar to the JSON readers, the XML reader will expect either a single record
(e. g. content ... ) or an array of
records (e. g. content ...
... )
The reader will be aligned with common transformators. "Normal" fields (e. g.
String, Integer) can be described by simple key-value pairs:
XML definition
content
123
Schema definition
{code}
{ "name": "testschema", "namespace": "nifi", "type": "record",
"fields": [ \{ "name": "field1", "type": "string" }, \{ "name": "field2",
"type": "int" } ] }
{code}
Parsing of attributes or nested fields require the definition of nested records
and a field name for the content (optional, a prefix for attributes can be
defined):
Property: CONTENT_FIELD=content_field
Property: ATTRIBUTE_PREFIX=attr.
XML definition
some text
some nested text
some other nested text
Schema definition
{
"name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{
"name": "field1",
"type":
{ "name": "NestedRecord", "type": "record", "fields"
: [ \\{"name": "attr.attribute", "type": "string"}
,
\{"name": "content_field", "type": "string"}
]
}
},
{
"name": "field2",
"type":
{ "name": "NestedRecord", "type": "record", "fields" : [
\\{"name": "attr.attribute", "type": "string"}
,
\{"name": "nested1", "type": "string"},
\{"name": "nested2", "type": "string"}
]
}
}
]
}
What do you say?
> Add XML record reader & writer services
> ---
>
> Key: NIFI-4185
> URL: https://issues.apache.org/jira/browse/NIFI-4185
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Andy LoPresto
>Priority: Major
> Labels: json, records, xml
>
> With the addition of the {{RecordReader}} and {{RecordSetWriter}} paradigm,
> XML conversion has not yet been targeted. This will replace the previous
> ticket for XML to JSON conversion.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4185) Add XML record reader & writer services
[
https://issues.apache.org/jira/browse/NIFI-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394487#comment-16394487
]
Johannes Peter edited comment on NIFI-4185 at 3/11/18 12:38 PM:
[~alopresto]:
Started implementing an XML Record Reader. Shall I create a separate ticket
for this?
Similar to the JSON readers, the XML reader will expect either a single record
(e. g. content ... ) or an array of
records (e. g. content ...
... )
The reader will be aligned with common transformators. "Normal" fields (e. g.
String, Integer) can be described by simple key-value pairs:
XML definition
content
123
Schema definition
{code}
{ "name": "testschema", "namespace": "nifi", "type": "record",
"fields": [ \{ "name": "field1", "type": "string" }, \{ "name": "field2",
"type": "int" } ] }
{code}
Parsing of attributes or nested fields require the definition of nested records
and a field name for the content (optional, a prefix for attributes can be
defined):
Property: CONTENT_FIELD=content_field
Property: ATTRIBUTE_PREFIX=attr.
XML definition
some text
some nested text
some other nested text
Schema definition
{
"name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{
"name": "field1",
"type":
{ "name": "NestedRecord", "type": "record", "fields"
: [ \\{"name": "attr.attribute", "type": "string"}
,
\{"name": "content_field", "type": "string"}
]
}
},
{
"name": "field2",
"type":
{ "name": "NestedRecord", "type": "record", "fields" : [
\\{"name": "attr.attribute", "type": "string"}
,
\{"name": "nested1", "type": "string"},
\{"name": "nested2", "type": "string"}
]
}
}
]
}
What do you say?
was (Author: jope):
[~alopresto]:
Started implementing an XML Record Reader. Shall I create a separate ticket
for this?
Similar to the JSON readers, the XML reader will expect either a single record
(e. g. content ... ) or an array of
records (e. g. content ...
... )
The reader will be aligned with common transformators. "Normal" fields (e. g.
String, Integer) can be described by simple key-value pairs:
XML definition
content
123
Schema definition
```json
{ "name": "testschema", "namespace": "nifi", "type": "record",
"fields": [ \{ "name": "field1", "type": "string" }, \{ "name": "field2",
"type": "int" } ] }
```
Parsing of attributes or nested fields require the definition of nested records
and a field name for the content (optional, a prefix for attributes can be
defined):
Property: CONTENT_FIELD=content_field
Property: ATTRIBUTE_PREFIX=attr.
XML definition
some text
some nested text
some other nested text
Schema definition
{
"name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{
"name": "field1",
"type":
{ "name": "NestedRecord", "type": "record", "fields"
: [ \\{"name": "attr.attribute", "type": "string"}
,
\{"name": "content_field", "type": "string"}
]
}
},
{
"name": "field2",
"type":
{ "name": "NestedRecord", "type": "record", "fields" : [
\\{"name": "attr.attribute", "type": "string"}
,
\{"name": "nested1", "type": "string"},
\{"name": "nested2", "type": "string"}
]
}
}
]
}
What do you say?
> Add XML record reader & writer services
> ---
>
> Key: NIFI-4185
> URL: https://issues.apache.org/jira/browse/NIFI-4185
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Andy LoPresto
>Priority: Major
> Labels: json, records, xml
>
> With the addition of the {{RecordReader}} and {{RecordSetWriter}} paradigm,
> XML conversion has not yet been targeted. This will replace the previous
> ticket for XML to JSON conversion.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4185) Add XML record reader & writer services
[
https://issues.apache.org/jira/browse/NIFI-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394487#comment-16394487
]
Johannes Peter edited comment on NIFI-4185 at 3/11/18 12:37 PM:
[~alopresto]:
Started implementing an XML Record Reader. Shall I create a separate ticket
for this?
Similar to the JSON readers, the XML reader will expect either a single record
(e. g. content ... ) or an array of
records (e. g. content ...
... )
The reader will be aligned with common transformators. "Normal" fields (e. g.
String, Integer) can be described by simple key-value pairs:
XML definition
content
123
Schema definition
```json
{ "name": "testschema", "namespace": "nifi", "type": "record",
"fields": [ \{ "name": "field1", "type": "string" }, \{ "name": "field2",
"type": "int" } ] }
```
Parsing of attributes or nested fields require the definition of nested records
and a field name for the content (optional, a prefix for attributes can be
defined):
Property: CONTENT_FIELD=content_field
Property: ATTRIBUTE_PREFIX=attr.
XML definition
some text
some nested text
some other nested text
Schema definition
{
"name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{
"name": "field1",
"type":
{ "name": "NestedRecord", "type": "record", "fields"
: [ \\{"name": "attr.attribute", "type": "string"}
,
\{"name": "content_field", "type": "string"}
]
}
},
{
"name": "field2",
"type":
{ "name": "NestedRecord", "type": "record", "fields" : [
\\{"name": "attr.attribute", "type": "string"}
,
\{"name": "nested1", "type": "string"},
\{"name": "nested2", "type": "string"}
]
}
}
]
}
What do you say?
was (Author: jope):
[~alopresto]:
Started implementing an XML Record Reader. Shall I create a separate ticket
for this?
Similar to the JSON readers, the XML reader will expect either a single record
(e. g. content ... ) or an array of
records (e. g. content ...
... )
The reader will be aligned with common transformators. "Normal" fields (e. g.
String, Integer) can be described by simple key-value pairs:
XML definition
content
123
Schema definition
{ "name": "testschema", "namespace": "nifi", "type": "record",
"fields": [ \{ "name": "field1", "type": "string" }, \{ "name": "field2",
"type": "int" } ] }
Parsing of attributes or nested fields require the definition of nested records
and a field name for the content (optional, a prefix for attributes can be
defined):
Property: CONTENT_FIELD=content_field
Property: ATTRIBUTE_PREFIX=attr.
XML definition
some text
some nested text
some other nested text
Schema definition
{
"name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{
"name": "field1",
"type":
{ "name": "NestedRecord", "type": "record", "fields"
: [ \\{"name": "attr.attribute", "type": "string"}
,
\{"name": "content_field", "type": "string"}
]
}
},
{
"name": "field2",
"type":
{ "name": "NestedRecord", "type": "record", "fields" : [
\\{"name": "attr.attribute", "type": "string"}
,
\{"name": "nested1", "type": "string"},
\{"name": "nested2", "type": "string"}
]
}
}
]
}
What do you say?
> Add XML record reader & writer services
> ---
>
> Key: NIFI-4185
> URL: https://issues.apache.org/jira/browse/NIFI-4185
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Andy LoPresto
>Priority: Major
> Labels: json, records, xml
>
> With the addition of the {{RecordReader}} and {{RecordSetWriter}} paradigm,
> XML conversion has not yet been targeted. This will replace the previous
> ticket for XML to JSON conversion.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4185) Add XML record reader & writer services
[
https://issues.apache.org/jira/browse/NIFI-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394487#comment-16394487
]
Johannes Peter edited comment on NIFI-4185 at 3/11/18 12:36 PM:
[~alopresto]:
Started implementing an XML Record Reader. Shall I create a separate ticket
for this?
Similar to the JSON readers, the XML reader will expect either a single record
(e. g. content ... ) or an array of
records (e. g. content ...
... )
The reader will be aligned with common transformators. "Normal" fields (e. g.
String, Integer) can be described by simple key-value pairs:
XML definition
content
123
Schema definition
{ "name": "testschema", "namespace": "nifi", "type": "record",
"fields": [ \{ "name": "field1", "type": "string" }, \{ "name": "field2",
"type": "int" } ] }
Parsing of attributes or nested fields require the definition of nested records
and a field name for the content (optional, a prefix for attributes can be
defined):
Property: CONTENT_FIELD=content_field
Property: ATTRIBUTE_PREFIX=attr.
XML definition
some text
some nested text
some other nested text
Schema definition
{
"name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{
"name": "field1",
"type":
{ "name": "NestedRecord", "type": "record", "fields"
: [ \\{"name": "attr.attribute", "type": "string"}
,
\{"name": "content_field", "type": "string"}
]
}
},
{
"name": "field2",
"type":
{ "name": "NestedRecord", "type": "record", "fields" : [
\\{"name": "attr.attribute", "type": "string"}
,
\{"name": "nested1", "type": "string"},
\{"name": "nested2", "type": "string"}
]
}
}
]
}
What do you say?
was (Author: jope):
[~alopresto]:
Started implementing an XML Record Reader. Shall I create a separate ticket
for this?
Similar to the JSON readers, the XML reader will expect either a single record
(e. g. content ... ) or an array of
records (e. g. content ...
... )
The reader will be aligned with common transformators. "Normal" fields (e. g.
String, Integer) can be described by simple key-value pairs:
XML definition
content
123
Schema definition
{ "name": "testschema",
"namespace": "nifi",
"type": "record", "fields": [
\{ "name": "field1", "type": "string" },
\{ "name": "field2", "type": "int" }
]
}
Parsing of attributes or nested fields require the definition of nested records
and a field name for the content (optional, a prefix for attributes can be
defined):
Property: CONTENT_FIELD=content_field
Property: ATTRIBUTE_PREFIX=attr.
XML definition
some text
some nested text
some other nested text
Schema definition
{
"name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{
"name": "field1",
"type": {
"name": "NestedRecord",
"type": "record",
"fields" : [
\{"name": "attr.attribute", "type": "string"},
\{"name": "content_field", "type": "string"}
]
}
},
{
"name": "field2",
"type": {
"name": "NestedRecord",
"type": "record",
"fields" : [
\{"name": "attr.attribute", "type": "string"},
\{"name": "nested1", "type": "string"},
\{"name": "nested2", "type": "string"}
]
}
}
]
}
What do you say?
> Add XML record reader & writer services
> ---
>
> Key: NIFI-4185
> URL: https://issues.apache.org/jira/browse/NIFI-4185
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Andy LoPresto
>Priority: Major
> Labels: json, records, xml
>
> With the addition of the {{RecordReader}} and {{RecordSetWriter}} paradigm,
> XML conversion has not yet been targeted. This will replace the previous
> ticket for XML to JSON conversion.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4185) Add XML record reader & writer services
[
https://issues.apache.org/jira/browse/NIFI-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394487#comment-16394487
]
Johannes Peter edited comment on NIFI-4185 at 3/11/18 12:34 PM:
[~alopresto]:
Started implementing an XML Record Reader. Shall I create a separate ticket
for this?
Similar to the JSON readers, the XML reader will expect either a single record
(e. g. content ... ) or an array of
records (e. g. content ...
... )
The reader will be aligned with common transformators. "Normal" fields (e. g.
String, Integer) can be described by simple key-value pairs:
XML definition
content
123
Schema definition
{ "name": "testschema",
"namespace": "nifi",
"type": "record", "fields": [
\{ "name": "field1", "type": "string" },
\{ "name": "field2", "type": "int" }
]
}
Parsing of attributes or nested fields require the definition of nested records
and a field name for the content (optional, a prefix for attributes can be
defined):
Property: CONTENT_FIELD=content_field
Property: ATTRIBUTE_PREFIX=attr.
XML definition
some text
some nested text
some other nested text
Schema definition
{
"name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{
"name": "field1",
"type": {
"name": "NestedRecord",
"type": "record",
"fields" : [
\{"name": "attr.attribute", "type": "string"},
\{"name": "content_field", "type": "string"}
]
}
},
{
"name": "field2",
"type": {
"name": "NestedRecord",
"type": "record",
"fields" : [
\{"name": "attr.attribute", "type": "string"},
\{"name": "nested1", "type": "string"},
\{"name": "nested2", "type": "string"}
]
}
}
]
}
What do you say?
was (Author: jope):
[~alopresto]:
Started implementing an XML Record Reader. Shall I create a separate ticket
for this?
Similar to the JSON readers, the XML reader will expect either a single record
(e. g. content ... ) or an array of
records (e. g. content ...
... )
The reader will be aligned with common transformators. "Normal" fields (e. g.
String, Integer) can be described by simple key-value pairs:
XML definition
content
123
Schema definition
{
"name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{ "name": "field1", "type": "string" }
,
{ "name": "field2", "type": "int" }
]
}
Parsing of attributes or nested fields require the definition of nested records
and a field name for the content (optional, a prefix for attributes can be
defined):
Property: CONTENT_FIELD=content_field
Property: ATTRIBUTE_PREFIX=attr.
XML definition
some text
some nested text
some other nested text
Schema definition
{
"name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{
"name": "field1",
"type": {
"name": "NestedRecord",
"type": "record",
"fields" : [
{"name": "attr.attribute", "type": "string"}
,
{"name": "content_field", "type": "string"}
]
}
},
{
"name": "field2",
"type": {
"name": "NestedRecord",
"type": "record",
"fields" : [
{"name": "attr.attribute", "type": "string"}
,
{"name": "nested1", "type": "string"}
,
{"name": "nested2", "type": "string"}
]
}
}
]
}
What do you say?
> Add XML record reader & writer services
> ---
>
> Key: NIFI-4185
> URL: https://issues.apache.org/jira/browse/NIFI-4185
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Andy LoPresto
>Priority: Major
> Labels: json, records, xml
>
> With the addition of the {{RecordReader}} and {{RecordSetWriter}} paradigm,
> XML conversion has not yet been targeted. This will replace the previous
> ticket for XML to JSON conversion.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4185) Add XML record reader & writer services
[
https://issues.apache.org/jira/browse/NIFI-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394487#comment-16394487
]
Johannes Peter edited comment on NIFI-4185 at 3/11/18 12:29 PM:
[~alopresto]:
Started implementing an XML Record Reader. Shall I create a separate ticket
for this?
Similar to the JSON readers, the XML reader will expect either a single record
(e. g. content ... ) or an array of
records (e. g. content ...
... )
The reader will be aligned with common transformators. "Normal" fields (e. g.
String, Integer) can be described by simple key-value pairs:
XML definition
content
123
Schema definition
{
"name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{ "name": "field1", "type": "string" }
,
{ "name": "field2", "type": "int" }
]
}
Parsing of attributes or nested fields require the definition of nested records
and a field name for the content (optional, a prefix for attributes can be
defined):
Property: CONTENT_FIELD=content_field
Property: ATTRIBUTE_PREFIX=attr.
XML definition
some text
some nested text
some other nested text
Schema definition
{
"name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{
"name": "field1",
"type": {
"name": "NestedRecord",
"type": "record",
"fields" : [
{"name": "attr.attribute", "type": "string"}
,
{"name": "content_field", "type": "string"}
]
}
},
{
"name": "field2",
"type": {
"name": "NestedRecord",
"type": "record",
"fields" : [
{"name": "attr.attribute", "type": "string"}
,
{"name": "nested1", "type": "string"}
,
{"name": "nested2", "type": "string"}
]
}
}
]
}
What do you say?
was (Author: jope):
[~alopresto]:
Started implementing an XML Record Reader. Shall I create a separate ticket
for this?
Similar to the JSON readers, the XML reader will expect either a single record
(e. g. content ... ) or an array of
records (e. g. content ...
... )
The reader will be aligned with common transformators. "Normal" fields (e. g.
String, Integer) can be described by simple key-value pairs:
XML definition
content
123
Schema definition
{
"name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{ "name": "field1", "type": "string" },
{ "name": "field2", "type": "int" }
]
}
Parsing of attributes or nested fields require the definition of nested records
and a field name for the content (optional, a prefix for attributes can be
defined):
Property: CONTENT_FIELD=content_field
Property: ATTRIBUTE_PREFIX=attr.
XML definition
some text
some nested text
some other nested text
Schema definition
{
"name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{
"name": "field1",
"type": {
"name": "NestedRecord",
"type": "record",
"fields" : [
{"name": "attr.attribute", "type":
"string"},
{"name": "content_field", "type":
"string"}
]
}
},
{
"name": "field2",
"type": {
"name": "NestedRecord",
"type": "record",
"fields" : [
{"name": "attr.attribute", "type":
"string"},
{"name": "nested1", "type": "string"},
{"name": "nested2", "type": "string"}
]
}
}
]
}
What do you say?
> Add XML record reader & writer services
> ---
>
> Key: NIFI-4185
> URL: https://issues.apache.org/jira/browse/NIFI-4185
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Andy LoPresto
>Priority: Major
> Labels: json, records, xml
>
> With the addition of the {{RecordReader}} and {{RecordSetWriter}} paradigm,
> XML conversion has not yet been targeted. This will replace the previous
> ticket for XML to JSON conversion.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4185) Add XML record reader & writer services
[
https://issues.apache.org/jira/browse/NIFI-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394487#comment-16394487
]
Johannes Peter commented on NIFI-4185:
--
[~alopresto]:
Started implementing an XML Record Reader. Shall I create a separate ticket
for this?
Similar to the JSON readers, the XML reader will expect either a single record
(e. g. content ... ) or an array of
records (e. g. content ...
... )
The reader will be aligned with common transformators. "Normal" fields (e. g.
String, Integer) can be described by simple key-value pairs:
XML definition
content
123
Schema definition
{
"name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{ "name": "field1", "type": "string" },
{ "name": "field2", "type": "int" }
]
}
Parsing of attributes or nested fields require the definition of nested records
and a field name for the content (optional, a prefix for attributes can be
defined):
Property: CONTENT_FIELD=content_field
Property: ATTRIBUTE_PREFIX=attr.
XML definition
some text
some nested text
some other nested text
Schema definition
{
"name": "testschema",
"namespace": "nifi",
"type": "record",
"fields": [
{
"name": "field1",
"type": {
"name": "NestedRecord",
"type": "record",
"fields" : [
{"name": "attr.attribute", "type":
"string"},
{"name": "content_field", "type":
"string"}
]
}
},
{
"name": "field2",
"type": {
"name": "NestedRecord",
"type": "record",
"fields" : [
{"name": "attr.attribute", "type":
"string"},
{"name": "nested1", "type": "string"},
{"name": "nested2", "type": "string"}
]
}
}
]
}
What do you say?
> Add XML record reader & writer services
> ---
>
> Key: NIFI-4185
> URL: https://issues.apache.org/jira/browse/NIFI-4185
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Andy LoPresto
>Priority: Major
> Labels: json, records, xml
>
> With the addition of the {{RecordReader}} and {{RecordSetWriter}} paradigm,
> XML conversion has not yet been targeted. This will replace the previous
> ticket for XML to JSON conversion.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
