[jira] [Updated] (NIFI-1942) Create a processor to validate CSV against a user-supplied schema
[ https://issues.apache.org/jira/browse/NIFI-1942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-1942: - Attachment: ValidateCSV.xml Template to validate the processor. > Create a processor to validate CSV against a user-supplied schema > - > > Key: NIFI-1942 > URL: https://issues.apache.org/jira/browse/NIFI-1942 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Minor > Fix For: 1.0.0 > > Attachments: ValidateCSV.xml > > > In order to extend the set of "quality control" processors, it would be > interesting to have a processor validating CSV formatted flow files against a > user-specified schema. > Flow file validated against schema would be routed to "valid" relationship > although flow file not validated against schema would be routed to "invalid" > relationship. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1972) New processor PutIgniteCache
[ https://issues.apache.org/jira/browse/NIFI-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-1972: - Resolution: Fixed Status: Resolved (was: Patch Available) > New processor PutIgniteCache > > > Key: NIFI-1972 > URL: https://issues.apache.org/jira/browse/NIFI-1972 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 0.6.1 > Environment: All >Reporter: Mans Singh >Assignee: Mans Singh > Labels: cache,, ignite,, put,, streaming, > Fix For: 1.0.0 > > Original Estimate: 48h > Remaining Estimate: 48h > > Create Nifi Ignite Processor to insert data into Apache Ignite Cache -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1965) Create a QueryDNS processor
[ https://issues.apache.org/jira/browse/NIFI-1965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412413#comment-15412413 ] Pierre Villard commented on NIFI-1965: -- Merged to master, thanks [~trixpan]! > Create a QueryDNS processor > --- > > Key: NIFI-1965 > URL: https://issues.apache.org/jira/browse/NIFI-1965 > Project: Apache NiFi > Issue Type: Bug >Reporter: Andre > Fix For: 1.0.0 > > > As part of a data pipeline security teams frequently must enrich data using > DNS enabled APIs such as: > ShadowServer BGP and ASN lookup via DNS > https://www.shadowserver.org/wiki/pmwiki.php/Services/IP-BGP#toc7 > Team Cymru Malware Hash Registry > http://www.team-cymru.org/MHR.html > Spamhaus (SBL, XBL, etc) > and others > QueryDNS will use an expression language enabled property to run a query > against DNS and add the raw result to an attribute (for later processing if > necessary). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-1965) Create a QueryDNS processor
[ https://issues.apache.org/jira/browse/NIFI-1965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-1965. -- Resolution: Fixed Fix Version/s: 1.0.0 > Create a QueryDNS processor > --- > > Key: NIFI-1965 > URL: https://issues.apache.org/jira/browse/NIFI-1965 > Project: Apache NiFi > Issue Type: Bug >Reporter: Andre > Fix For: 1.0.0 > > > As part of a data pipeline security teams frequently must enrich data using > DNS enabled APIs such as: > ShadowServer BGP and ASN lookup via DNS > https://www.shadowserver.org/wiki/pmwiki.php/Services/IP-BGP#toc7 > Team Cymru Malware Hash Registry > http://www.team-cymru.org/MHR.html > Spamhaus (SBL, XBL, etc) > and others > QueryDNS will use an expression language enabled property to run a query > against DNS and add the raw result to an attribute (for later processing if > necessary). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-2516) Extract version info into parent pom, upgrade to commons-io 2.5
[ https://issues.apache.org/jira/browse/NIFI-2516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-2516. -- Resolution: Fixed Fix Version/s: 1.0.0 > Extract version info into parent pom, upgrade to commons-io 2.5 > --- > > Key: NIFI-2516 > URL: https://issues.apache.org/jira/browse/NIFI-2516 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Bryan Rosander >Assignee: Bryan Rosander > Fix For: 1.0.0 > > > Parent pom at root of nifi project should contain the dependency versions. > commons-io 2.5 is required for its BoundedReader which facilitates putting a > cap on the amount of bytes read during the payload deserialization. This is > useful in avoiding an arbitrarily large payload sent by a malicious client. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2516) Extract version info into parent pom, upgrade to commons-io 2.5
[ https://issues.apache.org/jira/browse/NIFI-2516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414213#comment-15414213 ] Pierre Villard commented on NIFI-2516: -- Merged into master, thanks [~bryanrosan...@gmail.com]! > Extract version info into parent pom, upgrade to commons-io 2.5 > --- > > Key: NIFI-2516 > URL: https://issues.apache.org/jira/browse/NIFI-2516 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Bryan Rosander >Assignee: Bryan Rosander > Fix For: 1.0.0 > > > Parent pom at root of nifi project should contain the dependency versions. > commons-io 2.5 is required for its BoundedReader which facilitates putting a > cap on the amount of bytes read during the payload deserialization. This is > useful in avoiding an arbitrarily large payload sent by a malicious client. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2521) License and Notice - TestSNMPAgent classes appear to be copied from external sources
[ https://issues.apache.org/jira/browse/NIFI-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414114#comment-15414114 ] Pierre Villard commented on NIFI-2521: -- [~joewitt] it clearly seems that there is more digging required from me to ensure we are safe here. Besides there is a unit test failing time to time. At this point, with releases coming, the safest way to move forward is to remove the tests. I'll submit a new PR shortly to address the tests of this bundle. In the meantime, the PR you submitted LGTM. +1 > License and Notice - TestSNMPAgent classes appear to be copied from external > sources > > > Key: NIFI-2521 > URL: https://issues.apache.org/jira/browse/NIFI-2521 > Project: Apache NiFi > Issue Type: Bug >Reporter: Joseph Witt >Assignee: Pierre Villard >Priority: Blocker > Fix For: 1.0.0, 0.8.0 > > > TestSNMPAgentV1 and TestSNMPAgentV2c appear to be sourced at least in large > part from https://code.google.com/archive/p/springside-sub/ > TestSNMPAgentV3 appears to be sourced from XYZ as found here > https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-snmp-bundle/nifi-snmp-processors/src/test/java/org/apache/nifi/snmp/processors/TestSnmpAgentV3.java#L68. > These classes should be completely removed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2577) Increase default ORC stripe size to 64 MB in ConvertAvroToORC
[ https://issues.apache.org/jira/browse/NIFI-2577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-2577: - Resolution: Fixed Status: Resolved (was: Patch Available) Merged into master, thanks. > Increase default ORC stripe size to 64 MB in ConvertAvroToORC > - > > Key: NIFI-2577 > URL: https://issues.apache.org/jira/browse/NIFI-2577 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.0.0 >Reporter: Matt Burgess >Assignee: Matt Burgess > Fix For: 1.0.0 > > > The current default stripe size for the ConvertAvroToORC processor is 100KB > which would cause performance issues for the normal use case. Suggestion is > to increase it to 64MB, which is what the default (for Apache ORC) is > currently specified as: > https://orc.apache.org/docs/hive-config.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] (NIFI-1962) NPE in Expression Language toDate()
Title: Message Title Pierre Villard updated NIFI-1962 Apache NiFi / NIFI-1962 NPE in _expression_ Language toDate() Change By: Pierre Villard Resolution: Fixed Fix Version/s: 1.2.0 Status: Patch Available Resolved Add Comment This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d)
[jira] (NIFI-3179) MergeContent extracts demarcator property value bytes without specifying charset encoding
Title: Message Title Pierre Villard updated NIFI-3179 Apache NiFi / NIFI-3179 MergeContent extracts demarcator property value bytes without specifying charset encoding Change By: Pierre Villard Resolution: Fixed Status: Patch Available Resolved Add Comment This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d)
[jira] [Resolved] (NIFI-3408) Add an attribute for failure reason to InvokeHTTP in all cases to match docs
[ https://issues.apache.org/jira/browse/NIFI-3408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-3408. -- Resolution: Fixed Fix Version/s: 1.2.0 > Add an attribute for failure reason to InvokeHTTP in all cases to match docs > > > Key: NIFI-3408 > URL: https://issues.apache.org/jira/browse/NIFI-3408 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joseph Percivall >Priority: Minor > Fix For: 1.2.0 > > > Originally brought up on the mailing list[1], the InvokeHTTP docs state: > ``` > The original FlowFile will be routed on any type of connection failure, > timeout or general exception. It will have new attributes detailing the > request. > ``` > Adding attributes currently only happens though when a response is received > from the server, so not attributes if it fails to connect or the socket times > out. > These attributes should be added. > [1] > http://apache-nifi-users-list.2361937.n4.nabble.com/InvokeHTTP-and-SocketTimeoutException-td743.html -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-1125) ERROR [NiFi Web Server-22655] o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred: java.lang.NullPointerException
[ https://issues.apache.org/jira/browse/NIFI-1125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-1125: - Resolution: Fixed Fix Version/s: 1.2.0 Status: Resolved (was: Patch Available) > ERROR [NiFi Web Server-22655] o.a.nifi.web.api.config.ThrowableMapper An > unexpected error has occurred: java.lang.NullPointerException > -- > > Key: NIFI-1125 > URL: https://issues.apache.org/jira/browse/NIFI-1125 > Project: Apache NiFi > Issue Type: Bug > Components: Configuration >Affects Versions: 0.4.0 > Environment: Centos 6.7 >Reporter: Mark Petronic >Assignee: Koji Kawamura >Priority: Minor > Fix For: 1.2.0 > > Attachments: StatsIngestFlow.xml > > > Running with a latest master branch build off commit > dbf0c7893fef964bfbb3a4c039c756396587ce12. > Steps to recreate: > 1. All processors stopped > 2. Add uuid to "Attributes to Send" in the InvokeHttp Processor > 3. Save and close config dialog > 4. Open same config dialog and remove uuid field so that is is "No value set" > 5. Apply changes and web UI will crash and indicate error has occurred > and below will seen in user log > 6. Hit F5 and browser reloads UI just fine > See attached template. > 2015-11-07 19:42:58,369 ERROR [NiFi Web Server-22655] > o.a.nifi.web.api.config.ThrowableMapper An unexpected error has > occurred: java.lang.NullPointerException. Returning Internal Server > Error response. > java.lang.NullPointerException: null > at > org.apache.nifi.processors.standard.InvokeHTTP.onPropertyModified(InvokeHTTP.java:121) > ~[na:na] > at > org.apache.nifi.controller.AbstractConfiguredComponent.removeProperty(AbstractConfiguredComponent.java:163) > ~[nifi-framework-core-api-0.3.1-SNAPSHOT.jar:0.3.1-SNAPSHOT] > at > org.apache.nifi.web.dao.impl.StandardProcessorDAO.configureProcessor(StandardProcessorDAO.java:174) > ~[classes/:na] > at > org.apache.nifi.web.dao.impl.StandardProcessorDAO.updateProcessor(StandardProcessorDAO.java:391) > ~[classes/:na] > at > org.apache.nifi.web.dao.impl.StandardProcessorDAO$$FastClassBySpringCGLIB$$779e089b.invoke() > ~[spring-core-4.1.6.RELEASE.jar:na] > at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) > ~[spring-core-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:717) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.apache.nifi.audit.ProcessorAuditor.updateProcessorAdvice(ProcessorAuditor.java:120) > ~[classes/:na] > at sun.reflect.GeneratedMethodAccessor176.invoke(Unknown Source) ~[na:na] > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > ~[na:1.8.0_51] > at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_51] > at > org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:68) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:168) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:653) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.apache.nifi.web.dao.impl.StandardProcessorDAO$$EnhancerBySpringCGLIB$$f8bfa279.updateProcessor() > ~[spring-core-4.1.6.RELEASE.jar:na] > at > org.apache.nifi.web.StandardNiFiServiceFacade$2.execute(StandardNiFiServiceFacade.java:412) > ~[classes/:0.3.1-SNAPSHOT] > at > org.apache.nifi.web.StandardOptimisticLockingManager.configureFlow(StandardOptimisticLockingManager.java:83) >
[jira] [Updated] (NIFI-3434) nifi-assembly dependencies should allow users to skip the creation of ZIP and TAR.GZ packages
[ https://issues.apache.org/jira/browse/NIFI-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-3434: - Resolution: Fixed Fix Version/s: 1.2.0 Status: Resolved (was: Patch Available) > nifi-assembly dependencies should allow users to skip the creation of ZIP and > TAR.GZ packages > - > > Key: NIFI-3434 > URL: https://issues.apache.org/jira/browse/NIFI-3434 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.1.1 >Reporter: Andre >Assignee: Andre > Fix For: 1.2.0 > > > devs, > Currently calling 'mvn clean install' creates a ZIP, a TAR.GZ and a > directory containing the same code. This leads to wasted disk space and a > lot of wasted disk writes (something that a lot of folks using SSDs prefer > to avoid). > Would anyone oppose the idea of moving the ZIP and TAR.GZ assemblies into a > "release" profile (or whatever name we agree to). This way we could > maintain the directory "format" (which I suspect most of us use during > development), while still providing a convenient way of creating the ZIP > and TAR.GZ packages. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (NIFI-3447) PutSplunk - Broken Pipe - Fails to reconnect
Pierre Villard created NIFI-3447:
Summary: PutSplunk - Broken Pipe - Fails to reconnect
Key: NIFI-3447
URL: https://issues.apache.org/jira/browse/NIFI-3447
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Affects Versions: 1.1.1
Reporter: Pierre Villard
Assignee: Pierre Villard
PutSplunk processor is not able to recover from a connection loss: if the
Splunk server goes down, the processor will indefinitely fail sending data when
the server is back:
{noformat}
2017-02-07 13:17:29,044 ERROR [Timer-Driven Process Thread-2]
o.a.nifi.processors.splunk.PutSplunk
PutSplunk[id=015a101d-be60-183a-7e7e-6437f7600da5] Failed to send
StandardFlowFileRecord[uuid=940bd376-adc1-4a7a-8f86-8a635fab621a,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1485896025601-1, container=default,
section=1], offset=67695,
length=2963],offset=2582,name=nifi-app.27253-30216.log,size=128]; routing to
'failure'; last failure reason reported was java.io.IOException: Broken pipe;:
java.io.IOException: Broken pipe
2017-02-07 13:17:29,045 ERROR [Timer-Driven Process Thread-2]
o.a.nifi.processors.splunk.PutSplunk
java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcherImpl.write0(Native Method) ~[na:1.8.0_77]
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
~[na:1.8.0_77]
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
~[na:1.8.0_77]
at sun.nio.ch.IOUtil.write(IOUtil.java:65) ~[na:1.8.0_77]
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
~[na:1.8.0_77]
at
org.apache.nifi.remote.io.socket.SocketChannelOutputStream.write(SocketChannelOutputStream.java:87)
~[nifi-utils-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.nifi.remote.io.socket.SocketChannelOutputStream.write(SocketChannelOutputStream.java:76)
~[nifi-utils-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.nifi.processor.util.put.sender.SocketChannelSender.write(SocketChannelSender.java:83)
~[nifi-processor-utils-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.nifi.processor.util.put.sender.ChannelSender.send(ChannelSender.java:83)
~[nifi-processor-utils-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.nifi.processors.splunk.PutSplunk.processSingleMessage(PutSplunk.java:202)
~[nifi-splunk-processors-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.nifi.processors.splunk.PutSplunk.onTrigger(PutSplunk.java:162)
~[nifi-splunk-processors-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1099)
[nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
[nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
[nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
[nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_77]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[na:1.8.0_77]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[na:1.8.0_77]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[na:1.8.0_77]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_77]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_77]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (NIFI-1503) Include timezone on NiFI log messages
[
https://issues.apache.org/jira/browse/NIFI-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856036#comment-15856036
]
Pierre Villard commented on NIFI-1503:
--
For this one, I believe this is just a matter of logback configuration:
https://logback.qos.ch/manual/layouts.html#conversionWord
logback.xml configuration file should be updated by user to correctly define
the layout with something like (for instance):
{code}
%date{HH:mm:ss.SSS, Australia/Perth}
{code}
This would need to be updated where we have:
{code}
%date %level [%thread] %logger{40} %msg%n
{code}
But IMO, this is something that users should perform on their own when
installing NiFi, no?
> Include timezone on NiFI log messages
> -
>
> Key: NIFI-1503
> URL: https://issues.apache.org/jira/browse/NIFI-1503
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 0.4.1
>Reporter: Matthew Clarke
>Assignee: Andre
>Priority: Minor
>
> While we can stress the importance of using NTP to make sure clocks across
> multiple system running NIFi remain in sync, we cannot enforce a requirement
> that all system set their systems to a common timezone. The logs currently
> do not show the timezone on each log entry which can make it confusing for
> users when trying to trace log events across multiple NiFI instances that are
> configured for different timezones. This would be increasing difficult if
> logs were moved to a central logging server.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (NIFI-1503) Include timezone on NiFI log messages
[ https://issues.apache.org/jira/browse/NIFI-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856052#comment-15856052 ] Pierre Villard commented on NIFI-1503: -- I don't see any objection to set UTC as a default timezone. However, I believe this would be a breaking change that needs to wait a major release. If people developed flows to parse NiFi logs or configured third-party tools to analyze/monitor NiFi logs, this could be an issue, don't you think? > Include timezone on NiFI log messages > - > > Key: NIFI-1503 > URL: https://issues.apache.org/jira/browse/NIFI-1503 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 0.4.1 >Reporter: Matthew Clarke >Assignee: Andre >Priority: Minor > > While we can stress the importance of using NTP to make sure clocks across > multiple system running NIFi remain in sync, we cannot enforce a requirement > that all system set their systems to a common timezone. The logs currently > do not show the timezone on each log entry which can make it confusing for > users when trying to trace log events across multiple NiFI instances that are > configured for different timezones. This would be increasing difficult if > logs were moved to a central logging server. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3223) Allow PublishAMQP to use NiFi expression language
[ https://issues.apache.org/jira/browse/NIFI-3223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-3223: - Resolution: Fixed Status: Resolved (was: Patch Available) > Allow PublishAMQP to use NiFi expression language > - > > Key: NIFI-3223 > URL: https://issues.apache.org/jira/browse/NIFI-3223 > Project: Apache NiFi > Issue Type: Wish > Components: Extensions >Affects Versions: 0.7.0, 1.1.0, 0.7.1 >Reporter: Brian >Assignee: Oleg Zhurakousky > Fix For: 1.2.0 > > > Enable the use of NiFi expression language for the PublishAMQP processors, > Routing Key value to allow it to be better used within the NiFi workflows. > PublishAMQP fields to enable: > "Routing Key" -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (NIFI-3372) PutSQL cannot insert True for a BIT field when using ConvertJSONToSQL
[
https://issues.apache.org/jira/browse/NIFI-3372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard resolved NIFI-3372.
--
Resolution: Fixed
Fix Version/s: 1.2.0
> PutSQL cannot insert True for a BIT field when using ConvertJSONToSQL
> -
>
> Key: NIFI-3372
> URL: https://issues.apache.org/jira/browse/NIFI-3372
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.0.1
> Environment: SqlServer
>Reporter: Ryan Persaud
> Fix For: 1.2.0
>
>
> As noted in NIFI-1613, using the column width to truncate fields often yields
> incorrect and undesired results for non-string fields in ConvertJSONToSQL. I
> have encountered a situation where it is impossible to populate a BIT field
> in SqlServer with true (1) using ConvertJSONToSQL and PutSQL. The notable
> snippets of code are:
> org.apache.nifi.processors.standard.ConvertJSONToSQL (449-455):
> if (!fieldNode.isNull()) {
> String fieldValue = fieldNode.asText();
> if (colSize != null && fieldValue.length() > colSize) {
> fieldValue = fieldValue.substring(0, colSize);
> }
> attributes.put("sql.args." + fieldCount + ".value",
> fieldValue);
> }
> org.apache.nifi.processors.standard.PutSQL (757-761):
> switch (jdbcType) {
> case Types.BIT:
> case Types.BOOLEAN:
> stmt.setBoolean(parameterIndex,
> Boolean.parseBoolean(parameterValue));
> break;
> java.lang.Boolean (121-123):
> public static boolean parseBoolean(String s) {
> return ((s != null) && s.equalsIgnoreCase("true"));
> }
> In PutSQL, the case for BIT has no body or break, so execution proceeds to
> the BOOLEAN case. Here, parseBoolean() attempts to parse parameterValue into
> a boolean value. Looking at parseBoolean(), we can see that the
> parameterValue must contain "true" in order for true to be returned. Since
> the code in ConvertJSONToSQL will truncate the string to its first character,
> the string sent to PutSQL will never be equal to "true", and parseBoolean()
> will never return true.
> One easy fix for this issue (below) while ConvertJSONToSQL gets sorted out is
> to allow 1 and t (case-insensitive) to also represent true in the BIT case in
> PutSQL. Then, we can pass true/false 1/0 to ConvertJSONTOSQL, and still be
> able to correctly populate BIT columns. Since we are also ORing a call to
> parseBoolean(), this modification should not break any existing NiFi flows
> that depend on the current BIT handling of PutSQL.
> switch (jdbcType) {
> case Types.BIT:
> stmt.setBoolean(parameterIndex,
> "1".equals(parameterValue) || "t".equalsIgnoreCase(parameterValue) ||
> Boolean.parseBoolean(parameterValue));
> break;
> case Types.BOOLEAN:
> As a stopgap, I am currently using the following Python ExecuteScript
> processor in between my ConvertJSONToSQL and PutSQL processors in order to
> properly populate BIT fields:
> flowFile = session.get()
> properties = context.getProperties()
> if (flowFile != None):
> attributes = flowFile.getAttributes()
> for key in attributes.keys():
> if attributes[key] == "-7":
> value_key = key.replace(".type", ".value")
> new_value = "true" if attributes[value_key] == "1" or
> attributes[value_key].lower() == "t" or attributes[value_key].lower() ==
> "true" else "false"
> flowFile = session.putAttribute(flowFile, value_key, new_value)
> session.transfer(flowFile, REL_SUCCESS)
> I modified PutSQL and created some tests in TestPutSQL, and I'll create a PR
> shortly.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (NIFI-3372) PutSQL cannot insert True for a BIT field when using ConvertJSONToSQL
[
https://issues.apache.org/jira/browse/NIFI-3372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-3372:
-
Component/s: (was: Core Framework)
Extensions
> PutSQL cannot insert True for a BIT field when using ConvertJSONToSQL
> -
>
> Key: NIFI-3372
> URL: https://issues.apache.org/jira/browse/NIFI-3372
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.0.1
> Environment: SqlServer
>Reporter: Ryan Persaud
> Fix For: 1.2.0
>
>
> As noted in NIFI-1613, using the column width to truncate fields often yields
> incorrect and undesired results for non-string fields in ConvertJSONToSQL. I
> have encountered a situation where it is impossible to populate a BIT field
> in SqlServer with true (1) using ConvertJSONToSQL and PutSQL. The notable
> snippets of code are:
> org.apache.nifi.processors.standard.ConvertJSONToSQL (449-455):
> if (!fieldNode.isNull()) {
> String fieldValue = fieldNode.asText();
> if (colSize != null && fieldValue.length() > colSize) {
> fieldValue = fieldValue.substring(0, colSize);
> }
> attributes.put("sql.args." + fieldCount + ".value",
> fieldValue);
> }
> org.apache.nifi.processors.standard.PutSQL (757-761):
> switch (jdbcType) {
> case Types.BIT:
> case Types.BOOLEAN:
> stmt.setBoolean(parameterIndex,
> Boolean.parseBoolean(parameterValue));
> break;
> java.lang.Boolean (121-123):
> public static boolean parseBoolean(String s) {
> return ((s != null) && s.equalsIgnoreCase("true"));
> }
> In PutSQL, the case for BIT has no body or break, so execution proceeds to
> the BOOLEAN case. Here, parseBoolean() attempts to parse parameterValue into
> a boolean value. Looking at parseBoolean(), we can see that the
> parameterValue must contain "true" in order for true to be returned. Since
> the code in ConvertJSONToSQL will truncate the string to its first character,
> the string sent to PutSQL will never be equal to "true", and parseBoolean()
> will never return true.
> One easy fix for this issue (below) while ConvertJSONToSQL gets sorted out is
> to allow 1 and t (case-insensitive) to also represent true in the BIT case in
> PutSQL. Then, we can pass true/false 1/0 to ConvertJSONTOSQL, and still be
> able to correctly populate BIT columns. Since we are also ORing a call to
> parseBoolean(), this modification should not break any existing NiFi flows
> that depend on the current BIT handling of PutSQL.
> switch (jdbcType) {
> case Types.BIT:
> stmt.setBoolean(parameterIndex,
> "1".equals(parameterValue) || "t".equalsIgnoreCase(parameterValue) ||
> Boolean.parseBoolean(parameterValue));
> break;
> case Types.BOOLEAN:
> As a stopgap, I am currently using the following Python ExecuteScript
> processor in between my ConvertJSONToSQL and PutSQL processors in order to
> properly populate BIT fields:
> flowFile = session.get()
> properties = context.getProperties()
> if (flowFile != None):
> attributes = flowFile.getAttributes()
> for key in attributes.keys():
> if attributes[key] == "-7":
> value_key = key.replace(".type", ".value")
> new_value = "true" if attributes[value_key] == "1" or
> attributes[value_key].lower() == "t" or attributes[value_key].lower() ==
> "true" else "false"
> flowFile = session.putAttribute(flowFile, value_key, new_value)
> session.transfer(flowFile, REL_SUCCESS)
> I modified PutSQL and created some tests in TestPutSQL, and I'll create a PR
> shortly.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Resolved] (NIFI-3429) ConvertJSONToSQL Does not work with table identifiers that need to be quoted
[ https://issues.apache.org/jira/browse/NIFI-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-3429. -- Resolution: Fixed > ConvertJSONToSQL Does not work with table identifiers that need to be quoted > > > Key: NIFI-3429 > URL: https://issues.apache.org/jira/browse/NIFI-3429 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Nicholas Carenza >Priority: Minor > Fix For: 1.2.0 > > > I have a table identifer that needs to be quoted. If i supple the table > identifer unquoted in the properties, the processor can convert the json to > sql but the output sql doesn't have the table name quoted even if I select > the option Quote Identifiers because, true to the description, it only > applies to column identifiers. If I supply the table identifier quoted in the > properties, then the processor fails completely because it can't find the > table. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3429) ConvertJSONToSQL Does not work with table identifiers that need to be quoted
[ https://issues.apache.org/jira/browse/NIFI-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-3429: - Affects Version/s: (was: 1.2.0) Component/s: (was: Core Framework) Extensions Issue Type: Improvement (was: Bug) > ConvertJSONToSQL Does not work with table identifiers that need to be quoted > > > Key: NIFI-3429 > URL: https://issues.apache.org/jira/browse/NIFI-3429 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Nicholas Carenza >Priority: Minor > Fix For: 1.2.0 > > > I have a table identifer that needs to be quoted. If i supple the table > identifer unquoted in the properties, the processor can convert the json to > sql but the output sql doesn't have the table name quoted even if I select > the option Quote Identifiers because, true to the description, it only > applies to column identifiers. If I supply the table identifier quoted in the > properties, then the processor fails completely because it can't find the > table. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3427) Data Provenance date/time sort ignores the date
[ https://issues.apache.org/jira/browse/NIFI-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-3427: - Resolution: Fixed Fix Version/s: 1.2.0 Status: Resolved (was: Patch Available) > Data Provenance date/time sort ignores the date > --- > > Key: NIFI-3427 > URL: https://issues.apache.org/jira/browse/NIFI-3427 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.1.1 >Reporter: Justin Rittenhouse >Assignee: James Wing >Priority: Minor > Fix For: 1.2.0 > > Attachments: Screen Shot 2017-02-01 at 9.49.11 AM.png > > > The Data provenance display window that pops up doesn't properly sort by > date/time. It sorts by time, with events dated 11pm from January 31 > appearing before/after (depending on ascending or descending sort) events > dated February 1 before 11pm. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3430) PutSQL Errors attempting to coerce date string to timestamp
[
https://issues.apache.org/jira/browse/NIFI-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-3430:
-
Component/s: (was: Core Framework)
Extensions
> PutSQL Errors attempting to coerce date string to timestamp
> ---
>
> Key: NIFI-3430
> URL: https://issues.apache.org/jira/browse/NIFI-3430
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Reporter: Nicholas Carenza
>Priority: Minor
> Labels: convertjsontosql, putsql
>
> I am attempting to write a flat json object to my postgresql database using
> ConvertJSONToSQL followed by PutSQL. My json has a string date field in ISO
> 8601 format which postgres can accept as a timestamp. The column in the
> database is of type 'timestamp without time zone'.
> org.apache.nifi.processor.exception.ProcessException: The value of the
> sql.args.33.value is '2017-02-01T23:12:21+00:00', which cannot be converted
> to a timestamp
> Is PutSQL trying to do some preprocessing here that it should just let the
> database handle?
> select cast('2017-02-01T23:12:21+00:00' as timestamp without time zone);
> timestamp
> -
> 2017-02-01 23:12:21
> (1 row)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Resolved] (NIFI-3430) PutSQL Errors attempting to coerce date string to timestamp
[
https://issues.apache.org/jira/browse/NIFI-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard resolved NIFI-3430.
--
Resolution: Fixed
Fix Version/s: 1.2.0
> PutSQL Errors attempting to coerce date string to timestamp
> ---
>
> Key: NIFI-3430
> URL: https://issues.apache.org/jira/browse/NIFI-3430
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Reporter: Nicholas Carenza
>Priority: Minor
> Labels: convertjsontosql, putsql
> Fix For: 1.2.0
>
>
> I am attempting to write a flat json object to my postgresql database using
> ConvertJSONToSQL followed by PutSQL. My json has a string date field in ISO
> 8601 format which postgres can accept as a timestamp. The column in the
> database is of type 'timestamp without time zone'.
> org.apache.nifi.processor.exception.ProcessException: The value of the
> sql.args.33.value is '2017-02-01T23:12:21+00:00', which cannot be converted
> to a timestamp
> Is PutSQL trying to do some preprocessing here that it should just let the
> database handle?
> select cast('2017-02-01T23:12:21+00:00' as timestamp without time zone);
> timestamp
> -
> 2017-02-01 23:12:21
> (1 row)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (NIFI-3390) Add support for LDAP HA
[
https://issues.apache.org/jira/browse/NIFI-3390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-3390:
-
Assignee: Pierre Villard
Status: Patch Available (was: Open)
> Add support for LDAP HA
> ---
>
> Key: NIFI-3390
> URL: https://issues.apache.org/jira/browse/NIFI-3390
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>
> Update LDAP provider mechanism to support multiple LDAP servers.
> Accept:
> {noformat}
> ldap://node1:port1,ldap://node2:port2
> {noformat}
> In {{LdapProvider.java}}, update
> {noformat}
> context.setUrl(url);
> {noformat}
> to use {{setUrls(String[] urls)}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] (NIFI-3180) unable to import a flow template to NIFI 1.2 SNAPSHOT with "null" error
Title: Message Title Pierre Villard updated NIFI-3180 Apache NiFi / NIFI-3180 unable to import a flow template to NIFI 1.2 SNAPSHOT with "null" error Change By: Pierre Villard Resolution: Fixed Status: Patch Available Resolved Add Comment This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d)
[jira] (NIFI-3161) nifi-mock pulls two different versions of ASM libraries
Title: Message Title Pierre Villard updated NIFI-3161 Apache NiFi / NIFI-3161 nifi-mock pulls two different versions of ASM libraries Change By: Pierre Villard Resolution: Fixed Status: Patch Available Resolved Add Comment This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d)
[jira] [Updated] (NIFI-3393) REST API - Inconsistency on /controller/cluster/nodes/{id}
[
https://issues.apache.org/jira/browse/NIFI-3393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-3393:
-
Assignee: Pierre Villard
Status: Patch Available (was: Open)
> REST API - Inconsistency on /controller/cluster/nodes/{id}
> --
>
> Key: NIFI-3393
> URL: https://issues.apache.org/jira/browse/NIFI-3393
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Minor
>
> According to which node is requested through the REST API, the end point
> {noformat}
> /controller/cluster/nodes/{id}
> {noformat}
> will not return the same result. This is due to the fact that in
> {{ControllerResource}}, the request is not replicated to the cluster
> coordinator.
> If this is not the expected behavior, the following code should be added:
> {noformat}
> if (isReplicateRequest()) {
> return replicate(HttpMethod.GET, getClusterCoordinatorNode());
> }
> {noformat}
> At the moment, if requesting the cluster coordinator, I get something like:
> {noformat}
> {u'node': {u'status': u'CONNECTED', u'roles': [], u'nodeId':
> u'd403e2c0-44a7-4c1c-aaa4-fed2ee3d6993', u'apiPort': 8443, u'events':
> [.], u'nodeStartTime': u'01/25/2017 10:21:38 CET', u'address': u'node-1',
> u'heartbeat': u'01/25/2017 11:24:47 CET', u'queued': u'0 / 0 bytes',
> u'activeThreadCount': 0}}
> {noformat}
> And, on another node:
> {noformat}
> {u'node': {u'status': u'CONNECTED', u'roles': [], u'nodeId':
> u'd403e2c0-44a7-4c1c-aaa4-fed2ee3d6993', u'apiPort': 8443, u'address':
> u'node-1', u'events': []}}
> {noformat}
> In other words, all this part is missing:
> {noformat}
> // only connected nodes have heartbeats
> if (nodeHeartbeat != null) {
> final Date heartbeat = new Date(nodeHeartbeat.getTimestamp());
> nodeDto.setHeartbeat(heartbeat);
> nodeDto.setNodeStartTime(new
> Date(nodeHeartbeat.getSystemStartTime()));
>
> nodeDto.setActiveThreadCount(nodeHeartbeat.getActiveThreadCount());
>
> nodeDto.setQueued(FormatUtils.formatCount(nodeHeartbeat.getFlowFileCount()) +
> " / " + FormatUtils.formatDataSize(nodeHeartbeat.getFlowFileBytes()));
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (NIFI-3393) REST API - Inconsistency on /controller/cluster/nodes/{id}
Pierre Villard created NIFI-3393:
Summary: REST API - Inconsistency on /controller/cluster/nodes/{id}
Key: NIFI-3393
URL: https://issues.apache.org/jira/browse/NIFI-3393
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Reporter: Pierre Villard
Priority: Minor
According to which node is requested through the REST API, the end point
{noformat}
/controller/cluster/nodes/{id}
{noformat}
will not return the same result. This is due to the fact that in
{{ControllerResource}}, the request is not replicated to the cluster
coordinator.
If this is not the expected behavior, the following code should be added:
{noformat}
if (isReplicateRequest()) {
return replicate(HttpMethod.GET, getClusterCoordinatorNode());
}
{noformat}
At the moment, if requesting the cluster coordinator, I get something like:
{noformat}
{u'node': {u'status': u'CONNECTED', u'roles': [], u'nodeId':
u'd403e2c0-44a7-4c1c-aaa4-fed2ee3d6993', u'apiPort': 8443, u'events': [.],
u'nodeStartTime': u'01/25/2017 10:21:38 CET', u'address': u'node-1',
u'heartbeat': u'01/25/2017 11:24:47 CET', u'queued': u'0 / 0 bytes',
u'activeThreadCount': 0}}
{noformat}
And, on another node:
{noformat}
{u'node': {u'status': u'CONNECTED', u'roles': [], u'nodeId':
u'd403e2c0-44a7-4c1c-aaa4-fed2ee3d6993', u'apiPort': 8443, u'address':
u'node-1', u'events': []}}
{noformat}
In other words, all this part is missing:
{noformat}
// only connected nodes have heartbeats
if (nodeHeartbeat != null) {
final Date heartbeat = new Date(nodeHeartbeat.getTimestamp());
nodeDto.setHeartbeat(heartbeat);
nodeDto.setNodeStartTime(new
Date(nodeHeartbeat.getSystemStartTime()));
nodeDto.setActiveThreadCount(nodeHeartbeat.getActiveThreadCount());
nodeDto.setQueued(FormatUtils.formatCount(nodeHeartbeat.getFlowFileCount()) + "
/ " + FormatUtils.formatDataSize(nodeHeartbeat.getFlowFileBytes()));
}
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-3295) Node Reconnects When Deleting through API
[
https://issues.apache.org/jira/browse/NIFI-3295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-3295:
-
Assignee: Pierre Villard
Status: Patch Available (was: Open)
> Node Reconnects When Deleting through API
> -
>
> Key: NIFI-3295
> URL: https://issues.apache.org/jira/browse/NIFI-3295
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 1.0.1
>Reporter: Josh Meyer
>Assignee: Pierre Villard
>Priority: Minor
>
> When deleting a node using the API a 200 code is returned, but the node will
> be reconnected. It seems as if it is required to first disconnect the node
> and then delete then node. It would be nice if a 400 code (invalid request)
> or something like this was returned explaining the error.
> Reproduce the node being deleted and reconnecting:
> {code}
> curl -X DELETE -k -v -i
> ':9091/nifi-api/controller/cluster/nodes/' --cert
> :
> {code}
> Getting the deletion to stick:
> {code}
> curl -X PUT -k -v -i
> ':9091/nifi-api/controller/cluster/nodes/' -H 'Origin:
> :9091' -H 'Content-Type: application/json' --data-binary
> '{"node":{"nodeId":"","status":"DISCONNECTING"}}' --cert
> :
> curl -X DELETE -k -v -i
> ':9091/nifi-api/controller/cluster/nodes/' --cert
> :
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (NIFI-3397) UI - Refresh after a node joins cluster
Pierre Villard created NIFI-3397: Summary: UI - Refresh after a node joins cluster Key: NIFI-3397 URL: https://issues.apache.org/jira/browse/NIFI-3397 Project: Apache NiFi Issue Type: Bug Components: Core UI Affects Versions: 1.1.1 Reporter: Pierre Villard Priority: Minor Context : 3 nodes cluster (node-1, node-2, node-3), node-2 is Primary and Coordinator Workflow : GenerateFlowFile -> LogAttribute, all running Actions : - disconnect node-1 - stop generate flow file on node-1 - wait for queued flow files to be drained on node-1 - reconnect node-1 node-1 correctly reconnects, but if I was already on UI through node-1, the GFF appears to be STOPPED even though I right click / refresh the canvas. If looking at the UI from another node, GFF is RUNNING. I have to completely refresh the UI (on node-1) to get the last version of the workflow. GFF is correctly running on every nodes (processor status has been correctly updated when the node joined the cluster), but only the UI is not perfectly refreshed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-3409) Batch users/groups import - LDAP
Pierre Villard created NIFI-3409:
Summary: Batch users/groups import - LDAP
Key: NIFI-3409
URL: https://issues.apache.org/jira/browse/NIFI-3409
Project: Apache NiFi
Issue Type: Sub-task
Components: Core Framework, Core UI
Reporter: Pierre Villard
Assignee: Pierre Villard
Creating the sub task to answer:
{quote}
Batch user import
* Whether the users are providing client certificates, LDAP credentials, or
Kerberos tickets to authenticate, the canonical source of identity is still
managed by NiFi. I propose a mechanism to quickly define multiple users in the
system (without affording any policy assignments). Here I am looking for
substantial community input on the most common/desired use cases, but my
initial thoughts are:
** LDAP-specific
*** A manager DN and password (similar to necessary for LDAP authentication)
are used to authenticate the admin/user manager, and then a LDAP query string
(i.e. {{ou=users,dc=nifi,dc=apache,dc=org}}) is provided and the dialog
displays/API returns a list of users/groups matching the query. The admin can
then select which to import to NiFi and confirm.
{quote}
In particular the initial implementation would be to add a feature allowing to
sync users and groups with LDAP based on additional parameters given in the
login identity provider configuration file and custom filters provided by the
user through the UI.
It is not foreseen to delete users/groups that exist in NiFi but are not
retrieved in the LDAP. It'd be only creating/updating users/groups based on
what is in LDAP server.
The feature would be exposed through a new REST API endpoint. In case another
identity provider is configured (not LDAP), an unsupported operation exception
would be returned at the moment.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-3490) TLS Toolkit - define SAN in standalone mode
[ https://issues.apache.org/jira/browse/NIFI-3490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-3490: - Fix Version/s: 1.2.0 Status: Patch Available (was: Open) > TLS Toolkit - define SAN in standalone mode > --- > > Key: NIFI-3490 > URL: https://issues.apache.org/jira/browse/NIFI-3490 > Project: Apache NiFi > Issue Type: Improvement > Components: Tools and Build >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Minor > Labels: tls-toolkit > Fix For: 1.2.0 > > > Following NIFI-3331, it would be useful to have the same option (add Subject > Alternative Names in certificates) when using the TLS toolkit in standalone > mode. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (NIFI-3528) Include dynamic JAAS configuration for Kafka processors 0.10+
Pierre Villard created NIFI-3528: Summary: Include dynamic JAAS configuration for Kafka processors 0.10+ Key: NIFI-3528 URL: https://issues.apache.org/jira/browse/NIFI-3528 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Pierre Villard Kafka 0.10.2.0 has been released few days ago and introduced KAFKA-4259. It should now be possible to dynamically specify the client when using Kafka client library. Consequently, in a multi-tenant context, it won't be necessary anymore to write as a single user (defined in JAAS configuration file and loaded by the JVM) in all running Kafka processors. More details here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-85%3A+Dynamic+JAAS+configuration+for+Kafka+clients -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3508) InvokeHTTP send request body on PATCH if PROP_SEND_BODY
[ https://issues.apache.org/jira/browse/NIFI-3508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-3508: - Fix Version/s: 1.2.0 > InvokeHTTP send request body on PATCH if PROP_SEND_BODY > --- > > Key: NIFI-3508 > URL: https://issues.apache.org/jira/browse/NIFI-3508 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Nicholas Carenza >Priority: Minor > Labels: InvokeHTTP, processor > Fix For: 1.2.0 > > > I am dealing with an API that updates documents using the PATCH request > method and I need to send a request body but InvokeHTTP will only send a > request body if the request method is POST or PUT without any way to override. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3508) InvokeHTTP send request body on PATCH if PROP_SEND_BODY
[ https://issues.apache.org/jira/browse/NIFI-3508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-3508: - Component/s: (was: Core Framework) Extensions > InvokeHTTP send request body on PATCH if PROP_SEND_BODY > --- > > Key: NIFI-3508 > URL: https://issues.apache.org/jira/browse/NIFI-3508 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Nicholas Carenza >Priority: Minor > Labels: InvokeHTTP, processor > Fix For: 1.2.0 > > > I am dealing with an API that updates documents using the PATCH request > method and I need to send a request body but InvokeHTTP will only send a > request body if the request method is POST or PUT without any way to override. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3328) Listen SNMP Trap
[ https://issues.apache.org/jira/browse/NIFI-3328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866441#comment-15866441 ] Pierre Villard commented on NIFI-3328: -- Hey [~joewitt], I believe this one can also be closed/deleted in favor of NIFI-3320. > Listen SNMP Trap > > > Key: NIFI-3328 > URL: https://issues.apache.org/jira/browse/NIFI-3328 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.1.1 >Reporter: Balakrishnan R > > As part NIFI-1537 SNMP Get, Walk and Set were introduced. However SNMP Traps > sent to NIFI servers cannot be converted to a flow file currently. This is a > fairly useful feature. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (NIFI-3455) GenerateTableFetch fails for very large row counts
[ https://issues.apache.org/jira/browse/NIFI-3455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-3455. -- Resolution: Fixed > GenerateTableFetch fails for very large row counts > -- > > Key: NIFI-3455 > URL: https://issues.apache.org/jira/browse/NIFI-3455 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.2.0 >Reporter: Peter Wicks >Assignee: Peter Wicks >Priority: Minor > Fix For: 1.2.0 > > > When retrieving the row count for the total number of rows to page over > GenerateTableFetch gets the row count as an integer. In some cases the value > returned is greater then the maximum value of Int32 causing a looping of the > value, resulting in a negative number. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3452) Add Wait processor Wait Mode property
[
https://issues.apache.org/jira/browse/NIFI-3452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-3452:
-
Resolution: Fixed
Fix Version/s: 1.2.0
Status: Resolved (was: Patch Available)
> Add Wait processor Wait Mode property
> -
>
> Key: NIFI-3452
> URL: https://issues.apache.org/jira/browse/NIFI-3452
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.2.0
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
> Fix For: 1.2.0
>
> Attachments: wait-for-a-par-of-flow-to-finish.png
>
>
> NiFi back pressure is handled per relationship and as long as a relationship
> has room to receive more flow files, source processor is scheduled to run.
> However, this behavior is not ideal in some cases. For example, when there is
> very computationally expensive task and user wants to limit the number of
> FlowFiles can be processed at a given time, it's not always possible to limit
> the rate by existing RateControl nor back-pressure mechanism.
> As a more practical example, in the following flow, it's expected the GetSQS
> is triggered only when the previous FlowFile has been processed completely.
> Node 1 is parsing a flow file (indicated by the X in the connection between
> FetchS3Object and Parse). Both connections have a back-pressure threshold of
> 1, but because the object is already fetched, the first connection is empty
> and can thus be filled. This means that, if a new item becomes available in
> the queue, both of the following cases can happen with equal probability:
> {code}
> Case 1:
> -- - -
> Node 1: | GetSQS | -X-> | FetchS3Object | -X-> | Parse |
> -- - -
> -- - -
> Node 2: | GetSQS | ---> | FetchS3Object | ---> | Parse |
> -- - -
> Case 2:
> -- - -
> Node 1: | GetSQS | ---> | FetchS3Object | -X-> | Parse |
> -- - -
> -- - -
> Node 2: | GetSQS | -X-> | FetchS3Object | ---> | Parse |
> -- - -
> {code}
> To achieve that, we could improve Wait processor as follows.
> NiFi scheduler checks downstream relationship availability, when it's full,
> the processor won't be scheduled to run. In case a source processor has
> multiple outgoing relationships, and if ANY of those is full, the processor
> won't be scheduled.
> (This is how processor scheduling works with back-pressure, but can
> alter with @TriggerWhenAnyDestinationAvailable annotation. DistributeLoad is
> the only processor annotated with this)
> We could use this mechanism to keep the source processor waiting to be
> scheduled, by following flow:
> {code}
> GetSQS
> -- success --> FetchS3Object --> Parse --> Notify
> -- success --> Wait
> {code}
> To make it work as expected, we need to improve Wait so that user can choose
> how waiting FlowFile is handled, from either:
> "Route to 'wait' relationship" or "Keep in the Upstream connection".
> Currently it has only option to route to 'wait'.
> Use "Keep in the Upstream connection" Wait mode with the flow above,
> the incoming flow file in GetSQS -> Wait connection stays there until actual
> data processing finishes and Notify sends a notification signal.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Resolved] (NIFI-3016) UI - Allow template overriding
[ https://issues.apache.org/jira/browse/NIFI-3016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-3016. -- Resolution: Later This will be addressed with the NiFi Registry sub-project. > UI - Allow template overriding > -- > > Key: NIFI-3016 > URL: https://issues.apache.org/jira/browse/NIFI-3016 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Minor > > It would be useful that, when saving a template with a name of an already > existing template, the user is proposed the option to override the existing > one. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (NIFI-3331) TLS Toolkit - add the possibility to define a SAN in issued certificates
[ https://issues.apache.org/jira/browse/NIFI-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard reassigned NIFI-3331: Assignee: Pierre Villard > TLS Toolkit - add the possibility to define a SAN in issued certificates > > > Key: NIFI-3331 > URL: https://issues.apache.org/jira/browse/NIFI-3331 > Project: Apache NiFi > Issue Type: Improvement > Components: Tools and Build >Reporter: Pierre Villard >Assignee: Pierre Villard > > To ease the deployment of a load balancer in front of NiFi, it would be nice > to allow users to define a SAN in certificates issued by the CA. > To load balance the access to the UI or even with a ListenHTTP processor, > both will cause errors with a "Host mismatch" kind of error because of > different fqdn between nodes certificate and LB certificate. This is also > discussed here: http://stackoverflow.com/questions/40035356/nifi-load-balancer -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3331) TLS Toolkit - add the possibility to define a SAN in issued certificates
[ https://issues.apache.org/jira/browse/NIFI-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-3331: - Labels: tls-toolkit (was: ) Fix Version/s: 1.2.0 Status: Patch Available (was: Open) > TLS Toolkit - add the possibility to define a SAN in issued certificates > > > Key: NIFI-3331 > URL: https://issues.apache.org/jira/browse/NIFI-3331 > Project: Apache NiFi > Issue Type: Improvement > Components: Tools and Build >Reporter: Pierre Villard >Assignee: Pierre Villard > Labels: tls-toolkit > Fix For: 1.2.0 > > > To ease the deployment of a load balancer in front of NiFi, it would be nice > to allow users to define a SAN in certificates issued by the CA. > To load balance the access to the UI or even with a ListenHTTP processor, > both will cause errors with a "Host mismatch" kind of error because of > different fqdn between nodes certificate and LB certificate. This is also > discussed here: http://stackoverflow.com/questions/40035356/nifi-load-balancer -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3490) TLS Toolkit - define SAN in standalone mode
[ https://issues.apache.org/jira/browse/NIFI-3490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-3490: - Description: Following NIFI-3331, it would be useful to have the same option (add Subject Alternative Names in certificates) when using the TLS toolkit in standalone mode. > TLS Toolkit - define SAN in standalone mode > --- > > Key: NIFI-3490 > URL: https://issues.apache.org/jira/browse/NIFI-3490 > Project: Apache NiFi > Issue Type: Improvement > Components: Tools and Build >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Minor > Labels: tls-toolkit > > Following NIFI-3331, it would be useful to have the same option (add Subject > Alternative Names in certificates) when using the TLS toolkit in standalone > mode. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (NIFI-3490) TLS Toolkit - define SAN in standalone mode
Pierre Villard created NIFI-3490: Summary: TLS Toolkit - define SAN in standalone mode Key: NIFI-3490 URL: https://issues.apache.org/jira/browse/NIFI-3490 Project: Apache NiFi Issue Type: Improvement Components: Tools and Build Reporter: Pierre Villard Assignee: Pierre Villard Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (NIFI-3014) GetSNMP: add textual oid attribute to flowfile
[ https://issues.apache.org/jira/browse/NIFI-3014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-3014. -- Resolution: Fixed > GetSNMP: add textual oid attribute to flowfile > -- > > Key: NIFI-3014 > URL: https://issues.apache.org/jira/browse/NIFI-3014 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.1.0 >Reporter: João Henrique Ferreira de Freitas >Priority: Minor > Fix For: 1.2.0 > > > Sometimes is good to have OID textual descripton. So we could easily to > create a JSON document or whatever you like. The final flowfile attribute > should be: snmp$textualOid -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3014) GetSNMP: add textual oid attribute to flowfile
[ https://issues.apache.org/jira/browse/NIFI-3014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-3014: - Fix Version/s: 1.2.0 Component/s: (was: Core Framework) Extensions > GetSNMP: add textual oid attribute to flowfile > -- > > Key: NIFI-3014 > URL: https://issues.apache.org/jira/browse/NIFI-3014 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.1.0 >Reporter: João Henrique Ferreira de Freitas >Priority: Minor > Fix For: 1.2.0 > > > Sometimes is good to have OID textual descripton. So we could easily to > create a JSON document or whatever you like. The final flowfile attribute > should be: snmp$textualOid -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3431) Support batch update in Notify processor
[
https://issues.apache.org/jira/browse/NIFI-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-3431:
-
Resolution: Fixed
Fix Version/s: 1.2.0
Status: Resolved (was: Patch Available)
> Support batch update in Notify processor
>
>
> Key: NIFI-3431
> URL: https://issues.apache.org/jira/browse/NIFI-3431
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.2.0
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
> Fix For: 1.2.0
>
>
> NIFI-3216 added ability to wait for N signals. It supports waiting for N
> fragments split by Split processors. However, since Notify processor has
> to increase count one by one by calling expensive replace cache operation
> over network, it doesn't provide a practical performance when user configured
> a flow looks like below as N glow:
> {code}
> Split
> --> (original) -> Wait for N
> --> (split) -> Do something -> Notify 1
> {code}
> This JIRA improves Notify processor by:
> - Add "Signal Buffer Count" to specify max number of flow files that can be
> buffered and update cache at once
> - Add "Signal Counter Delta" to specify delta grater than 1, EL supported
> "Signal Buffer Count" would be useful in a flow like this:
> {code}
> Split
> --> (original) -> Wait for N
> --> (split) -> Filter or something -> Notify M
> {code}
> Buffer incoming M flow files and perform cache replace operation once.
> So does "Signal Counter Delta":
> {code}
> Split
> --> (original) -> Wait for N
> --> (split) -> Filter or something -> Merge M -> PutXXX -> Notify M
> {code}
> Specify 'M' via Attribute Expression Language.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (NIFI-3405) Add uptime to JVM section in System Diagnostics
[ https://issues.apache.org/jira/browse/NIFI-3405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-3405: - Assignee: Pierre Villard Fix Version/s: 1.2.0 Status: Patch Available (was: Open) > Add uptime to JVM section in System Diagnostics > --- > > Key: NIFI-3405 > URL: https://issues.apache.org/jira/browse/NIFI-3405 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Aldrin Piri >Assignee: Pierre Villard > Fix For: 1.2.0 > > > When diagnosing system performance, it would be helpful to see how long the > current JVM has been alive. In one case, this could provide context helpful > for interpreting garbage collection. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3459) Weird blue highlighting of breadcrumbs
[ https://issues.apache.org/jira/browse/NIFI-3459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-3459: - Resolution: Fixed Fix Version/s: 1.2.0 Status: Resolved (was: Patch Available) > Weird blue highlighting of breadcrumbs > -- > > Key: NIFI-3459 > URL: https://issues.apache.org/jira/browse/NIFI-3459 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Reporter: Scott Aslan >Assignee: Scott Aslan >Priority: Trivial > Fix For: 1.2.0 > > > When clicking on the breadcrumbs to navigate to a group there seems to be > some weird blue highlighting. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-1657) Provide support for various locales in CI builds
[ https://issues.apache.org/jira/browse/NIFI-1657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-1657: - Resolution: Fixed Fix Version/s: 1.2.0 Status: Resolved (was: Patch Available) > Provide support for various locales in CI builds > > > Key: NIFI-1657 > URL: https://issues.apache.org/jira/browse/NIFI-1657 > Project: Apache NiFi > Issue Type: Task > Components: Tools and Build >Reporter: Aldrin Piri >Assignee: Andre >Priority: Minor > Fix For: 1.2.0 > > > There were issues stemming from locale and building on more than the default > would help us catch similar issues moving forward. > From https://github.com/apache/nifi/pull/292, this can be accomplished via > something like: > -Duser.language=fr -Duser.region=FR -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3403) NPE in InvokeHTTP
[
https://issues.apache.org/jira/browse/NIFI-3403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-3403:
-
Assignee: Pierre Villard
Fix Version/s: 1.2.0
Status: Patch Available (was: Open)
> NPE in InvokeHTTP
> -
>
> Key: NIFI-3403
> URL: https://issues.apache.org/jira/browse/NIFI-3403
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.1.1, 0.7.1
>Reporter: Brandon DeVries
>Assignee: Pierre Villard
>Priority: Minor
> Fix For: 1.2.0
>
>
> InvokeHTTP throws an NPE when running in "source" mode if given a value for
> "Attributes to Send". If this value is set, it attempts to do\[1\]...
> {code}
> requestFlowFile.getAttributes(){code}
> ...without checking if requestFlowFile == null (which will be the case when
> running in "source" mode).
> \[1\]
> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/InvokeHTTP.java#L859
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Resolved] (NIFI-3379) ListHDFS has no File Filter Regex
[ https://issues.apache.org/jira/browse/NIFI-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-3379. -- Resolution: Duplicate Assignee: Pierre Villard The pull request for NIFI-2859 implements a file regex in ListHDFS processor. Once the PR will be reviewed and merged, the feature will be available in NiFi. > ListHDFS has no File Filter Regex > - > > Key: NIFI-3379 > URL: https://issues.apache.org/jira/browse/NIFI-3379 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Jan Verbeeck >Assignee: Pierre Villard >Priority: Minor > > ListHDFS is missing 'File Filter Regex'. Look at listSFTP for the correct > behaviour. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2581) Context menus and tooltips getting hidden after automatic refresh
[ https://issues.apache.org/jira/browse/NIFI-2581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-2581: - Resolution: Fixed Status: Resolved (was: Patch Available) > Context menus and tooltips getting hidden after automatic refresh > - > > Key: NIFI-2581 > URL: https://issues.apache.org/jira/browse/NIFI-2581 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.0.0 >Reporter: Jeff Storck >Assignee: Matt Gilman >Priority: Minor > Fix For: 1.0.0 > > > When the NiFi UI's automatic refresh occurs, if there are context menus > (right-click) or tooltips (for validation and bulletins) visible, they get > hidden when the refresh completes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-1867) improve ModifyBytes to make it easy to remove all flowfile content
[ https://issues.apache.org/jira/browse/NIFI-1867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-1867. -- Resolution: Fixed Fix Version/s: 0.7.1 0.8.0 1.0.0 > improve ModifyBytes to make it easy to remove all flowfile content > -- > > Key: NIFI-1867 > URL: https://issues.apache.org/jira/browse/NIFI-1867 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 0.6.1 >Reporter: Ben Icore >Assignee: Joe Skora > Fix For: 1.0.0, 0.8.0, 0.7.1 > > > update ModifyBytes processor to include a "Remove all content" property. > this property shouild default to false so existing functionality is not > changed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2602) SelectHiveQL CSV output doesn't handle null values
[ https://issues.apache.org/jira/browse/NIFI-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-2602: - Resolution: Fixed Status: Resolved (was: Patch Available) > SelectHiveQL CSV output doesn't handle null values > -- > > Key: NIFI-2602 > URL: https://issues.apache.org/jira/browse/NIFI-2602 > Project: Apache NiFi > Issue Type: Bug >Reporter: Matt Burgess >Assignee: Matt Burgess > Fix For: 1.0.0 > > > A NullPointerException is issued by SelectHiveQL if CSV is selected as the > output format and the result rows have null values in them. Instead, the > entry for a null-valued column should be the empty string. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-1334) minor documentation issues
[ https://issues.apache.org/jira/browse/NIFI-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-1334. -- Resolution: Fixed Fix Version/s: 0.5.0 Closing the JIRA. It seems to be OK since a while. > minor documentation issues > --- > > Key: NIFI-1334 > URL: https://issues.apache.org/jira/browse/NIFI-1334 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation & Website >Affects Versions: 0.4.1 >Reporter: Lee Laim >Priority: Trivial > Labels: documentation > Fix For: 0.5.0 > > Original Estimate: 1h > Remaining Estimate: 1h > > I noticed a few subtle typos/inconsistencies in the expression-language > guide.They're minor but easily overlooked. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1342) PostHTTP User Agent property should be pre-populated with client default
[ https://issues.apache.org/jira/browse/NIFI-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-1342: - Status: Patch Available (was: Open) > PostHTTP User Agent property should be pre-populated with client default > > > Key: NIFI-1342 > URL: https://issues.apache.org/jira/browse/NIFI-1342 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 0.4.1 >Reporter: Aldrin Piri >Priority: Trivial > > Currently, PostHTTP shows an empty string for the User Agent property which > is used in web requests, but this actually results in the default of the > client being used. For clarity, and if the backing library supports it, > getting the User Agent string used by the library as the default processor > User Agent property would be a nice improvement. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2770) flow configuration history action details dialog only shows UUID
[ https://issues.apache.org/jira/browse/NIFI-2770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-2770: - Resolution: Fixed Status: Resolved (was: Patch Available) Merged to master. > flow configuration history action details dialog only shows UUID > > > Key: NIFI-2770 > URL: https://issues.apache.org/jira/browse/NIFI-2770 > Project: Apache NiFi > Issue Type: Bug >Reporter: Scott Aslan >Assignee: Scott Aslan > Fix For: 1.1.0 > > > In the NiFi history shell when I click to view details on a particular event > the action details dialog just shows the uuid of the component and no actual > details -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2752) Correct ReplaceText default pattern and unit tests
[
https://issues.apache.org/jira/browse/NIFI-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-2752:
-
Resolution: Fixed
Fix Version/s: 1.1.0
Status: Resolved (was: Patch Available)
Full build with contrib-check. Tested fix. +1
Merged to master.
[~jskora] I don't feel it is supposed to go into 0.x, let me know if it should.
> Correct ReplaceText default pattern and unit tests
> --
>
> Key: NIFI-2752
> URL: https://issues.apache.org/jira/browse/NIFI-2752
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.1.0, 0.8.0, 0.7.1
>Reporter: Joe Skora
>Assignee: Joe Skora
> Fix For: 1.1.0
>
>
> [{{ReplaceText.DEFAULT_REGEX}}|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ReplaceText.java#L87]
> is defined as "(?s:\^.\*$)", which is valid PCRE but must be expressed as
> "(?s)(\^.\*$)" in Java.
> The Java [Pattern class|https://docs.oracle.com/javase/8/docs/api/index.html]
> specifies that patterns like "(?idmsux-idmsux:X)" are _non-capturing_, so
> anything but the default pattern and replacement value result in empty
> output. This isn't caught by unit tests because the code short circuits if
> the default pattern and replacement are found in
> [ReplaceText.onTrigger()|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ReplaceText.java#L217].
> This hides the capture group problem from the unit tests and the default
> processor configuration, but causes the processor to produce empty output if
> using non-trivial patterns and replacements.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-2262) SQL processors fail if column name contains characters illegal for Avro
[ https://issues.apache.org/jira/browse/NIFI-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-2262: - Resolution: Fixed Status: Resolved (was: Patch Available) > SQL processors fail if column name contains characters illegal for Avro > --- > > Key: NIFI-2262 > URL: https://issues.apache.org/jira/browse/NIFI-2262 > Project: Apache NiFi > Issue Type: Bug >Reporter: Matt Burgess >Assignee: Matt Burgess > Fix For: 1.1.0 > > > For SQL processors that output Avro (ExecuteSQL, QueryDatabaseTable, etc.), > if the SQL result set contains columns whose names contain Avro-illegal > characters (such as a period), then an error occurs: > org.apache.avro.SchemaParseException: Illegal character in: user.gender > These processors should either normalize the names for Avro automatically or > have a property indicating whether names should be normalized. > A workaround for ExecuteSQL in many cases is to alias the columns in the SQL. > This approach doesn't currently work in QueryDatabaseTable for incremental > fetch (i.e. maximum-value columns) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-2373) GetHTTP - not working with sites that only support TLS
[ https://issues.apache.org/jira/browse/NIFI-2373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-2373. -- Resolution: Fixed Assignee: Andy LoPresto Fix Version/s: 0.8.0 1.1.0 Fixed with: https://github.com/apache/nifi/pull/999 > GetHTTP - not working with sites that only support TLS > -- > > Key: NIFI-2373 > URL: https://issues.apache.org/jira/browse/NIFI-2373 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.5.1, 0.7.0 > Environment: Windows & Debian >Reporter: Tim >Assignee: Andy LoPresto > Fix For: 1.1.0, 0.8.0 > > Attachments: Test_GetHTTP.xml > > > Hi, > I'm using GetHTTP processor to download a file from nvd.nist.gov. > Recently, they have disabled support for SSL and only allow connections to be > established through TLS 1.1 or TLS 1.2. > Since this change, the GetHTTP processor fails to download the file and > returns a generic "javax.net.ssl.SSLHandshakeException: Remote host closed > connection during handshake" error. > How to reproduce: > - Create GetHTTP Processor > - Create StandardSSLContextService > - Configure URL & test download: >https://nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-Modified.xml.gz > What I've tried so far: > - Changing the SSL Protocol attribute of the SSLContextService (to TLS, TLS > 1.1 or TLS 1.2, no difference) > - Disabling SSL with the following startup argument: > -Dhttps.protocols=TLSv1.2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-2266) GetHTTP and PutHTTP use hard-coded TLS protocol version
[
https://issues.apache.org/jira/browse/NIFI-2266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard resolved NIFI-2266.
--
Resolution: Fixed
Fix Version/s: 0.8.0
1.1.0
> GetHTTP and PutHTTP use hard-coded TLS protocol version
> ---
>
> Key: NIFI-2266
> URL: https://issues.apache.org/jira/browse/NIFI-2266
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 0.7.0, 0.6.1
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
> Labels: https, security, tls
> Fix For: 1.1.0, 0.8.0
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> As pointed out on the mailing list [1], the {{GetHTTP}} (and likely
> {{PutHTTP}}) processors use a hard-coded TLS protocol version. {{PostHTTP}}
> also did this and was fixed by [NIFI-1688].
> The same fix should apply here and unit tests already exist which can be
> applied to the other processors as well.
> For future notice, {{InvokeHTTP}} is a better processor for generic HTTP
> operations and has supported reading the TLS protocol version from the
> {{SSLContextService}} for some time.
> [1]
> https://lists.apache.org/thread.html/a48e2ebbc2231d685491ae6b856c760620efca5bff2c7249f915b24d@%3Cdev.nifi.apache.org%3E
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-2071) Support repeating capture groups in ExtractText
[ https://issues.apache.org/jira/browse/NIFI-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-2071: - Resolution: Fixed Status: Resolved (was: Patch Available) > Support repeating capture groups in ExtractText > --- > > Key: NIFI-2071 > URL: https://issues.apache.org/jira/browse/NIFI-2071 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joey Frazee >Assignee: Pierre Villard > Fix For: 1.1.0 > > > ExtractText doesn't currently support repeating capture groups so any > repeating patterns have to specified by hand and can only be repeated a fixed > number of times. > I think this is because it only uses find() and not findAll() in its pattern > matching [1]. > 1. > https://github.com/apache/nifi/blob/1bd2cf0d09a7111bcecffd0f473aa71c25a69845/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExtractText.java#L324 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1912) PutEmail doesn't send formatted email when there is an attachment with the message
[
https://issues.apache.org/jira/browse/NIFI-1912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-1912:
-
Assignee: Pierre Villard
Fix Version/s: 1.1.0
Status: Patch Available (was: Open)
> PutEmail doesn't send formatted email when there is an attachment with the
> message
> --
>
> Key: NIFI-1912
> URL: https://issues.apache.org/jira/browse/NIFI-1912
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 0.6.1
>Reporter: Ravi Bhushan Ratnakar
>Assignee: Pierre Villard
>Priority: Critical
> Fix For: 1.1.0
>
>
> When I am trying to send formatted email with an attachment using PutEmail
> processor, all the html tags are displayed as it is like a text(This).
> After going through the code what i observed that when there is an
> attachment, in that case content type is hard coded to 'text/plain'. Below
> code snippet from PutEmail
> if (context.getProperty(ATTACH_FILE).asBoolean()) {
> final MimeBodyPart mimeText = new
> PreencodedMimeBodyPart("base64");
> mimeText.setDataHandler(new DataHandler(new
> ByteArrayDataSource(
> Base64.encodeBase64(messageText.getBytes("UTF-8")),
> "text/plain; charset=\"utf-8\"")));
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-2848) Queues aren't fairly drained when leading to a single component
[ https://issues.apache.org/jira/browse/NIFI-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-2848: - Status: Patch Available (was: Open) > Queues aren't fairly drained when leading to a single component > --- > > Key: NIFI-2848 > URL: https://issues.apache.org/jira/browse/NIFI-2848 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 0.7.0, 1.0.0 >Reporter: Joseph Gresock >Assignee: Pierre Villard > Attachments: Backpressure_prioritization_test.xml, dominate_1.png, > dominate_2.png, queue_drain.png > > > Consider the scenario where multiple queues lead to a single component and > all of them are full due to back pressure. With the attached template, it is > easily observable that once a single queue starts to drain due to relieved > back pressure, it will continue to drain as long as it has incoming flow > files. This means that if there's a constant flow of incoming flow files to > this queue, the other queues will never be drained (at least, that's my > theory based on several hours of observation). > To reproduce this: > # Load the template into NiFi 1.0.0 > # Play all three GenerateFlowFile processors, but not the UpdateAttribute > processor (this simulates backpressure). Wait until each queue has 1,000 > flow files (max backpressure) > # Stop the GenerateFlowFile processors, and play the UpdateAttribute > processor (this relieves the backpressure) > # Observe which queue has started to drain, and start its GenerateFlowFile > processor > # Observe that the other two queues remain full indefinitely, while the > draining queue continues to replenish and be drained indefinitely -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-2398) New processor GetIgniteCache for getting values from Apache Ignite
[ https://issues.apache.org/jira/browse/NIFI-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-2398. -- Resolution: Fixed Fix Version/s: 1.1.0 > New processor GetIgniteCache for getting values from Apache Ignite > --- > > Key: NIFI-2398 > URL: https://issues.apache.org/jira/browse/NIFI-2398 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 0.6.1 > Environment: All >Reporter: Mans Singh >Assignee: Mans Singh > Labels: cache,, get, ignite, > Fix For: 1.1.0 > > Original Estimate: 48h > Remaining Estimate: 48h > > New component for getting value from Ignite Cache -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2792) Removal of a Template does not save the flow.
[ https://issues.apache.org/jira/browse/NIFI-2792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-2792: - Resolution: Fixed Status: Resolved (was: Patch Available) > Removal of a Template does not save the flow. > - > > Key: NIFI-2792 > URL: https://issues.apache.org/jira/browse/NIFI-2792 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Critical > Fix For: 1.1.0 > > > When removing a template, the flow is not saved. Consequently, if no other > actions are taken prior to shutting down the template will be reloaded upon > the next restart. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (NIFI-1170) TailFile "File to Tail" property should support Wildcards
[
https://issues.apache.org/jira/browse/NIFI-1170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard reassigned NIFI-1170:
Assignee: Pierre Villard
> TailFile "File to Tail" property should support Wildcards
> -
>
> Key: NIFI-1170
> URL: https://issues.apache.org/jira/browse/NIFI-1170
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 0.4.0
>Reporter: Andre
>Assignee: Pierre Villard
> Fix For: 1.1.0
>
>
> Because of challenges around log rotation of high volume syslog and app
> producers, it is customary to logging platform developers to promote file
> variables based file names such as DynaFiles (rsyslog), Macros(syslog-ng)as
> alternatives to getting SIGHUPs being sent to the syslog daemon upon every
> file rotation.
> (To certain extent, used even NiFi's has similar patterns, like for example,
> when one uses Expression Language to set PutHDFS destination file).
> The current TailFile strategy suggests rotation patterns like:
> {code}
> log_folder/app.log
> log_folder/app.log.1
> log_folder/app.log.2
> log_folder/app.log.3
> {code}
> It is possible to fool the system to accept wildcards by simply using a
> strategy like:
> {code}
> log_folder/test1
> log_folder/server1
> log_folder/server2
> log_folder/server3
> {code}
> And configure *Rolling Filename Pattern* to * but it feels like a hack,
> rather than catering for an ever increasingly prevalent use case
> (DynaFile/macros/etc).
> It would be great if instead, TailFile had the ability to use wildcards on
> File to Tail property
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-1959) TailFile not ingesting data when tailed file is moved with no rolling pattern
[ https://issues.apache.org/jira/browse/NIFI-1959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-1959: - Resolution: Fixed Fix Version/s: 1.1.0 Status: Resolved (was: Patch Available) > TailFile not ingesting data when tailed file is moved with no rolling pattern > - > > Key: NIFI-1959 > URL: https://issues.apache.org/jira/browse/NIFI-1959 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 0.6.1 >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Minor > Fix For: 1.1.0 > > > In case "Rolling Filename Pattern" is not set by user and if the tailed file > is moved, then the new file will not be tailed. > Besides, in such case, the processor will be endlessly triggered without > ingesting data: it creates a lot of tasks and consumes CPU. The reason is it > never goes in if statement L448. > A solution is to look at size() and lastUpdated() of the tailed file to > detect a "rollover". However it won't allow the processor to ingest the > potential data added in the tailed file just before being moved. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-2794) Expose a getOneFromEachConnection method in ProcessSession
[ https://issues.apache.org/jira/browse/NIFI-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-2794. -- Resolution: Won't Fix Closing at "Won't fix". Thanks for your insights! > Expose a getOneFromEachConnection method in ProcessSession > -- > > Key: NIFI-2794 > URL: https://issues.apache.org/jira/browse/NIFI-2794 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Minor > Attachments: image.png > > > In case a processor has multiple incoming connections, it could be > interesting to expose a method allowing users to get exactly one flow file > per incoming connection in a row. > That could unlock opportunities such as performing join operations on > datasets. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-1844) Cannot download new files from FTP server without re-downloading old files
[ https://issues.apache.org/jira/browse/NIFI-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-1844. -- Resolution: Fixed Fix Version/s: 1.1.0 > Cannot download new files from FTP server without re-downloading old files > -- > > Key: NIFI-1844 > URL: https://issues.apache.org/jira/browse/NIFI-1844 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Subhash Sriram > Fix For: 1.1.0 > > > Hello, > It appears that currently, there is no way to download files from a FTP > server, and keep track of what files have been downloaded in NiFi. > For example, I am trying to get files from a third party FTP server, from > which I cannot download the original, and it does not support SFTP. > When I use the GetFTP processor, it downloads the same files over and over, > and I cannot use the ListSFTP processor. > It would be great if there was a ListFTP/FetchFTP processor that could allow > someone to acquire only the files from a FTP server that have not yet been > downloaded. > Thank you very much! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-2720) UI - in view configuration textarea is not fully resizable
[ https://issues.apache.org/jira/browse/NIFI-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-2720. -- Resolution: Fixed Fix Version/s: 1.1.0 Checked the fix and confirmed that we now have the expected result. Merge into master. Thanks [~scottyaslan]! > UI - in view configuration textarea is not fully resizable > -- > > Key: NIFI-2720 > URL: https://issues.apache.org/jira/browse/NIFI-2720 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.0.0 >Reporter: Pierre Villard >Assignee: Scott Aslan >Priority: Minor > Fix For: 1.1.0 > > > When a processor is started, accessing 'view configuration', and clicking on > a property to display its content, the text area is only vertically > resizable. For some processors like ExecuteScript it'd better to be able to > fully resize the text area (as it is now when editing the configuration while > the processor is stopped). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-2810) Allow Content Type to be set in PutS3Object processor
[ https://issues.apache.org/jira/browse/NIFI-2810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-2810. -- Resolution: Fixed Visually verified code. Full build with contrib-check and tested with simple workflow. Merged into master. Thanks [~evega]! > Allow Content Type to be set in PutS3Object processor > - > > Key: NIFI-2810 > URL: https://issues.apache.org/jira/browse/NIFI-2810 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.1.0 >Reporter: Edgardo Vega >Priority: Minor > Fix For: 1.1.0 > > > The PutS3 Processor does not allow you to set the Content Type. Add this > ability in this processor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2815) Cannot change script engine in InvokeScriptedProcessor
[ https://issues.apache.org/jira/browse/NIFI-2815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-2815: - Resolution: Fixed Status: Resolved (was: Patch Available) > Cannot change script engine in InvokeScriptedProcessor > -- > > Key: NIFI-2815 > URL: https://issues.apache.org/jira/browse/NIFI-2815 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.0.0 >Reporter: Matt Burgess >Assignee: Matt Burgess > Fix For: 1.1.0 > > > Due to a regression introduced in NIFI-2665, the script engine for an > InvokeScriptedProcessor is instantiated only once. For instances already in > the flow, the Script Engine property setting will be honored, but it cannot > be changed. For new InvokeScriptedProcessor instances, the script engine will > be set to ECMAScript and also cannot be changed. > Seems that before NIFI-2665, script engine setup was happening too often, now > it is not happening enough. The script engine should be reset when the > property has been modified. > A workaround is to change the processor to the way you want it, then save it > as a template, then delete the original and re-load it from the template. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-2055) Processor bulletins are displayed oldest first, which makes debugging difficult
[ https://issues.apache.org/jira/browse/NIFI-2055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-2055. -- I don't know what is the exact commit that solved the issue, but this is fixed in the 1.0.0 version. > Processor bulletins are displayed oldest first, which makes debugging > difficult > --- > > Key: NIFI-2055 > URL: https://issues.apache.org/jira/browse/NIFI-2055 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Reporter: Randy Gelhausen > Fix For: 1.0.0 > > Attachments: Screen Shot 2016-06-18 at 5.29.30 PM.png > > > When debugging a flow, it's common to change processor configuration items > iteratively until everything is set correctly/as desired. > By default, bulletins expire after 5 minutes, which is helpful. However, > bulletins are displayed (when mousing over the processor) by oldest timestamp > first. This means it's difficult to see errors/warnings generated by my > newest configurations unless I wait 5 minutes for all the old ones to expire. > I propose changing the bulletin display order to most-recent bulletins first. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2072) Support named captures in ExtractText
[ https://issues.apache.org/jira/browse/NIFI-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15512784#comment-15512784 ] Pierre Villard commented on NIFI-2072: -- [~jfrazee] I was looking at this JIRA and I agree that it would be a great addition. However it seems this is not possible to get the group names from the Pattern expression unless if we are going a bit nasty... http://stackoverflow.com/questions/15588903/get-group-names-in-java-regex > Support named captures in ExtractText > - > > Key: NIFI-2072 > URL: https://issues.apache.org/jira/browse/NIFI-2072 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joey Frazee > > ExtractText currently captures and creates attributes using numeric indices > (e.g, attribute.name.0, attribute.name.1, etc.) whether or not the capture > groups are named, i.e., patterns like (?\w+). > In addition to being more faithful to the provided regexes, named captures > could help simplify data flows because you wouldn't have to add superfluous > UpdateAttribute steps which are just renaming the indexed captures to more > interpretable names. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-2802) The implementation classes don't support orderByClause.
[
https://issues.apache.org/jira/browse/NIFI-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard resolved NIFI-2802.
--
Resolution: Fixed
Fix Version/s: 1.1.0
[~combine] Thanks for catching that! Visually verified code, full build with
contrib-check. Merged to master.
> The implementation classes don't support orderByClause.
> ---
>
> Key: NIFI-2802
> URL: https://issues.apache.org/jira/browse/NIFI-2802
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.0.0
>Reporter: Byunghwa Yun
>Priority: Minor
> Fix For: 1.1.0
>
>
> The implementation classes don't support orderByClause.
> The below code is the part of GenericDatabaseAdapter class(even
> OracleDatabaseAdapter and DerbyDatabaseAdapter).
> if (!StringUtils.isEmpty(orderByClause)) {
> query.append(" ORDER BY ");
> query.append(whereClause);
> }
> Expected:
> if (!StringUtils.isEmpty(orderByClause)) {
> query.append(" ORDER BY ");
> query.append(orderByClause);
> }
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (NIFI-1893) Add processor for validating JSON
[
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard resolved NIFI-1893.
--
Resolution: Fixed
Fix Version/s: 1.1.0
> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
> Issue Type: New Feature
>Reporter: Matt Burgess
> Fix For: 1.1.0
>
>
> NiFi has a ValidateXml processor to validate incoming XML files against a
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
> name: "Test",
> timestamp: 1463499695,
> tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
> },
> fields: {
> "cpu": 10.2,
> "load": 15.6
> }
> }
> Could be validated successfully against the following "schema":
> {
> "type": "object",
> "required": ["name", "tags", "timestamp", "fields"],
> "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
> }
> }
> There is at least one ASF-friendly library that could be used for
> implementation: https://github.com/everit-org/json-schema
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-2071) Support repeating capture groups in ExtractText
[ https://issues.apache.org/jira/browse/NIFI-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-2071: - Assignee: Pierre Villard Fix Version/s: 1.1.0 Status: Patch Available (was: Open) > Support repeating capture groups in ExtractText > --- > > Key: NIFI-2071 > URL: https://issues.apache.org/jira/browse/NIFI-2071 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joey Frazee >Assignee: Pierre Villard > Fix For: 1.1.0 > > > ExtractText doesn't currently support repeating capture groups so any > repeating patterns have to specified by hand and can only be repeated a fixed > number of times. > I think this is because it only uses find() and not findAll() in its pattern > matching [1]. > 1. > https://github.com/apache/nifi/blob/1bd2cf0d09a7111bcecffd0f473aa71c25a69845/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExtractText.java#L324 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2805) Add fragment attributes for SplitAvro
[ https://issues.apache.org/jira/browse/NIFI-2805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-2805: - Resolution: Fixed Status: Resolved (was: Patch Available) Build with contrib-check, verified attributes in a basic workflow, merged into master. > Add fragment attributes for SplitAvro > - > > Key: NIFI-2805 > URL: https://issues.apache.org/jira/browse/NIFI-2805 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Matt Burgess >Assignee: Matt Burgess > Fix For: 1.1.0 > > > Some "splitting" processors such as SplitText and SplitContent write > attributes to the split flow files indicating their fragment index, the total > count, and the filename of the original flow file. This is done to support a > form of "micro-batching" and/or a split-join pattern in a data flow (i.e. > split the original file, do work on the individual pieces, and possibly merge > them together later). > For consistency and capability, the SplitAvro processor should write these > same attributes for its split flow files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2794) Expose a getOneFromEachConnection method in ProcessSession
[ https://issues.apache.org/jira/browse/NIFI-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-2794: - Attachment: image.png The idea was to cover this kind of situation: !image.png! Example: I have a processor A going through a list of IDs and generating one FlowFile per ID. I have a processor B fetching data from one source for the given ID and the processor C does the same from another source. The processor D could use this method to get FlowFiles from both B and C for the same ID and join the data into one flow file. I do realize now that this would be covered by the ongoing work with NIFI-2590, NIFI-1926, NIFI-2735. There are more general patterns to cover here and this, without any doubt, can be achieved in a more elegant/general way. > Expose a getOneFromEachConnection method in ProcessSession > -- > > Key: NIFI-2794 > URL: https://issues.apache.org/jira/browse/NIFI-2794 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Minor > Attachments: image.png > > > In case a processor has multiple incoming connections, it could be > interesting to expose a method allowing users to get exactly one flow file > per incoming connection in a row. > That could unlock opportunities such as performing join operations on > datasets. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2683) Tests rely on locale
Pierre Villard created NIFI-2683:
Summary: Tests rely on locale
Key: NIFI-2683
URL: https://issues.apache.org/jira/browse/NIFI-2683
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Affects Versions: 1.0.0
Environment: Apache Maven 3.3.9
(bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T17:41:47+01:00)
Java version: 1.8.0_74, vendor: Oracle Corporation
Default locale: fr_FR, platform encoding: Cp1252
OS name: "windows 10", version: "10.0", arch: "amd64", family: "dos"
Reporter: Pierre Villard
Priority: Minor
The following test fails because of the locale language.
{noformat}
Failed tests:
StandardHttpResponseMergerSpec.MergeResponses: #responseEntities.size() HTTP
200 #httpMethod responses for #requestUriPart:123 Condition not satisfied:
returnedJson == expectedJson
|| |
||
{"id":"1","permissions":{"canRead":false,"canWrite":false},"status":{"aggregateSnapshot":{"flowFilesIn":0,"bytesIn":1000,"input":"0
(1,000 bytes)","flowFilesOut":0,"bytesOut":0,"output":"
0 (0 bytes)","flowFilesQueued":0,"bytesQueued":0,"queued":"0 (0
bytes)","queuedSize":"0 bytes","queuedCount":"0"}}}
|false
|1 difference (99% similarity)
|
{"id":"1","permissions":{"canRead":false,"canWrite":false},"status":{"aggregateSnapshot":{"flowFilesIn":0,"bytesIn":1000,"input":"0
(1( )000 bytes)","flowFilesOut":0,"bytesOut":0,"output":"0
(0 bytes)","flowFilesQueued":0,"bytesQueued":0,"queued":"0 (0
bytes)","queuedSize":"0 bytes","queuedCount":"0"}}}
|
{"id":"1","permissions":{"canRead":false,"canWrite":false},"status":{"aggregateSnapshot":{"flowFilesIn":0,"bytesIn":1000,"input":"0
(1(,)000 bytes)","flowFilesOut":0,"bytesOut":0,"output":"0
(0 bytes)","flowFilesQueued":0,"bytesQueued":0,"queued":"0 (0
bytes)","queuedSize":"0 bytes","queuedCount":"0"}}}
{"id":"1","permissions":{"canRead":false,"canWrite":false},"status":{"aggregateSnapshot":{"flowFilesIn":0,"bytesIn":1000,"input":"0
(1 000 bytes)","flowFilesOut":0,"bytesOut":0,"output":"0 (0 bytes)","fl
owFilesQueued":0,"bytesQueued":0,"q
{noformat}
An easy option is to have a little change in the main pom.xml:
{noformat}
org.apache.maven.plugins
maven-surefire-plugin
2.18
**/*Test.class
**/Test*.class
**/*Spec.class
true
-Xmx1G
-Djava.net.preferIPv4Stack=true -Duser.language=en -Duser.region=US
org.apache.maven.surefire
surefire-junit4
2.18
{noformat}
otherwise the test needs to be changed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-766) UI should indicate when backpressure is configured for a Connection
[ https://issues.apache.org/jira/browse/NIFI-766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-766: Attachment: normal.png backpressure_and_expiration.png backpressure.png Screenshots with submitted PR. > UI should indicate when backpressure is configured for a Connection > --- > > Key: NIFI-766 > URL: https://issues.apache.org/jira/browse/NIFI-766 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Core UI >Reporter: Mark Payne > Attachments: backpressure.png, backpressure_and_expiration.png, > normal.png > > > It is sometimes unclear why a Processor is not running, if it is due to > backpressure. Recommend we add an icon to the Connection label to indicate > that backpressure is configured. If backpressure is "applied" (i.e., the > backpressure threshold has been reached), that icon should be highlighted > somehow. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-766) UI should indicate when backpressure is configured for a Connection
[ https://issues.apache.org/jira/browse/NIFI-766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-766: Fix Version/s: 1.1.0 Status: Patch Available (was: Open) > UI should indicate when backpressure is configured for a Connection > --- > > Key: NIFI-766 > URL: https://issues.apache.org/jira/browse/NIFI-766 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Core UI >Reporter: Mark Payne >Assignee: Pierre Villard > Fix For: 1.1.0 > > Attachments: backpressure.png, backpressure_and_expiration.png, > normal.png > > > It is sometimes unclear why a Processor is not running, if it is due to > backpressure. Recommend we add an icon to the Connection label to indicate > that backpressure is configured. If backpressure is "applied" (i.e., the > backpressure threshold has been reached), that icon should be highlighted > somehow. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2731) MergeContent default max number of flow files and max number of bins should be smaller
[ https://issues.apache.org/jira/browse/NIFI-2731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-2731: - Fix Version/s: 1.1.0 Status: Patch Available (was: Open) > MergeContent default max number of flow files and max number of bins should > be smaller > -- > > Key: NIFI-2731 > URL: https://issues.apache.org/jira/browse/NIFI-2731 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joseph Witt >Assignee: Pierre Villard > Fix For: 1.1.0 > > > Presently there is no default on max entries. It should probably be set to > 1000 by default. These are flow files and their objects are read into memory > and can add up quickly. Further, if we have 100 default max bins we could > end up with 100s of thousands of flow file objects held in memory during > common dataflow scenarios. Recommend moving to max 5 different bins by > default and max 1000 flow files by default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2603) Bringing Some UI Color Back
[ https://issues.apache.org/jira/browse/NIFI-2603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15543150#comment-15543150 ] Pierre Villard commented on NIFI-2603: -- [~rmoran] +1 > Bringing Some UI Color Back > --- > > Key: NIFI-2603 > URL: https://issues.apache.org/jira/browse/NIFI-2603 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Peter Wicks >Priority: Minor > Attachments: status-icon-color.png > > > In the new 1.0 UI all of the colors associated with status (except the orange > triangle) are gone; replaced with a dark gray color. > I propose bringing back color. The screenshots are in the format of before > on the top and after on the bottom, except were labeled in the picture itself: > - Top Status Menu: https://goo.gl/photos/se1JnvhRwU7N4Fap7 > - Process Group: https://goo.gl/photos/dqjG4KvC6xqxQfgT7 > - Processes (Running/Stopped/Invalid): > https://goo.gl/photos/dSS8vgE2RkrXtc77A > - Operate Play/Stop buttons (only on mouse hover): > https://goo.gl/photos/Am5SUEEn7G9RjmMe6 > - Processor/Processor Group Context Menu: > https://goo.gl/photos/Jq3qFg4ezaN91qms5 > This is not a "100% done, I've covered everything" before/after list. I know > I need to do the NiFi summary page also at the minimum. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2848) Queues aren't fairly drained when leading to a single component
[ https://issues.apache.org/jira/browse/NIFI-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15536700#comment-15536700 ] Pierre Villard commented on NIFI-2848: -- After looking into it, I believe this could be related to NIFI-2751. Indeed in processors such as UpdateAttribute or LogAttribute we are getting batch of flow files. Will look further into it. > Queues aren't fairly drained when leading to a single component > --- > > Key: NIFI-2848 > URL: https://issues.apache.org/jira/browse/NIFI-2848 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.0.0, 0.7.0 >Reporter: Joseph Gresock > Attachments: Backpressure_prioritization_test.xml, dominate_1.png, > dominate_2.png, queue_drain.png > > > Consider the scenario where multiple queues lead to a single component and > all of them are full due to back pressure. With the attached template, it is > easily observable that once a single queue starts to drain due to relieved > back pressure, it will continue to drain as long as it has incoming flow > files. This means that if there's a constant flow of incoming flow files to > this queue, the other queues will never be drained (at least, that's my > theory based on several hours of observation). > To reproduce this: > # Load the template into NiFi 1.0.0 > # Play all three GenerateFlowFile processors, but not the UpdateAttribute > processor (this simulates backpressure). Wait until each queue has 1,000 > flow files (max backpressure) > # Stop the GenerateFlowFile processors, and play the UpdateAttribute > processor (this relieves the backpressure) > # Observe which queue has started to drain, and start its GenerateFlowFile > processor > # Observe that the other two queues remain full indefinitely, while the > draining queue continues to replenish and be drained indefinitely -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (NIFI-2829) PutSQL assumes all Date and Time values are provided in Epoch
[ https://issues.apache.org/jira/browse/NIFI-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard reopened NIFI-2829: -- > PutSQL assumes all Date and Time values are provided in Epoch > - > > Key: NIFI-2829 > URL: https://issues.apache.org/jira/browse/NIFI-2829 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Paul Gibeault > Fix For: 1.1.0 > > Original Estimate: 24h > Remaining Estimate: 24h > > This bug is the same as NIFI-2576 only extended to data types DATE and TIME. > https://issues.apache.org/jira/browse/NIFI-2576 > When PutSQL sees a DATE or TIME data type it assumes that it's being provided > as a Long in Epoch format. > This doesn't make much sense since the Query Database tools that return Avro > return DATES and TIME values as strings; and thus following the > Avro->JSON->JSON To SQL Route leads to DATE and TIME fields as being strings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2829) PutSQL assumes all Date and Time values are provided in Epoch
[ https://issues.apache.org/jira/browse/NIFI-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-2829: - Status: Patch Available (was: Reopened) > PutSQL assumes all Date and Time values are provided in Epoch > - > > Key: NIFI-2829 > URL: https://issues.apache.org/jira/browse/NIFI-2829 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Paul Gibeault > Fix For: 1.1.0 > > Original Estimate: 24h > Remaining Estimate: 24h > > This bug is the same as NIFI-2576 only extended to data types DATE and TIME. > https://issues.apache.org/jira/browse/NIFI-2576 > When PutSQL sees a DATE or TIME data type it assumes that it's being provided > as a Long in Epoch format. > This doesn't make much sense since the Query Database tools that return Avro > return DATES and TIME values as strings; and thus following the > Avro->JSON->JSON To SQL Route leads to DATE and TIME fields as being strings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2832) State management should be part of the documentation/usage
Pierre Villard created NIFI-2832: Summary: State management should be part of the documentation/usage Key: NIFI-2832 URL: https://issues.apache.org/jira/browse/NIFI-2832 Project: Apache NiFi Issue Type: Improvement Components: Documentation & Website Reporter: Pierre Villard Assignee: Pierre Villard Priority: Minor At the moment, ``@Stateful`` annotation if only used in the 'View State' panel. This may not be intuitive for users (NIFI-2705) and this information should be part of the documentation (in the 'Usage' panel and the web documentation as well). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2832) State management should be part of the documentation/usage
[ https://issues.apache.org/jira/browse/NIFI-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-2832: - Fix Version/s: 1.1.0 Status: Patch Available (was: Open) > State management should be part of the documentation/usage > -- > > Key: NIFI-2832 > URL: https://issues.apache.org/jira/browse/NIFI-2832 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation & Website >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Minor > Labels: documentation > Fix For: 1.1.0 > > > At the moment, ``@Stateful`` annotation if only used in the 'View State' > panel. This may not be intuitive for users (NIFI-2705) and this information > should be part of the documentation (in the 'Usage' panel and the web > documentation as well). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-1971) Create a batch capable pseudo-whois ("netcat") enrichment Processor
[
https://issues.apache.org/jira/browse/NIFI-1971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard resolved NIFI-1971.
--
Resolution: Fixed
> Create a batch capable pseudo-whois ("netcat") enrichment Processor
> ---
>
> Key: NIFI-1971
> URL: https://issues.apache.org/jira/browse/NIFI-1971
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Andre
>Assignee: Andre
> Fix For: 1.1.0
>
>
> While the QueryDNS created can be used on low to medium volume enrichment and
> to licensed DNS based lookups (e.g. commercial use of SpamHaus) many
> enrichment providers prefer the use of bulk queries using pseudo whois API
> (a.k.a. netcat interface).
> as documented
> [here|https://www.shadowserver.org/wiki/pmwiki.php/Services/IP-BGP#toc6] the
> bulk interfaces work by connecting to port 43/TCP and sending a payload like:
> {code}
> begin origin
> 4.5.4.3
> 17.112.152.32
> 208.77.188.166
> end
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-2989) UI - Improve Update Policy dialog (Title, Text and Buttons)
[ https://issues.apache.org/jira/browse/NIFI-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-2989: - Resolution: Fixed Status: Resolved (was: Patch Available) +1. Merged into master. > UI - Improve Update Policy dialog (Title, Text and Buttons) > --- > > Key: NIFI-2989 > URL: https://issues.apache.org/jira/browse/NIFI-2989 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Andrew Lim >Assignee: Matt Gilman >Priority: Minor > Fix For: 1.1.0 > > Attachments: NIFI-2989_UpdatePolicyDialog.png > > > Referencing the attached screenshot: > 1. Change the title "Update Policy" to "Delete Policy" > 2. After the text "Are you sure you want to delete this policy?" add another > paragraph with: > "By deleting this policy, the inherited policy will be reinstated." > 3. Change the buttons from "Yes/No" to "Cancel/Delete". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-3021) Multi-line text boxes in View Configuration dialog (NFEL editors) do not honor newlines
[ https://issues.apache.org/jira/browse/NIFI-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-3021: - Affects Version/s: 1.0.0 Fix Version/s: 1.1.0 > Multi-line text boxes in View Configuration dialog (NFEL editors) do not > honor newlines > > > Key: NIFI-3021 > URL: https://issues.apache.org/jira/browse/NIFI-3021 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.0.0 >Reporter: Matt Burgess >Assignee: Scott Aslan > Labels: ui > Fix For: 1.1.0 > > Attachments: Configure_multiline.png, View_Configuration_multiline.png > > > In multi-line editors for properties in a Configure dialog, if Shift+Enter is > pressed, a newline will be inserted into the text. This is honored for the > Configure dialog. > However if the processor is running (such that Configure is not available and > instead View Configuration is), the newlines are not honored in the property > when viewed as a multi-line text box (editor/viewer). > One such processor that has this behavior is ExecuteScript, the Script Body > property (see attached screenshots). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-3021) Multi-line text boxes in View Configuration dialog (NFEL editors) do not honor newlines
[ https://issues.apache.org/jira/browse/NIFI-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-3021: - Resolution: Fixed Status: Resolved (was: Patch Available) > Multi-line text boxes in View Configuration dialog (NFEL editors) do not > honor newlines > > > Key: NIFI-3021 > URL: https://issues.apache.org/jira/browse/NIFI-3021 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.0.0 >Reporter: Matt Burgess >Assignee: Scott Aslan > Labels: ui > Fix For: 1.1.0 > > Attachments: Configure_multiline.png, View_Configuration_multiline.png > > > In multi-line editors for properties in a Configure dialog, if Shift+Enter is > pressed, a newline will be inserted into the text. This is honored for the > Configure dialog. > However if the processor is running (such that Configure is not available and > instead View Configuration is), the newlines are not honored in the property > when viewed as a multi-line text box (editor/viewer). > One such processor that has this behavior is ExecuteScript, the Script Body > property (see attached screenshots). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2912) Processor to generate a flow file from a string
[ https://issues.apache.org/jira/browse/NIFI-2912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-2912: - Fix Version/s: 1.2.0 Status: Patch Available (was: Open) > Processor to generate a flow file from a string > --- > > Key: NIFI-2912 > URL: https://issues.apache.org/jira/browse/NIFI-2912 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Alessio Palma >Assignee: Pierre Villard >Priority: Minor > Fix For: 1.2.0 > > > Currently to build a flow file from a string we need ReplaceText and > GenerateFlowFile with a size of 0B. > Is here a way to have use only the GenerateFlowFile processor ? > This will be handy when you are using templates in workflows. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2912) Processor to generate a flow file from a string
[ https://issues.apache.org/jira/browse/NIFI-2912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-2912: - Assignee: Pierre Villard Component/s: Extensions > Processor to generate a flow file from a string > --- > > Key: NIFI-2912 > URL: https://issues.apache.org/jira/browse/NIFI-2912 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Alessio Palma >Assignee: Pierre Villard >Priority: Minor > > Currently to build a flow file from a string we need ReplaceText and > GenerateFlowFile with a size of 0B. > Is here a way to have use only the GenerateFlowFile processor ? > This will be handy when you are using templates in workflows. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-3026) S2S initial connection behavior enhancement
[ https://issues.apache.org/jira/browse/NIFI-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15656478#comment-15656478 ] Pierre Villard commented on NIFI-3026: -- +1 thanks Koji, it'll be very useful! > S2S initial connection behavior enhancement > --- > > Key: NIFI-3026 > URL: https://issues.apache.org/jira/browse/NIFI-3026 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Core UI >Reporter: Koji Kawamura >Assignee: Koji Kawamura > > s2s client behavior and initial connection improvement is needed. > Current experience is this: I, as a client (e.g. minifi), connect to a nifi > cluster of e.g. 10 nodes. but i need to specify 1 node URL to establish this > connection. this node may not be available 100% and go down, in which case my > initial connection won't work. > Once S2S makes the first connection, it then has a list of all nodes, and can > check their status. But first connection failure would be a concern if the > specified URL is somehow not working. Usually for these problems, the client > should be able to specify multiple urls (according to multiple target cluster > nodes), comma-separated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-3016) UI - Allow template overriding
Pierre Villard created NIFI-3016: Summary: UI - Allow template overriding Key: NIFI-3016 URL: https://issues.apache.org/jira/browse/NIFI-3016 Project: Apache NiFi Issue Type: Improvement Components: Core UI Reporter: Pierre Villard Priority: Minor It would be useful that, when saving a template with a name of an already existing template, the user is proposed the option to override the existing one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-3013) GetSNMP start/stop/start null point exception
[ https://issues.apache.org/jira/browse/NIFI-3013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-3013: - Affects Version/s: (was: 1.1.0) > GetSNMP start/stop/start null point exception > - > > Key: NIFI-3013 > URL: https://issues.apache.org/jira/browse/NIFI-3013 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.0.0 > Environment: Red Hat 7.x with net-snmp >Reporter: João Henrique Ferreira de Freitas > Labels: patch > Fix For: 1.1.0 > > > Starting a GetSNMP processor previously stoped causes null pointer exception. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
