ered with my upgraded version of NiFi.
On Mon, Mar 20, 2017 at 12:14 PM, Oleg Zhurakousky
<ozhurakou...@hortonworks.com<mailto:ozhurakou...@hortonworks.com>> wrote:
James
Basically my next question would be if admin-wise the new user had been given
the correct permissions as I beli
MQP. Thanks
very much for looking more closely at this. -Jim
On Mon, Mar 20, 2017 at 11:55 AM, Oleg Zhurakousky
<ozhurakou...@hortonworks.com<mailto:ozhurakou...@hortonworks.com>> wrote:
James, while I am looking couple of questions.
1. Did you actual RabbitMQ version has changed si
help. I am very interested in why I seem to be
getting this bad header message now. -Jim
On Mon, Mar 20, 2017 at 10:00 AM, Oleg Zhurakousky
<ozhurakou...@hortonworks.com<mailto:ozhurakou...@hortonworks.com>> wrote:
James, any chance you can provide stack trace from the logs?
Cheers
Oleg
James, any chance you can provide stack trace from the logs?
Cheers
Oleg
> On Mar 20, 2017, at 15:34, James McMahon wrote:
>
> Good morning. I am using a NiFi 0.7 code base. I recently upgraded to that
> from a 0.6.x version. I'm limited to that 0.7.x baseline now. I
Jim
I see where you’re going with this and I do believe that it is a valuable
enterprise feature. Unfortunately, template XML files are not the bootstrap
source of the flow such as flow.gz file. They are only used for
save/move/migrate/etc., which means any modifications to them are not going
Olav
The fact that the processor does not detect the dead process that was initiated
by it does appear to be a bug. Would you mind raising JIRA -
https://issues.apache.org/jira/browse/NIFI so we can take a look.
Cheers
Oleg
On Mar 1, 2017, at 7:29 PM, Olav Jordens
have
implemented to deal with failures.
- Nick
On Wed, Mar 1, 2017 at 12:29 PM, Oleg Zhurakousky
<ozhurakou...@hortonworks.com<mailto:ozhurakou...@hortonworks.com>> wrote:
Nick
Since you’ve already designed Process Group (PG) that is specific to failed
flow files, I am not sure I
Nick
Since you’ve already designed Process Group (PG) that is specific to failed
flow files, I am not sure I understand your last question “. . .How do you
manage failure relationships?. . .”.
I am assuming that within your global flow all failure relationships are sent
to this PG, which
Adam,
I think if my math is correct, what you seeing is correct since 22T23:01:23 + 7
= 23T06:01:23
Am i missing something?
Cheers
Oleg
On Feb 22, 2017, at 6:42 PM, Adam Lamar
> wrote:
Hi,
I recently noticed some issues with time parsing in
Manojkumar
The short answer is “yes" it is possible since NAR by definition is a
self-sustaining (resources, class path etc) package.
However, I am a bit confused about the UI changes you are mentioning and can’t
seem to find a correlation to NAR in general. Are you saying you have changed
don't
know file name which is stored inside directory .
${absolute.path}\${filename}
Note: while using GetFile processor doesn't have upstream connections but i
need processor which fetch file inside directory using directory name without
give filename.
Many thanks
On Thu, Feb 2, 2017 at 7
Prabhu
Not sure I fully understand.
While indeed GetFile does not allow for an incoming connection, it does allow
for your use case to happen indirectly by monitoring a predefined directory.
So, one PG finishes and produces a file that is being put into a directory
monitored by another PG’s
Brian
You are correct, at the moment it is somewhat static doe to the fact that for
the routing key the ‘expressionLanguageSupported’ defaults to false. We can
definitely make that change which would allow you to use NiFi Expression
Language to make it dynamic.
Please raise a JIRA ticket -
Alessio
Outside of screen shot I am not sure you have many options, at least at the
moment.
Printing something like a flow is more complicated that it may seem at first,
due to formatting issues. Landscape or Portrait, the paper size, etc., and what
if the flow doesn’t fit? Should it get
Panos
What version of NiFi are you using?
The issue with the NiFi templates diffs has actually been addressed (see
https://issues.apache.org/jira/browse/NIFI-826). So as of NiFi 1.0 you can
export templates in a deterministic way and then execute diff on them to see
only what was changed. It
Nicolas
I don’t believe there is. A possible workaround may be to use ExecuteScript
processor, but I’ll be honest I have not tried.
That said, what you are describing seems to be rather valuable feature and if
the underlying API (or other libraries) allows for it then we can definitely
make
Actually it may be related to the known JDK issue that has been reported
several times.
Does it look something like this (see below)? If so then please upgrade your
JDK to something newer. Something above 1.8.0_65
Cheers
Oleg
. . .
to that process let me
try that. Thank you. -Jim
On Wed, Dec 14, 2016 at 2:57 PM, Oleg Zhurakousky
<ozhurakou...@hortonworks.com<mailto:ozhurakou...@hortonworks.com>> wrote:
James
Not sure what kind of liberties you have with your environment, but here is
what you can do outside o
rrectly. Let me ask you
this: is there any way I can build this into my code baseline - either through
a code mod and rebuild or as a custom plugin feature specific to my flow?
Thanks very much for your help.
On Wed, Dec 14, 2016 at 10:59 AM, Oleg Zhurakousky
<ozhurakou...@hortonworks.com&l
James
Could you also let us know what version of NiFi you are using? The issue with
properly handling InvalidPathException was fixed in NiFi 0.7.0 as part of
https://issues.apache.org/jira/browse/NIFI-920
It essentially has this catch block:
} catch (final ProcessException |
Brian
Thank you for detailed explanation.
I don't believe you're doing anything wrong. We just need do add the feature
you describe (pulling credentials from certificate).
Would you mind creating JIRA ticket and if at all possible attach the sample
code that demonstrates exactly what you're
JIRA tickets and in the
meanwhile will battle to bring our rMQ out of the dark ages. Thank you very
much once again for your help investigating this behavior Oleg.
On Thu, Dec 8, 2016 at 3:08 PM, Oleg Zhurakousky
<ozhurakou...@hortonworks.com<mailto:ozhurakou...@hortonworks.com>> wrote:
Ja
d E. While I won't be able to upgrade my NiFi
baseline because of other organizational dependencies, I can certainly
eliminate the problems due to the rMQ and Erlang versions.
Does this seem like a reasonable approach in your opinion?
On Thu, Dec 8, 2016 at 1:45 PM, Oleg Zhurakousky
On Dec 8, 2016, at 1:04 PM, Oleg Zhurakousky
<ozhurakou...@hortonworks.com<mailto:ozhurakou...@hortonworks.com>> wrote:
Got it, let me dig in, i got time now
On Dec 8, 2016, at 12:46 PM, James McMahon
<jsmcmah...@gmail.com<mailto:jsmcmah...@gmail.com>> wrote:
N
ange by NiFi.
On Thu, Dec 8, 2016 at 12:41 PM, Oleg Zhurakousky
<ozhurakou...@hortonworks.com<mailto:ozhurakou...@hortonworks.com>> wrote:
James
My apology , but I am not sure I am clear on the second part of your
explanation. It appears you are saying that when you bound the Exch
a Connection that is shown with State of blocking.
and that is how it remains indefinitely. Nothing shows up in the exchange or in
the queue.
Jim
On Thu, Dec 8, 2016 at 8:26 AM, Oleg Zhurakousky
<ozhurakou...@hortonworks.com<mailto:ozhurakou...@hortonworks.com>> wrote:
Fair enough
s
to me how / if that needs to be done. I'll dig for that.
I am using NiFi 0.6.1.
Cheers and thanks again,
Jim
On Thu, Dec 8, 2016 at 7:46 AM, Oleg Zhurakousky
<ozhurakou...@hortonworks.com<mailto:ozhurakou...@hortonworks.com>> wrote:
James I have not used AMQP in a while (although
ng the exchange, but I
thought it best to ask. I see nothing on the management console that alludes to
enabling the exchange or the queue.
Jim
On Thu, Dec 8, 2016 at 7:24 AM, Oleg Zhurakousky
<ozhurakou...@hortonworks.com<mailto:ozhurakou...@hortonworks.com>> wrote:
James
Aside from
James
Aside from “closing” failure message, do you see any other errors or stack
traces in the logs?
Meanwhile I’ll try to dig more with what I have?
Oleg
On Dec 8, 2016, at 7:13 AM, James McMahon
> wrote:
Hello I am trying to use a Publish
or Libraries in my
application code? or can I just build a HTTP Server and tell Nifi JVM that "hey
here is my Application (it runs on specific IP and port) and it uses HTTP".
On Wed, Dec 7, 2016 at 12:00 PM, Oleg Zhurakousky
<ozhurakou...@hortonworks.com<mailto:ozhurakou...@hortonwork
with my HTTP based server?
On Wed, Dec 7, 2016 at 4:55 AM, Oleg Zhurakousky
<ozhurakou...@hortonworks.com<mailto:ozhurakou...@hortonworks.com>> wrote:
Kant
There are couple of questions here so, let me try one at the time.
1. “why Nifi would act as a server?”. Well, NiFi is a runtime e
ave Java client library as well.
>
> http://nsq.io/clients/tcp_protocol_spec.html
>
>
> On Tue, Dec 6, 2016 at 5:14 PM, Oleg Zhurakousky
> <ozhurakou...@hortonworks.com<mailto:ozhurakou...@hortonworks.com>> wrote:
>>
>> Hi Kant
>>
>> What you’re try
Hi Kant
What you’re trying to accomplish is definitely possible, however more
information may be needed from you.
For example, the way I understand your statement about “integration with many
systems” is something like JMS, Kafka, TCP, FTP etc…. If that is the case such
integration is
I am +1 on this as I’ve seen many cases in the field where it is applicable and
as you mention exponential back off is one of the common one. That said, I am
wondering if that has to be a processor at all? Actually let me answer my own
question. There are definitely cases where it has to be a
Sravani
Would you be able to provide a full stack trace of the connection exception.
Also, while I assume you are providing the correct connection properties (i.e.,
host,port etc) I would still recommend to check the they are correct, but in
any event the full stack trace would definitely help
ses we
> should figure out what happened and make it not possible. We must
> always work to eliminate the possibility of corruption causing events
> and work to recover well in the face of corruption...
>
> On Fri, Nov 11, 2016 at 2:57 PM, Oleg Zhurakousky
> <ozhurakou.
ro’ to filenames (for reading in Spark subsequently)
11. Save to HDFS
Does this help at all?
If you need anything else just shout.
Regards
Conrad
additional out of shot
• compression level : 1
• Keep Path : false
From: Oleg Zhurakousky
<ozhurakou...@hortonworks.c
Conrad
Any chance you an provide a bit more info about your flow?
I was able to find a condition when something like this can happen, but it
would have to be with some legacy NiFi distribution, so it’s a bit puzzling,
but i really want o see if we can close the loop on this.
In any event I
currently do not have support.
Once again, please raise the JIRA if you believe that it’s important to add
such feature.
Cheers
Oleg
On Nov 1, 2016, at 9:29 AM, Oleg Zhurakousky
<ozhurakou...@hortonworks.com<mailto:ozhurakou...@hortonworks.com>> wrote:
Nirmal
While ConsumeJMS w
an I programmatically create a template during
> development time?'
>
> We don't have any direct public API's to accomplish this that I am
> aware of though it is an interesting idea.
>
> Can you help direct us to where you're more interested right now so we
> can help mos
Just to add to what James has already stated, templates also have been polished
quite a bit to be more deterministic and source control friendly so they are an
ideal artifact to be versioned and kept in source control repo.
Cheers
Oleg
On Sep 14, 2016, at 3:02 PM, James Wing
Also, if you care to share the code for you processor we’ll be happy to look
and help you find the problem.
As Joe pointed out we make every possible attempt to stop it, but if your
custom code is blocking and does not react to interrupts then there is not much
we can do stoping short of
Just to add to what Mark said, perhaps you can raise a JIRA (as new Feature) to
add an optional property to the processor config where such property would
point to an additional class path (to support your case and many others).
We are doing it with few processors already.
Cheers
Oleg
On Jul
PM, Oleg Zhurakousky
<ozhurakou...@hortonworks.com<mailto:ozhurakou...@hortonworks.com>> wrote:
Also, the NPE was a bug that was fixed a while back and will be available on
O.7.
Any chance you can build from 0.x branch?
Cheers
Oleg
Sent from my iPhone
On Jun 11, 2016, at 16:58,
Also, the NPE was a bug that was fixed a while back and will be available on
O.7.
Any chance you can build from 0.x branch?
Cheers
Oleg
Sent from my iPhone
On Jun 11, 2016, at 16:58, Juan Sequeiros
> wrote:
Hi Pat,
Looks to me that on your
ings.put(matcher, value);
}
}
The exception is coming from the line where it is looping over the entryset:
for (Map.Entry<CharSequence, String> entry : mappings.entrySet()) {
Right after that loop it calls mappings.clear() so if the RequestFactory is
shared by multiple threads (which I think it is
I am not sure I understand the question
Sent from my iPhone
> On Jun 8, 2016, at 18:23, Igor Kravzov wrote:
>
> Guys, what can be wrong?
Huagen, it appears you have DEBUG level logging enabled.
Oleg
On Jun 4, 2016, at 21:27, Huagen peng
> wrote:
Hi,
I got excessive logging from my NiFi instance. I suddenly see logging like the
following going into nidi-bootstrap.log very
Kumiko
It appears that the current state of the source you linked in is not in sync
with what is in the stack trace. Perhaps you have made some code modifications
(e.g., line 218 is an empty line in code while it has a pointer in the star
trace).
In any event, from what I can see the error is
Huagen,
I also want to apologize for my spell-checker butchering your name ;)
Cheers
Oleg
On Jun 3, 2016, at 8:03 AM, Oleg Zhurakousky
<ozhurakou...@hortonworks.com<mailto:ozhurakou...@hortonworks.com>> wrote:
Huge
Just to close the loop on this one, I also wanted to point o
Huge
Just to close the loop on this one, I also wanted to point out this JIRA
https://issues.apache.org/jira/browse/NIFI-1926 for general purpose aggregation
processor which indeed would support multiple connections, configurable
aggregation, release and correlation strategies.
It would be
Igor, you may want to go they Kafka mailing list. I've seen this several times,
and all I know it has to do with broker
I'll look in more details once back on line
Oleg
Sent from my iPhone
On May 26, 2016, at 17:20, Igor Kravzov
> wrote:
Source)
[na:1.8.0_91]
at java.lang.Thread.run(Unknown Source) [na:1.8.0_91]
On Thu, May 26, 2016 at 4:28 PM, Oleg Zhurakousky
<ozhurakou...@hortonworks.com<mailto:ozhurakou...@hortonworks.com>> wrote:
That is because the processor is already in the process of being stopped and
you pro
That is because the processor is already in the process of being stopped and
you probably tried to start it again.
That said, the fact that you see it is probably a bug since you should not be
able to see ‘start’ action until processor is fully stopped.
Could you please share more details as to
As some of you know, we are in the process of providing a set of new Processors
that uses the new Kafka API (0.9+). This is a new NAR and will live for a while
along side with the old Kafka NAR.
The new Processors are called PublishKafka and ConsumeKafka (specifically to
emphasize the new
ps://issues.apache.org/jira/browse/NIFI-1837?filter=12331874
>
> As a community is there a preference given to JIRA's with Patch or GH PR's
> or are they all treated with the same priority?
>
> Thanks,
> Andrew
>
> On Tue, May 3, 2016 at 11:12 AM, Oleg Zhurakousky &l
Guys
I’d like to use this opportunity to address all members of the NiFi community
hence this email is sent to both mailing lists (dev/users)
While somewhat skeptical when I started 6 month ago, I have to admit that now I
am very excited to observe the growth and adaption of the Apache NiFi
Ralf
While we are indeed working on it at the moment, please do file an enhancement
request with all the details. It’s always good for tracking purposes.
Cheers
Oleg
> On May 2, 2016, at 10:25 AM, Perko, Ralph J wrote:
>
> Thanks for the update. Is there any discussion
Manish
That is an interesting suggestion. I personally don't see issues with it and
generally do believe it would improve user experience, but interested what
others think.
Cheers
Oleg
Sent from my iPhone
On Apr 24, 2016, at 08:46, Manish Gupta 8
Yep, just watch the JIRA for PR
Sent from my iPhone
On Apr 7, 2016, at 12:35, indus well
<indusw...@gmail.com<mailto:indusw...@gmail.com>> wrote:
Thank you, Oleg. I'll be testing it out once it is available next week.
Regards,
Indus
On Thu, Apr 7, 2016 at 10:48 AM, Oleg
Not at the moment unless you are using HDP distribution of Kafka broker 0.8
which has Kerberos support while ASF version of 0.8 Kafka doesn't.
That said, we are working on new pair of Kafka processors that will rely on
Kafka 0.9 API and broker where such support is provided. It is slated for
29, 2016, at 9:26 AM, Oleg Zhurakousky
<ozhurakou...@hortonworks.com<mailto:ozhurakou...@hortonworks.com>> wrote:
Vincent
I do have a suggestion for you but need a bit more time to craft my response.
Give me till tonight EST.
Cheers
Oleg
On Mar 29, 2016, at 8:55 AM, Vin
Vincent
This sounds more like an architectural question and even outside of NiFi in
order to achieve that especially in the distributed environment one would need
some kind of coordination component. And while we can think of variety of way
to accomplish that I am not entirely convinced that
back seems to correspond to this line in PutKafka.java
>
> final byte[] value = new byte[(int) flowFile.getSize()];
>
>
> Can the flow file size be negative?
>
>
>
> On 3/23/16, 6:24 PM, "Oleg Zhurakousky" <ozhurakou...@hortonworks.com> wrote
Russell
First, thank you for taking initiative!
Indeed we need to bring that JIRA to a closure. Having said that there is
actually a link in our Contributor Guide on interactive debugging
ew-Kafka-0-8-2-0-1-kafka1-3-1-p0-9-Parcel-What-are-the-changes/td-p/30506
http://archive.cloudera.com/kafka/parcels/1.3.1/
Michael
On Tue, Mar 15, 2016 at 12:01 PM, Oleg Zhurakousky
<ozhurakou...@hortonworks.com<mailto:ozhurakou...@hortonworks.com>> wrote:
Michael
What is KAFKA-0.8.
.1 Kafka NAR still work in 0.5.x?
If the NAR doesn't depend on any other NARs I would think it would still work,
and could be a work around for those that need to stay on Kafka 0.8.2.
On Sunday, February 21, 2016, Oleg Zhurakousky
<ozhurakou...@hortonworks.com<mailto:ozhurakou...@horto
est <josh_w...@bose.com<mailto:josh_w...@bose.com>>
Bose Corporation
On Sun, 2016-02-21 at 15:02 +, Oleg Zhurakousky wrote:
Josh
Also, keep in mind that there are incompatible property names in Kafka between
the 0.7 and 0.8 releases. One of the change that wen
on this property it’s
value was completely ignored with 0.8 client libraries (you could actually see
the WARN message to that effect) and now it is not ignored, so take a look and
see if tinkering with its value changes something.
Cheers
Oleg
On Feb 20, 2016, at 6:47 PM, Oleg Zhurakousky
016-02-20 at 15:46 -0500, Joe Witt wrote:
And also what version of Kafka are you using?
On Feb 20, 2016 3:37 PM, "Oleg Zhurakousky"
<ozhurakou...@hortonworks.com<mailto:ozhurakou...@hortonworks.com>> wrote:
Josh
Any chance to attache the app-log or relevant stack trace?
Josh
Any chance to attache the app-log or relevant stack trace?
Thanks
Oleg
On Feb 20, 2016, at 3:30 PM, West, Joshua
> wrote:
Hi folks,
I've upgraded from Nifi 0.4.1 to 0.5.0 and I am no longer able to use the
GetKafka processor. I'm seeing
Jeff, what you are describing is in works and actively discussed
https://cwiki.apache.org/confluence/display/NIFI/Extension+Registry
and
https://cwiki.apache.org/confluence/display/NIFI/Component+documentation+improvements
The last one may not directly speaks to the “ExtensionRegistry”, but if
Obaid
Thanks for reaching out!
Currently it is not possible to wire a flow the way you describe.
Wha you are asking is a true Event Driven Consumer paradigm which been
discussed internally a lot lately, so it would be very interesting to get your
perspective as to why do you believe it is
Suddep
Also, when you say push/retrieve do you mean publish/subscribe or put/get since
Redis is both storage and messaging system. As Joe mentioned we don’t have any
at the moment but would be nice to start prioritizing.
Cheers
Oleg
> On Jan 13, 2016, at 12:41 AM, Joe Witt
Chris
Are you sure you are providing the correct logs? I can’t see a single mention
of JMS nor any stack traces which would definitely be there given what you see
in UI.
Also, the fact that you see NPE is definitely a bug that we have to fix (user’s
should never see NPE), so that can be filed.
This is definitely possible and been done. What makes it difficult at times is
to have all required NiFi dependencies in the process space of a given test.
I've actually proposed a separate module for these types of 'headless' flow
tests. It actually helped me to discover some of the bugs as
Douglass
Just looked at the code and would like to ask you if you could try it with our
latest snapshot. This was a bug that was fixed.
Thanks
Oleg
On Dec 1, 2015, at 11:23 AM, Douglas Doughty
> wrote:
Hi All,
I’ve enjoyed working with nifi,
Adam, thanks for doing all the research and pointing out where the problem is.
It is definitely a bug.
Joe I’ve raised the https://issues.apache.org/jira/browse/NIFI-1200
Cheers
Oleg
On Nov 19, 2015, at 1:18 PM, Joe Percivall
> wrote:
I am still a bit confused with the problem that is being solved here.
“replication” implies some type of redundancy allowing processing that failed
“here” to be resumed “there”.
What I am reading here is more about "content based routing” - (route to their
respective workflows based on their
Multiple processors or multiple instance of the same processor?
Could you also elaborate on your use case a bit more, simply because their may
be several ways of accomplishing your goal and to pick the best understanding
of the underlying problem would help.
Thanks
Oleg
On Nov 10, 2015, at
Unfortunately I can’t seem to even see how it would be possible for NiFi to
tell you that since your custom Processors are running within the same JVM as
NiFi.
Having said that the 800% tells me that you probably have some processor with
custom thread pool where each thread is spinning in the
Chris
Could you elaborate on your use case a bit more? Specifically about where is
the source of data you want to pump into hive (e.g., Streaming, bulk file load
etc.)
Cheers
Oleg
On Oct 16, 2015, at 8:56 AM, Christopher Wilson
> wrote:
Joe,
Ben
I don’t think it needs an incoming FlowFile. It is a scheduled component and
will retrieve contents based on how you configure scheduling.
Have you tried it without incoming FlowFiles?
Cheers
Oleg
On Oct 15, 2015, at 3:38 PM, Ben Meng
>
May be I am missing certain details, but are you asking about custom bundle
(NAR) or using existing NiFi bundles which would somehow have to delegate to
some custom JARs?
I am thinking that if you are using custom client, you may need a custom bundle.
Oleg
> On Oct 12, 2015, at 9:29 AM, Andre
84 matches
Mail list logo