here to resolve the
issue, when you said to add the column alias name. Can someone help clarify
what was done to fix this?
Thanks,
Scott
On Fri, May 8, 2020 at 11:46 AM Luis Carmona
wrote:
>
>
> Thanks Juan Pablo.
>
> It did work !!
>
> Thanks.
>
> LC
>
>
>
.
Azure Monitor Query client library for Java | Microsoft Docs
<https://docs.microsoft.com/en-us/java/api/overview/azure/monitor-query-readme?view=azure-java-stable>
Thanks,
Scott
, just pure
NiFi( Java ) solution.
Thanks,
Scott
challenges or could point me to a
solution or workaround.
Thanks,
Scott
I'm using openjdk-11.0.7.10-4 as I was on the previous version of NiFi.
I'll look around for a free Java profiler to use to dig deeper.
Thanks,
Scott
On Sat, Jul 31, 2021 at 7:56 PM Joe Witt wrote:
> Scott
>
> Nope this sounds pretty dang unique
>
> What JVM? May need to at
hardware issues
usually, all looks normal. I added 1/3 more memory to the JVM, now at 24GB,
just for good measure, but that had no effect.
Is it possible there are some hidden performance issues going on within the
JVM I need a special tool to see?
Any help would be greatly appreciated.
Thanks,
Scott
Joe,
I'm not sure. What would be involved? I'm not familiar with a NiFi blog,
can you point me to some examples?
Thanks,
Scott
On Tue, Jul 27, 2021 at 10:00 AM Joe Witt wrote:
> Scott
>
> This sounds pretty darn cool. Any chance you'd be interested in
> kicking out a blog on it
Matt/all,
I was able to solve my problem using the QueryNiFiReportingTask with
"SELECT * FROM CONNECTION_STATUS WHERE isBackPressureEnabled = true" and
the new LoggingRecordSink as you suggested. Everything is working
flawlessly now. Thank you again!
Scott
On Wed, Jul 21, 2021 at 5:
Excellent! Very much appreciate the help and for setting me on the right
path. I'll give the queryNiFiReportingTask code a try.
Scott
On Wed, Jul 21, 2021 at 3:26 PM Matt Burgess wrote:
> Scott et al,
>
> There are a number of options for monitoring flows, including
> backpress
this happens, I would be good.
I have looked at the new metrics and reporting tasks but still haven't
found the right thing to do to get notified when any queue in my
instance fills up. Are there any examples of using them for a similar task
you can share?
Thanks,
Scott
On Wed, Jul 21, 2021 at 11:29
this to be an automated
solution, so that is not going to work. Has anyone else tackled this
problem with a secure LDAP enabled cluster?
Thanks,
Scott
in this list. I think I am just
completely borking the decrypt.
Scott
> On Dec 18, 2018, at 1:22 PM, Bryan Bende wrote:
>
> What happens when you try to decrypt it? Is there an error?
>
> On Tue, Dec 18, 2018 at 2:16 PM Scott Howell <mailto:scotthow...@payitgov.com>> wrote:
a screenshot of my configuration currently. I’m figuring this is a
really easy problem to solve but I have been scratching my head.
Thanks,
Scott
-Registry?
Scott
Thank you for the responses Aurelien and Abdou. That reminded me that the
ListFile processor I'm using could filter with min and max age. That should
work for my use-case, as the files should only get generated once daily.
Scott
On Thu, Sep 13, 2018 at 5:43 AM Abdou B wrote:
> Hello,
> D
, and older files can
be discarded. Is there a way to filter the content of a queue this way?
Thanks,
Scott
le system you'll want to match
> the version of winutils.exe with the version of Hadoop (usually 2.7.3
> for slightly older NiFi versions or 3.0.0 for the latest version(s) I
> think) for best results.
>
> Regards,
> Matt
>
> [1] https://github.com/steveloughran/winutils
>
download and call from NiFi?
On Wed, Aug 15, 2018 at 12:01 PM, Mike Thomsen
wrote:
> > Mike, that's a good tip. I'll test that, but unfortunately, I've already
> committed to Windows.
>
> You can run both Docker and the standard NiFi docker image on Windows.
>
> On Wed, Aug 15,
Mike, that's a good tip. I'll test that, but unfortunately, I've already
committed to Windows.
What about a script? Is there some tool you know of that can just be called
by NiFi to convert an input CSV file to a Parquet file?
On Wed, Aug 15, 2018 at 8:32 AM, Mike Thomsen
wrote:
> Sc
>
>
> On Wed, Aug 15, 2018 at 10:16 AM, scott wrote:
> > If I install a Hadoop client on my NiFi host, would I be able to get past
> > this error?
> > I don't understand why this processor depends on Hadoop. Other projects
> like
> > Drill and Spark don't have su
<
gardellajuanpa...@gmail.com> wrote:
> It's a warning. You can ignore that.
>
> On Tue, 14 Aug 2018 at 18:53 Bryan Bende wrote:
>
>> Scott,
>>
>> Sorry I did not realize the Hadoop client would be looking for this
>> winutils.exe when running on Window
\winutils.exe in the
Hadoop binaries
Unable to load native-hadoop library for your platform... using
builtin-java classes where applicable
Failed to write due to java.io.IOException: No FileSystem for scheme
BTW, I'm using NiFi version 1.5
Thanks,
Scott
On Tue, Aug 14, 2018 at 12:44 PM, Bryan Bende
be using instead of
Parquet?
Thanks for your time,
Scott
flow file. Then use
those variables to specify the fields on the JSONPathReader. Might not be the
best method but I have used it and it works fairly well.
Scott
> On Jun 21, 2018, at 7:00 AM, Kelsey RIDER
> wrote:
>
> OK, thanks for the heads-up!
>
> If I could make
Joe,
We are using the GetFTP processor with differing Timers. In Dev and Staging we
are using a Time based Timer but in Production we use a CRON based timer
because of customer requirements.
Thanks,
Scott
> On May 4, 2018, at 9:11 AM, Joe Witt <joe.w...@gmail.com> wrote:
>
>
the flow is in.
Scott
Matt,
Thank you for your comments and for creating a Jira to add this
improvement. I'll take your advice on the logging bulletins, I agree
that can be done as a separate function.
Thanks,
Scott
On 04/11/2018 06:31 AM, Matt Gilman wrote:
tors you mentioned are bulletins. Bulletins
he.org> wrote:
>
> Thanks; that certainly narrows it down. It could be that you’ve uncovered a
> bug with the LdapIdentityProvider when using START_TLS. I’ll try to recreate
> it and dig into it on my end. Thanks for sharing all this info.
>
> Kevin
>
> From: Scott
I was able to remove the TLS information in the identity-provider.xml and was
able to use my remote LDAP to login. So I think I am narrowing down the issue.
> On Apr 10, 2018, at 2:57 PM, Kevin Doran <kdo...@apache.org> wrote:
>
> Thanks Scott,
>
> I don’t see any
not.
> On Apr 10, 2018, at 2:18 PM, Scott Howell <scotthow...@mobilgov.com> wrote:
>
> Thanks Kevin for sending that back,
>
> This is what I see when looking at the Headers on the login.
>
>
> The version of Nifi-Registry I am running is 0.1.0. What confuses
e interfering with HTTP headers?
>
> What version of NiFi Registry are you using? 0.1.0 or a version built from
> source?
>
> Hope this helps,
> Kevin
>
>
> On 4/10/18, 14:59, "Scott Howell" <scotthow...@mobilgov.com
> <mailto:scotthow...@mobi
Yes I changed that from USE_DN to USE_USERNAME to do some troubleshooting.
Sorry for not changing it back before I sent this stuff out.
> On Apr 10, 2018, at 2:01 PM, Mike Thomsen <mikerthom...@gmail.com> wrote:
>
> Scott,
>
> In your last email, the way I read it you foun
Yes I did, I had Nifi-registry working with a local instances of LDAP running.
It’s now not cooperating since I moved to using Jumpcloud.
> On Apr 10, 2018, at 1:56 PM, Kevin Doran <kdo...@apache.org> wrote:
>
> Hi Scott,
>
> Did you configure nif
Thanks for the all the help yesterday standing up LDAP for NIFI. I was able to
troubleshoot and fix the issues myself. I am running into a unique issue with
my Nifi-Registry when I try to login with my LDAP credentials like I do for the
nifi cluster I get in my logs with this:
2018-04-10
So I changed two things.
1. I updated the START_TLS
with START_TLS this allowed nifi to connect to Jumpcloud.
2. USE_DN from USER_USERNAME and
everything began to work.
> On Apr 9, 2018, at 3:14 PM, Andy LoPresto <alopre...@apache.org> wrote:
>
> Scott,
>
> One n
That is what is inside of
> On Apr 9, 2018, at 3:03 PM, Scott Howell <scotthow...@mobilgov.com> wrote:
>
> Yep let me send it over.
>
>
>ldap-provider
>org.apache.nifi.ldap.LdapProvider
>ANONYMOUS
>
>uid=nifi,ou=Us
ldap://ldap.jumpcloud.com:389
ou=Users,o={redacted},dc=jumpcloud,dc=com
uid={0}
USE_USERNAME
12 hours
> On Apr 9, 2018, at 3:01 PM, Kevin Doran <kdo...@apache.org> wrote:
>
> Scott,
>
> I've never implemented NiFi with JumpC
of “Unable
to validate the supplied credentials. Please contact the system administrator.”
In the UI in my nifi-user.log I am seeing [LDAP: error code 32 - No Such
Object] when its trying to look up the LDAP user.
Scott
way to do that? Or, at the very least, can I make
them last a lot longer?
Thanks,
Scott
On 04/06/2018 07:20 AM, Matt Gilman wrote:
Scott,
What version of NiFi are you running? If your running NiFi 1.x then
the parent process group ID shouldn't be necessary. The fully
qualified URL
within my
flow, so that I can build a fully qualified URL and put it into my log
to make operational troubleshooting easier. Does anyone have any advice
on how I can get the processGroupId as a variable/attribute I can
include in my log?
Thanks,
Scott
servers seemed to
fix it but that hasn’t worked this time.
> On Apr 5, 2018, at 11:12 AM, Andy LoPresto <alopre...@apache.org> wrote:
>
> Hi Scott,
>
> Can you check the contents of your nifi.properties (specifically the
> nifi.cluster.* and nifi.zookeeper.* sections
: Failed
to create socket due to: java.net.UnknownHostException:
nifi-2.dev.{redacted}.com
I had this problem before and just completely rebuilt the cluster, I did this,
this time and that doesn’t seem to be working. If anyone has any insight that
would be great.
Thanks,
Scott
tance I am running separate to the cluster.
> On Mar 30, 2018, at 10:43 AM, Pierre Villard <pierre.villard...@gmail.com>
> wrote:
>
> Oh ok now I see what you mean with the double quotes... didn't notice in the
> first place =/
> How did you generate t
success for
uid=scott,ou=users,dc={redacted},dc=com
2018-03-30 15:32:42,325 INFO [NiFi Web Server-18]
o.a.n.w.s.NiFiAuthenticationFilter Attempting request for
(
> On Mar 30, 2018, at 9:48 AM, Pierre Villard <pierre.villard...@gmail.com>
> wrote:
>
> Hi Scott,
>
> Can you have a look at the authorizations.xml file? (and share the content of
> it to confirm that nod
and in
the nifi-user.log.
This is what I see in the UI: Untrusted proxy CN="nifi-2.dev.{redacted}.com,
OU=Nifi”
In my authorizers.xml I have this:
file-provider
org.apache.nifi.authorization.FileAuthorizer
/opt/config/authorizations.xml
/opt/config/users.xml
uid=scott,ou=
il.com> wrote:
>
> Ok, can you provide the stacktrace from nifi-app.log when it gets the
> socket timeout?
>
> On Wed, Mar 28, 2018 at 3:45 PM, Scott Howell <scotthow...@mobilgov.com>
> wrote:
>> Yes I have https: defined on the registry client.
>>
>
Yes I have https: defined on the registry client.
> On Mar 28, 2018, at 2:29 PM, Bryan Bende <bbe...@gmail.com> wrote:
>
> Just to double-check, when you defined the Registry Client in NiFi,
> did you enter the URL starting with "https" ?
>
> On Wed, Mar
anyone ran into this when using a self-signed certificate and
not the nifi-tls-toolkit?
Thanks,
Scott
Daniel,
That worked perfectly. Thank you for the help, and for creating this
great tool.
Scott
On 03/25/2018 02:50 AM, Daniel Chaffelson wrote:
Hi Scott,
NiPyApi provides a python client for this purpose. There are calls to
get/set variables in the canvas module.
https://github.com
Thank you Daniel and Charlie. Excellent tips. That's what I needed to
get started.
Scott
On 03/25/2018 02:50 AM, Daniel Chaffelson wrote:
Hi Scott,
NiPyApi provides a python client for this purpose. There are calls to
get/set variables in the canvas module.
https://github.com/Chaffelson
mated
job when the remote host changes. This would be especially useful for
processors that don't allow an input relationship.
Any suggestions or comments would be welcome.
Thanks,
Scott
ith two
> /proxy policies, one for each node.
>
> So if you do that then your cluster starts up, but when you access the
> UI then you get the untrusted proxy?
>
>
> On Wed, Mar 21, 2018 at 12:58 PM, Scott Howell <scotthow...@mobilgov.com>
> wrote:
>> use
l
>
> On Wed, Mar 21, 2018 at 12:37 PM, Scott Howell <scotthow...@mobilgov.com>
> wrote:
>> Thanks I have checked that and the whitespace is correct in user.xml.
>>
>> I did make a change to my authorizer.xml
>>
>>
>>
>>file-pro
Thanks I have checked that and the whitespace is correct in user.xml.
I did make a change to my authorizer.xml
file-provider
org.apache.nifi.authorization.FileAuthorizer
/opt/config/authorizations.xml
/opt/config/users.xml
uid=scott,ou=users,dc=mobilgov,dc=com
One other thing I am seeing and I don’t know if this is an issue or not in my
authorizations.xml I do not have a policy for /proxy with action=“R” only
action=“W”.
> On Mar 21, 2018, at 11:03 AM, Scott Howell <scotthow...@mobilgov.com> wrote:
>
> Thanks for that. I am still get
21, 2018, at 10:23 AM, Bryan Bende <bbe...@gmail.com> wrote:
>
> All identity strings are case & whitespace sensitive.
>
> The node identities in your authorizers.xml have no whitespace, and
> the identity showing in the logs does.
>
> On Wed, Mar 21, 2018 a
org.apache.nifi.authorization.FileAuthorizer
/opt/config/authorizations.xml
/opt/config/users.xml
uid=scott,ou=users,dc={redacted},dc=com
CN=node-1-nifi-dev.{redacted}.com,OU={redacted},O={redacted},L=Kansas
City,ST=Missouri,C=US
CN=node-2-nifi-dev.{redacted}.com,OU={redacted},O={redacted},L
egistry.properties=
I think I have everything set correctly but I have not been able to start an
instances up.
Thanks,
Scott
> On Mar 19, 2018, at 4:35 PM, Bryan Bende <bbe...@gmail.com> wrote:
>
> The base file is here for comparison:
>
> https://github.com/apache/ni
Thanks Mike. That's what I thought. I'd rather not have to customize the
processors for this. What about the new variables feature? Is there a
way to update variables through the API?
On 03/19/2018 04:41 AM, Mike Thomsen wrote:
Scott,
The "input requirement" is
tained invalid XML.
>
> Did you start by modifying the identity-providers.xml file that was
> already there? Can you share the file, or the contents (removing
> anything sensitive)?
>
> On Mon, Mar 19, 2018 at 1:09 PM, Scott Howell <scotthow...@mobilgov.com>
> wrote:
>>
e if/how to do
> this, but I assume there should be some way to configure it. It might require
> settings not exposed in the AWS web console.)
>
> On 3/19/18, 10:51, "Scott Howell" <scotthow...@mobilgov.com> wrote:
>
>Thanks Kevin,
>
>I am j
19, 2018, at 9:35 AM, Kevin Doran <kdo...@apache.org> wrote:
>
> Hey Scott,
>
> Assuming you are using two-way TLS with client certificates for
> authentication, I recommend configuring your ELB for TCP passthrough so that
> the TLS handshake is between the end-client a
t; What error are you getting when you cannot access the UI?
>
> Is there anything interesting in nifi-registry-app.log regarding
> authentication/authorization when this happens?
>
> Can you access the UI securely without going through the ELB?
>
> Thanks,
>
> Bryan
I was able to stand up nifi-registry behind an AWS ELB non-secure. Everything
was working great and was able to access the UI anonymously. I set up the
authorization just like on my nifi instances along with the authorizers and
identity-provider. The service comes up without errors and
can't use
attributes to control them. Is there any way around this, short of
modifying the processor code and creating a new custom processor? Is
there a way to convert the records from the table into variables instead?
Thanks for any help,
Scott
there was a solution using distributed cache or something
similar, but I couldn't find a way to iterate through all keys/values in
the cache without first knowing the names of the keys. Any thought's
about that approach?
Thanks again,
Scott
On 03/12/2018 07:26 AM, Matt Burgess wrote:
Scott,
I'm
fields for me to create the table with?
Thanks,
Scott
One of the really cool features of the registry UI is the ability to deep
link to a user, group, or bucket as well as the ability to deep link to
view ALL of the items the current user is authorized to view, deep link to
view ALL of the items in a specific bucket that the current user is
Does the web app work now as well?
On Wed, Nov 15, 2017 at 11:29 AM, Scott Aslan <scottyas...@gmail.com> wrote:
> ./node/yarn/bin/yarn install?
>
> Sorry, I am not familiar with yarn but what I am trying to get you to do
> is run the yarn that is installed by the fro
, Lou Tian <tian.lou@gmail.com> wrote:
> Hi Scott,
>
> The command in step 2 is not working.
> bash: ./node/yarn: is a directory
>
> I manually add the missing dependencies on package.json then build nifi,
> it works.
> I think this would be a solution?
>
>
successfully or not
On Wed, Nov 15, 2017 at 10:34 AM, Lou Tian <tian.lou@gmail.com> wrote:
> Hi Scott,
>
> I changed it to yarn, the build is success.
> But when I try to run the nifi, the ui is empty (only a background, no
> other components).
> It's m
Hello,
Yarn should only be downloading the client side dependencies at build time
so I think what you are proposing will work just fine. You will need to
update the web-ui pom to install and run yarn instead of npm but the
package.json is already there and should work.
-Scott Aslan
On Wed, Nov
James,
We don't have any InvokeScriptedProcessor processors anywhere on
our NiFi instance. Everything we do is using ExecuteScript (mainly
because I found that grabbing multiple FlowFile's during a single pass
yields so much better performance).
Sorry, and best of luck.
- Scott
are populated correctly and then the SQL script will have the
references to the correct column names in it.
Hope that helps,
Scott
James McMahon <mailto:jsmcmah...@gmail.com>
Friday, May 5, 2017 1:42 PM
Matt, here is what I have tried. No error is thrown, but I can't
reference attribut
top + 1, increment):
newFlowFile = session.clone(flowFile)
newFlowFile = session.putAttribute(newFlowFile, 'current', str(x))
session.transfer(newFlowFile, REL_SUCCESS)
session.remove(flowFile)
I hope these examples are helpful.
- Scott
James McMahon <mailto:j
scripts:
flowFiles = session.get(10)
for flowFile in flowFiles:
if flowFile is None:
continue
# Do stuff here
That seems to improve the throughput of the ExecuteScript processor
dramatically.
YMMV
- Scott
James McMahon <mailto:jsmcmah...@gmail.com>
Wednesday, April 5, 2017
of FlowFiles, and that is working for me now.
Thanks for your time and suggestion!
- Scott
Bryan Bende <mailto:bbe...@gmail.com>
Tuesday, February 28, 2017 8:53 AM
Scott,
Do you have a constant flow of data from your database, or is this
more like a large batch comes in and processes i
Matt,
Thanks for the suggestion. I did end up creating a ExecuteScript
processor that expanded out the range of the numbers into the content,
and it is working for this scenario.
Thanks again!
- Scott
Matt Foley <mailto:ma...@apache.org>
Monday, February 27, 2017 5:21 PM
t;Check if done" go
through a funnel, and set an Oldest FlowFile prioritizer, but it still
eventually ends up filling up the entire triangle of queues.
Does anyone have a suggestion as to how I could gracefully handle a
situation like this? I appreciate any advice.
Thanks!
- S
as one of the 'layers' even when they are zoomed in...
-Scott
On Wed, Sep 28, 2016 at 9:41 AM, Russell Bateman <
russell.bate...@perfectsearchcorp.com> wrote:
> After thinking on it a bit, I agree that Manish' suggestion could be a
> good idea as an option (the way *additional
8 PM, Scott Stroud <scottstr...@gmail.com> wrote:
> Is this only an issue for digest authentication?
>
> On Tue, Jun 28, 2016 at 11:05 AM, Scott Stroud <scottstr...@gmail.com>
> wrote:
>
>> If your sense is its triggering this based on http headers (or body)
>> c
Is this only an issue for digest authentication?
On Tue, Jun 28, 2016 at 11:05 AM, Scott Stroud <scottstr...@gmail.com>
wrote:
> If your sense is its triggering this based on http headers (or body)
> characteristics, I can play with those since its my own service thats being
>
e Android support, including
> logging. It's almost as if it's detected the wrong platform.
>
> Not an answer but the Android reference is probably from there.
>
> > On Jun 28, 2016, at 7:37 AM, Joe Witt <joe.w...@gmail.com> wrote:
> >
> > Scott
> >
&
]
o.a.n.c.t.ContinuallyRunProcessorTask
java.lang.NoClassDefFoundError: android/util/Log
at
com.burgstaller.okhttp.AuthenticationCacheInterceptor.intercept(AuthenticationCacheInterceptor.java:34)
On Tue, Jun 28, 2016 at 10:42 AM, Scott Stroud <scottstr...@gmail.com>
wrote:
> I do not see the full stack in the
n 'java -version' can you please share the output.
>
> Thanks
> Joe
>
> On Tue, Jun 28, 2016 at 7:23 AM, Scott Stroud <scottstr...@gmail.com>
> wrote:
> > Im attempting to use the InvokeHTTP component with the 0.6.1 release and
> > keep seeing this error. My J
Im attempting to use the InvokeHTTP component with the 0.6.1 release and
keep seeing this error. My JAVA_HOME is 1.8.0_72-b15 on my Mac. Im sure I
could go find the jar and manually add it to the classpath, but I assume it
should work OOB straight from the NiFi download or maybe its some path
87 matches
Mail list logo
