date:20160509

RE: Reg: Nifi Clustering

2016-05-09 Thread Sourav Gulati

Thanks Mark for the information

Regards,
Sourav Gulati

-Original Message-
From: Mark Payne [mailto:marka...@hotmail.com]
Sent: Monday, May 09, 2016 5:33 PM
To: dev@nifi.apache.org
Subject: Re: Reg: Nifi Clustering

Sourav,

Absolutely, Site-to-Site is a key enabler of NiFi and certainly will continue 
to be supported.
Whereas right now you enter the URL of the NCM, you would simply enter the URL 
of any of the nodes in the cluster. Your instance would then reach out to that 
node and get a list of other nodes in the cluster.

This will be similar to how the user will access the application, as well. In 
1.0, since there will be no NCM to connect to, you will be able to connect to 
any node in the cluster, and it should be able to perform the necessary 
functions.

Thanks
-Mark


> On May 9, 2016, at 5:15 AM, Sourav Gulati  wrote:
>
> Hello Team,
>
> We are doing some poc with Nifi . IF successful, we may be using it in 
> production some time in future. We are using site-to-site in our poc.  As 
> Joe, in the upcoming 1.0 release work is being done to eliminate this notion 
> of master/slave altogether.
>
> Will site-to-site be still supported? Reason I am asking this is currently 
> while configuring site-to-site we need to provide URL of NCM in Remote 
> process Group.
>
>
> Regards,
> Sourav Gulati
>
> -Original Message-
> From: Matthew Clarke [mailto:matt.clarke@gmail.com]
> Sent: Tuesday, May 03, 2016 4:41 PM
> To: dev@nifi.apache.org
> Subject: RE: Reg: Nifi Clustering
>
> Sourav,
>   A single instance of NiFi cannot be both a NCM and a Node at the same time. 
> In order to have both a NCM and a Node on a single server, you will need to 
> install two copies of NiFi. One will be configured to be the NCM while the 
> other is configured to be the Node. As Joe pointed out, these two instance of 
> NiFi cannot share any listening ports (HTTP or S2S ports) in their configs.
>
> Matt
> On May 3, 2016 2:03 AM, "Sourav Gulati"  wrote:
>
> Hi Joe,
>
> This is what I am getting while running both master and node
>
> IllegalStateException: Application may be configured as a cluster
> manager or a node, but not both
>
>
>
> Regards,
> Sourav Gulati
>
> -Original Message-
> From: Joe Witt [mailto:joe.w...@gmail.com]
> Sent: Monday, May 02, 2016 6:45 PM
> To: dev@nifi.apache.org
> Subject: Re: Reg: Nifi Clustering
>
> Sourav,
>
> The installation procedure does not differ.  However, you do need to be 
> careful to ensure you're not trying to use the same ports for both the master 
> and the node.
>
> If you're having trouble go ahead and send some of the configuration details 
> over and someone can try to assist.
>
> Thanks
> Joe
>
> On Mon, May 2, 2016 at 9:05 AM, Sourav Gulati
> 
> wrote:
>> Hello Joe,
>>
>> Could you please provide steps of creating cluster on single node in
> current version?
>>
>> Regards,
>> Sourav Gulati
>>
>> -Original Message-
>> From: Joe Witt [mailto:joe.w...@gmail.com]
>> Sent: Monday, May 02, 2016 6:28 PM
>> To: dev@nifi.apache.org
>> Subject: Re: Reg: Nifi Clustering
>>
>> Sourav,
>>
>> Yes.  And in the upcoming 1.0 release work is being done to eliminate
> this notion of master/slave altogether and move to instead that at any time 
> any node can carry the responsibility of being a coordinator for the cluster.
>>
>> Thanks
>> Joe
>>
>> On Mon, May 2, 2016 at 8:49 AM, Sourav Gulati
>> 
> wrote:
>>> Hi Team,
>>>
>>> While creating Nifi Cluster, Can I run master and slave on same node?
>>>
>>> Regards,
>>> Sourav Gulati
>>>
>>>
>>> 
>>>
>>>
>>>
>>>
>>>
>>>
>>> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is 
> intended solely for the named addressee. If received in error, please destroy 
> and notify the sender. Any use of this email is prohibited when received in 
> error. Impetus does not represent, warrant and/or guarantee, that the 
> integrity of this communication has been maintained nor that the 
> communication is free of errors, virus, interception or interference.
>>
>> 
>>
>>
>>
>>
>>
>>
>> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is 
> intended solely for the named addressee. If received in error, please destroy 
> and notify the sender. Any use of this email is prohibited when received in 
> error. Impetus does not represent, warrant and/or guarantee, that the 
> integrity of this communication has been maintained nor that the 
> communication is free of errors, virus, interception or interference.
>
> 
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential, proprietary, 
> privileged or otherwise protected by law. The

[GitHub] nifi pull request: [NIFI-1707] upgradeable angular components

2016-05-09 Thread scottyaslan

Github user scottyaslan closed the pull request at:

https://github.com/apache/nifi/pull/425


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: [NIFI-1707] upgradeable angular components

2016-05-09 Thread mcgilman

Github user mcgilman commented on the pull request:

https://github.com/apache/nifi/pull/425#issuecomment-217962785
  
Merged to master. Thanks @scottyaslan 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1858 Adding SiteToSiteProvenanceReportingT...

2016-05-09 Thread bbende

Github user bbende commented on a diff in the pull request:

https://github.com/apache/nifi/pull/419#discussion_r62548156
  
--- Diff: 
nifi-nar-bundles/nifi-site-to-site-reporting-bundle/nifi-site-to-site-reporting-task/src/main/java/org/apache/nifi/reporting/SiteToSiteProvenanceReportingTask.java
 ---
@@ -0,0 +1,354 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.reporting;
+
+import org.apache.nifi.annotation.behavior.Stateful;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.state.Scope;
+import org.apache.nifi.components.state.StateManager;
+import org.apache.nifi.controller.status.PortStatus;
+import org.apache.nifi.controller.status.ProcessGroupStatus;
+import org.apache.nifi.controller.status.ProcessorStatus;
+import org.apache.nifi.controller.status.RemoteProcessGroupStatus;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.provenance.ProvenanceEventRecord;
+import org.apache.nifi.remote.Transaction;
+import org.apache.nifi.remote.TransferDirection;
+
+import javax.json.Json;
+import javax.json.JsonArray;
+import javax.json.JsonArrayBuilder;
+import javax.json.JsonBuilderFactory;
+import javax.json.JsonObject;
+import javax.json.JsonObjectBuilder;
+import java.io.IOException;
+import java.net.MalformedURLException;
+import java.net.URL;
+import java.nio.charset.StandardCharsets;
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.TimeZone;
+import java.util.UUID;
+import java.util.concurrent.TimeUnit;
+
+@Tags({"provenance", "lineage", "tracking", "site", "site to site"})
+@CapabilityDescription("Publishes Provenance events using the Site To Site 
protocol.")
+@Stateful(scopes = Scope.LOCAL, description = "Stores the Reporting Task's 
last event Id so that on restart the task knows where it left off.")
+public class SiteToSiteProvenanceReportingTask extends 
AbstractSiteToSiteReportingTask {
+
+private static final String TIMESTAMP_FORMAT = 
"-MM-dd'T'HH:mm:ss.SSS'Z'";
+private static final String LAST_EVENT_ID_KEY = "last_event_id";
+
+static final PropertyDescriptor PLATFORM = new 
PropertyDescriptor.Builder()
+.name("Platform")
+.description("The value to use for the platform field in each 
provenance event.")
+.required(true)
+.expressionLanguageSupported(true)
+.defaultValue("nifi")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+private volatile long firstEventId = -1L;
+
+@Override
+protected List getSupportedPropertyDescriptors() {
+final List properties = new 
ArrayList<>(super.getSupportedPropertyDescriptors());
+properties.add(PLATFORM);
+return properties;
+}
+
+private String getComponentName(final ProcessGroupStatus status, final 
ProvenanceEventRecord event) {
+if (status == null) {
+return null;
+}
+
+final String componentId = event.getComponentId();
+if (status.getId().equals(componentId)) {
+return status.getName();
+}
+
+for (final ProcessorStatus procStatus : 
status.getProcessorStatus()) {
+if (procStatus.getId().equals(componentId)) {
+return procStatus.getName();
+}
+}
+
+for (final PortStatus portStatus :

Re: [DISCUSS] Removal of FlowFilePrioritizer as first-class extension point

2016-05-09 Thread Michael Moser

I do think there is (a little) merit to a separate sort by file size, but
only because adding UpdateAttribute feels like a work around.  Having it as
a native prioritizer should give people the impression that we thought
about it and consider it a valid use case.

-- Mike


On Mon, May 9, 2016 at 1:46 PM, Aldrin Piri  wrote:

> +1 for this.  It also occurred to me that these are currently not shown in
> the generated docs and only in the user guide.
>
> Mike,
>
> As far as sorting by size, do you think there is merit beyond the
> PriorityAttributePrioritizer for this case?  An update attribute "copying"
> fileSize to "priority" should accomplish the same effect (optionally
> negated depending on desired sort order).
>
> On Mon, May 9, 2016 at 1:37 PM, Michael Moser  wrote:
>
> > +1 as long as the existing 4 prioritizers remain as options.  I have seen
> > people use all of them.  I have also seen someone hack together what was
> > effectively a SmallestFileFirstPrioritizer and a
> > LargestFileFirstPrioritizer by using RouteOnAttribute on different
> > ${fileSize} values.  The use case was "I receive a batch of files and I
> > don't want the 1 excessively large file to delay the multitude of other
> > small files from moving on first".  Perhaps we can support that use case
> > too.
> >
> > -- Mike
> >
> >
> > On Fri, May 6, 2016 at 1:11 PM, Andy LoPresto 
> > wrote:
> >
> > > +1. I think the benefits of this move far outweigh the potential but
> > > unrealized value of extensible prioritizers.
> > >
> > > Andy LoPresto
> > > alopre...@apache.org
> > > *alopresto.apa...@gmail.com *
> > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> > >
> > > On May 6, 2016, at 9:49 AM, Brandon DeVries  wrote:
> > >
> > > +1.  This seems like something we should provide options for (as we
> do),
> > > but doesn't really need to be made / kept accessible for extension.
> > >
> > > Brandon
> > >
> > > On Fri, May 6, 2016 at 11:45 AM Mark Payne 
> wrote:
> > >
> > > I'm definitely a +1. In my experience, the way that most people think
> > > about prioritizing data is
> > > to either assign an absolute priority to a FlowFile and use the
> > > PriorityAttributePrioritizer or to
> > > use the FirstInFirstOut Prioritizer. Any number of processors can be
> used
> > > to extract the the
> > > 'priority' attribute and prioritize the data that way. I think this
> makes
> > > the extensibility less valuable,
> > > since the flow itself can be used to determine a 'priority' attribute
> > > based on FlowFile content, attributes,
> > > etc.
> > >
> > > On May 6, 2016, at 11:16 AM, Joe Witt  wrote:
> > >
> > > Team,
> > >
> > > I'd like to propose we remove the FlowFilePrioritizer [1] from the set
> > > of first class extension points we support.
> > >
> > > The background:
> > >
> > > FlowFilePrioritizer implementations are used to compare flow files as
> > > they are enqueued on a given connection in the flow.  This in turn
> > > means when flow files are pulled from the queue they are pulled in a
> > > manner that allows the most important data first to be operated on.
> > > This is a valuable feature and is heavily utilized.  Out of the box
> > > NiFi provides several obvious prioritizer implementations such as
> > > first in and out based on age of the flow file, first in based on
> > > entry order, and honoring a numeric representation of priority set as
> > > a specific attribute [2].  They are rarely changed and have so far not
> > > grown in numbers nor have there been any discussions of doing so.  If
> > > I think back to their usage over the past decade I actually think
> > > there have been only a few ever made.
> > >
> > > The concept and ability to sort queues is important and powerful and
> > > needs to be kept.  But making them a first-class extension point I am
> > > now questioning the value of.  The reason being is that as defined the
> > > interface is intuitive for the developer but much harder for the
> > > framework side.  That combined with their lack of ever being extended
> > > opens the debate.
> > >
> > > When the prioritizers were first envisioned we didn't support the
> > > concept of swapping out flowfiles to disk when the queues were huge.
> > > We now do.  But we cannot sort (at this time) the swapped out items.
> > > By getting rid of this extension point as it is now we can instead
> > > support these types of prioritizers in a different and more optimized
> > > manner albeit in a less extension friendly way (more coupled to the
> > > framework).  Rather than simply using comparators we can do absolute
> > > priority assignment and when swapping out flow files we can track the
> > > largest/smallest priority and thus enable prioritized swap-in.  This
> > > would also be helpful for doing things like auto-cluster load
> > > balancing or

Re: [DISCUSS] Removal of FlowFilePrioritizer as first-class extension point

2016-05-09 Thread Aldrin Piri

+1 for this.  It also occurred to me that these are currently not shown in
the generated docs and only in the user guide.

Mike,

As far as sorting by size, do you think there is merit beyond the
PriorityAttributePrioritizer for this case?  An update attribute "copying"
fileSize to "priority" should accomplish the same effect (optionally
negated depending on desired sort order).

On Mon, May 9, 2016 at 1:37 PM, Michael Moser  wrote:

> +1 as long as the existing 4 prioritizers remain as options.  I have seen
> people use all of them.  I have also seen someone hack together what was
> effectively a SmallestFileFirstPrioritizer and a
> LargestFileFirstPrioritizer by using RouteOnAttribute on different
> ${fileSize} values.  The use case was "I receive a batch of files and I
> don't want the 1 excessively large file to delay the multitude of other
> small files from moving on first".  Perhaps we can support that use case
> too.
>
> -- Mike
>
>
> On Fri, May 6, 2016 at 1:11 PM, Andy LoPresto 
> wrote:
>
> > +1. I think the benefits of this move far outweigh the potential but
> > unrealized value of extensible prioritizers.
> >
> > Andy LoPresto
> > alopre...@apache.org
> > *alopresto.apa...@gmail.com *
> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> >
> > On May 6, 2016, at 9:49 AM, Brandon DeVries  wrote:
> >
> > +1.  This seems like something we should provide options for (as we do),
> > but doesn't really need to be made / kept accessible for extension.
> >
> > Brandon
> >
> > On Fri, May 6, 2016 at 11:45 AM Mark Payne  wrote:
> >
> > I'm definitely a +1. In my experience, the way that most people think
> > about prioritizing data is
> > to either assign an absolute priority to a FlowFile and use the
> > PriorityAttributePrioritizer or to
> > use the FirstInFirstOut Prioritizer. Any number of processors can be used
> > to extract the the
> > 'priority' attribute and prioritize the data that way. I think this makes
> > the extensibility less valuable,
> > since the flow itself can be used to determine a 'priority' attribute
> > based on FlowFile content, attributes,
> > etc.
> >
> > On May 6, 2016, at 11:16 AM, Joe Witt  wrote:
> >
> > Team,
> >
> > I'd like to propose we remove the FlowFilePrioritizer [1] from the set
> > of first class extension points we support.
> >
> > The background:
> >
> > FlowFilePrioritizer implementations are used to compare flow files as
> > they are enqueued on a given connection in the flow.  This in turn
> > means when flow files are pulled from the queue they are pulled in a
> > manner that allows the most important data first to be operated on.
> > This is a valuable feature and is heavily utilized.  Out of the box
> > NiFi provides several obvious prioritizer implementations such as
> > first in and out based on age of the flow file, first in based on
> > entry order, and honoring a numeric representation of priority set as
> > a specific attribute [2].  They are rarely changed and have so far not
> > grown in numbers nor have there been any discussions of doing so.  If
> > I think back to their usage over the past decade I actually think
> > there have been only a few ever made.
> >
> > The concept and ability to sort queues is important and powerful and
> > needs to be kept.  But making them a first-class extension point I am
> > now questioning the value of.  The reason being is that as defined the
> > interface is intuitive for the developer but much harder for the
> > framework side.  That combined with their lack of ever being extended
> > opens the debate.
> >
> > When the prioritizers were first envisioned we didn't support the
> > concept of swapping out flowfiles to disk when the queues were huge.
> > We now do.  But we cannot sort (at this time) the swapped out items.
> > By getting rid of this extension point as it is now we can instead
> > support these types of prioritizers in a different and more optimized
> > manner albeit in a less extension friendly way (more coupled to the
> > framework).  Rather than simply using comparators we can do absolute
> > priority assignment and when swapping out flow files we can track the
> > largest/smallest priority and thus enable prioritized swap-in.  This
> > would also be helpful for doing things like auto-cluster load
> > balancing or cluster-wide prioritized site-to-site.
> >
> > So, in short, the interface would go from being a comparator to
> > instead providing a method which returns an absolute priority.  For
> > example, it would have a method called 'getPriority' which takes in a
> > flow file and returns a long.
> >
> > This approach would also still allow chaining prioritizers as we do
> >
> > today.
> >
> >
> > We still can support this as something which can be extended for those
> > who wish to do so just in a less friendly and more framework coupled
> >

Re: [DISCUSS] Removal of FlowFilePrioritizer as first-class extension point

2016-05-09 Thread Michael Moser

+1 as long as the existing 4 prioritizers remain as options.  I have seen
people use all of them.  I have also seen someone hack together what was
effectively a SmallestFileFirstPrioritizer and a
LargestFileFirstPrioritizer by using RouteOnAttribute on different
${fileSize} values.  The use case was "I receive a batch of files and I
don't want the 1 excessively large file to delay the multitude of other
small files from moving on first".  Perhaps we can support that use case
too.

-- Mike


On Fri, May 6, 2016 at 1:11 PM, Andy LoPresto  wrote:

> +1. I think the benefits of this move far outweigh the potential but
> unrealized value of extensible prioritizers.
>
> Andy LoPresto
> alopre...@apache.org
> *alopresto.apa...@gmail.com *
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On May 6, 2016, at 9:49 AM, Brandon DeVries  wrote:
>
> +1.  This seems like something we should provide options for (as we do),
> but doesn't really need to be made / kept accessible for extension.
>
> Brandon
>
> On Fri, May 6, 2016 at 11:45 AM Mark Payne  wrote:
>
> I'm definitely a +1. In my experience, the way that most people think
> about prioritizing data is
> to either assign an absolute priority to a FlowFile and use the
> PriorityAttributePrioritizer or to
> use the FirstInFirstOut Prioritizer. Any number of processors can be used
> to extract the the
> 'priority' attribute and prioritize the data that way. I think this makes
> the extensibility less valuable,
> since the flow itself can be used to determine a 'priority' attribute
> based on FlowFile content, attributes,
> etc.
>
> On May 6, 2016, at 11:16 AM, Joe Witt  wrote:
>
> Team,
>
> I'd like to propose we remove the FlowFilePrioritizer [1] from the set
> of first class extension points we support.
>
> The background:
>
> FlowFilePrioritizer implementations are used to compare flow files as
> they are enqueued on a given connection in the flow.  This in turn
> means when flow files are pulled from the queue they are pulled in a
> manner that allows the most important data first to be operated on.
> This is a valuable feature and is heavily utilized.  Out of the box
> NiFi provides several obvious prioritizer implementations such as
> first in and out based on age of the flow file, first in based on
> entry order, and honoring a numeric representation of priority set as
> a specific attribute [2].  They are rarely changed and have so far not
> grown in numbers nor have there been any discussions of doing so.  If
> I think back to their usage over the past decade I actually think
> there have been only a few ever made.
>
> The concept and ability to sort queues is important and powerful and
> needs to be kept.  But making them a first-class extension point I am
> now questioning the value of.  The reason being is that as defined the
> interface is intuitive for the developer but much harder for the
> framework side.  That combined with their lack of ever being extended
> opens the debate.
>
> When the prioritizers were first envisioned we didn't support the
> concept of swapping out flowfiles to disk when the queues were huge.
> We now do.  But we cannot sort (at this time) the swapped out items.
> By getting rid of this extension point as it is now we can instead
> support these types of prioritizers in a different and more optimized
> manner albeit in a less extension friendly way (more coupled to the
> framework).  Rather than simply using comparators we can do absolute
> priority assignment and when swapping out flow files we can track the
> largest/smallest priority and thus enable prioritized swap-in.  This
> would also be helpful for doing things like auto-cluster load
> balancing or cluster-wide prioritized site-to-site.
>
> So, in short, the interface would go from being a comparator to
> instead providing a method which returns an absolute priority.  For
> example, it would have a method called 'getPriority' which takes in a
> flow file and returns a long.
>
> This approach would also still allow chaining prioritizers as we do
>
> today.
>
>
> We still can support this as something which can be extended for those
> who wish to do so just in a less friendly and more framework coupled
> manner.  Basically, this would just be more like we support content
> repository or provenance repository extension where the developer
> needs to both understand the implementation they want but also the
> mechanics of getting that into the build and the deeper implications.
>
> Would like to hear if others are supportive of this or if they see any
> major problems posed by this.  Given we're working towards the 1.x
> release this is a good time to pull this cord.  If we do this we can
> document the steps and thinking needed to build/contribute new
> prioritizer schemes.
>
> Thanks
> Joe
>
> [1]
>
>
>

Re: deploying the web layer

2016-05-09 Thread Andy LoPresto

Karthik,

Thanks for your message. Please also consider raising a issue for this bug [1] 
and contributing your fix back to the community [2] if you are so inclined.

[1] https://issues.apache.org/jira/browse/NIFI 

[2] https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide 


Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On May 9, 2016, at 9:23 AM, Matt Gilman  wrote:
> 
> Karthik,
> 
> The WARs are unpacked into the working directory which defaults to
> /work/jetty. You should be able to replace the JS file that's
> there. Alternatively, you could just build the nifi-web-ui and the
> nifi-framework-nar and then copy the resulting framework NAR file into
> /lib.
> 
> Matt
> 
> On Mon, May 9, 2016 at 12:10 PM, karthik Narayanan 
> wrote:
> 
>> Hi,
>> 
>> I was successfully able to build the NIFI project from source . I was
>> working an issue that i see on the NIFI web ui. When i right click on a
>> process that is on the edge of my screen, the popup dialog, goes outside
>> the screen, which makes the menus unselectable. I have fixed the js code so
>> it looks at the boudaries and realigns the pop up so it completely shows
>> inside the browser window.
>> 
>> The problem i am having is in deploying this to my nifi binary. I do not
>> want to build the whole project, just because i change a js page. I tried
>> copying the war to several locations but my changes dont take affect. Can
>> someone guide me on how i can deploy just the WEB-UI so that the jetty
>> server deploys my latest code.
>> 
>> Karthik
>> 



signature.asc
Description: Message signed with OpenPGP using GPGMail

[GitHub] nifi pull request: NIFI-1862 User Guide corrections/improvements

2016-05-09 Thread alopresto

Github user alopresto commented on the pull request:

https://github.com/apache/nifi/pull/427#issuecomment-217917476
  
Reviewing. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1862 User Guide corrections/improvements

2016-05-09 Thread andrewmlim

GitHub user andrewmlim opened a pull request:

https://github.com/apache/nifi/pull/427

NIFI-1862 User Guide corrections/improvements

Made multiple edits to the User Guide documentation for correcting errors 
(spelling/grammatical) and improving readability.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/andrewmlim/nifi patch-3

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/427.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #427


commit 9eb4bdc5385b56b0d8487357f5935b9e8c839c90
Author: Andrew Lim 
Date:   2016-05-09T15:47:50Z

NIFI-1862 User Guide corrections/improvements

Made multiple edits to the User Guide documentation for correcting errors 
(spelling/grammatical) and improving readability.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1862 Corrections to User Guide

2016-05-09 Thread andrewmlim

Github user andrewmlim closed the pull request at:

https://github.com/apache/nifi/pull/426


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1862 Corrections to User Guide

2016-05-09 Thread andrewmlim

GitHub user andrewmlim opened a pull request:

https://github.com/apache/nifi/pull/426

NIFI-1862 Corrections to User Guide



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/andrewmlim/nifi patch-3

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/426.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #426


commit 5748912640482e57ac818e2361e73983df64d496
Author: Andrew Lim 
Date:   2016-05-09T14:43:28Z

Corrections to User Guide




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1858 Adding SiteToSiteProvenanceReportingT...

2016-05-09 Thread bbende

Github user bbende commented on a diff in the pull request:

https://github.com/apache/nifi/pull/419#discussion_r62505576
  
--- Diff: 
nifi-nar-bundles/nifi-site-to-site-reporting-bundle/nifi-site-to-site-reporting-task/src/main/java/org/apache/nifi/reporting/AbstractSiteToSiteReportingTask.java
 ---
@@ -0,0 +1,168 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.reporting;
+
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.events.EventReporter;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.remote.client.SiteToSiteClient;
+import org.apache.nifi.ssl.SSLContextService;
+
+import javax.net.ssl.SSLContext;
+import java.io.IOException;
+import java.net.URL;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * Base class for ReportingTasks that send data over site-to-site.
+ */
+public abstract class AbstractSiteToSiteReportingTask extends 
AbstractReportingTask {
+
+static final PropertyDescriptor DESTINATION_URL = new 
PropertyDescriptor.Builder()
+.name("Destination URL")
+.description("The URL to send the Provenance Events to. For 
example, to send to a NiFi instance running " +
+"at http://localhost:8080/nifi this value should be 
http://localhost:8080;)
+.required(true)
--- End diff --

@jvwing @mosermw I agree, I will push a commit that aligns it with the way 
RPGs take the url, thanks for reviewing


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1858 Adding SiteToSiteProvenanceReportingT...

2016-05-09 Thread mosermw

Github user mosermw commented on a diff in the pull request:

https://github.com/apache/nifi/pull/419#discussion_r62505265
  
--- Diff: 
nifi-nar-bundles/nifi-site-to-site-reporting-bundle/nifi-site-to-site-reporting-task/src/main/java/org/apache/nifi/reporting/AbstractSiteToSiteReportingTask.java
 ---
@@ -0,0 +1,168 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.reporting;
+
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.events.EventReporter;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.remote.client.SiteToSiteClient;
+import org.apache.nifi.ssl.SSLContextService;
+
+import javax.net.ssl.SSLContext;
+import java.io.IOException;
+import java.net.URL;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * Base class for ReportingTasks that send data over site-to-site.
+ */
+public abstract class AbstractSiteToSiteReportingTask extends 
AbstractReportingTask {
+
+static final PropertyDescriptor DESTINATION_URL = new 
PropertyDescriptor.Builder()
+.name("Destination URL")
+.description("The URL to send the Provenance Events to. For 
example, to send to a NiFi instance running " +
+"at http://localhost:8080/nifi this value should be 
http://localhost:8080;)
+.required(true)
--- End diff --

I agree, it would be nice to avoid hard coding the /nifi context path.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: [NIFI-1707] upgradeable angular components

2016-05-09 Thread scottyaslan

GitHub user scottyaslan opened a pull request:

https://github.com/apache/nifi/pull/425

[NIFI-1707] upgradeable angular components



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/scottyaslan/nifi responsiveDevBranch

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/425.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #425


commit 7cfc2016ac38894a4755f43f6ead754ec16a21f5
Author: Scott Aslan 
Date:   2016-05-08T05:34:54Z

[NIFI-1707] upgradeable angular components




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1850 - Initial Commit for JSON-to-JSON Sch...

2016-05-09 Thread YolandaMDavis

GitHub user YolandaMDavis opened a pull request:

https://github.com/apache/nifi/pull/424

NIFI-1850 - Initial Commit for JSON-to-JSON Schema Converter Editor

This is an initial commit for review of the TransformJson Advanced Editor. 
Please see https://issues.apache.org/jira/browse/NIFI-1850 for details.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/YolandaMDavis/nifi NIFI-1850-0.x

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/424.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #424


commit 9d0b2efdcddb9e629e64067a5266df44ee8eb06c
Author: Yolanda M. Davis 
Date:   2016-05-09T12:20:10Z

NIFI-1850 - Initial Commit for JSON-to-JSON Schema Converter Editor
(cherry picked from commit 2f7a95a)




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1860 Added ContainerRequestFilter to redir...

2016-05-09 Thread mcgilman

Github user mcgilman commented on the pull request:

https://github.com/apache/nifi/pull/422#issuecomment-217849959
  
Thanks @ijokarumawak !

This has been pushed to master.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1860 Added ContainerRequestFilter to redir...

2016-05-09 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/422


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Reg: Get files from ftp

2016-05-09 Thread Mark Payne

Sourav,

We have begun transitioning from many of the Get*** Processors to List*** and 
Fetch*** Processors.
There is a ListSFTP / FetchSFTP processor set but not currently a List/Fetch 
FTP. Is SFTP a possibility
for you? Would you be interested in working on a List/Fetch FTP Processor set?

Thanks
-Mark

> On May 9, 2016, at 5:48 AM, Sourav Gulati  wrote:
> 
> Hi Team,
> 
> I need a suggestion.
> 
> I want to get files from ftp server for which GetFtp processor is available. 
> However, as I cannot delete files from source, I need to put a check that 
> this processor does not pick a file more than once. What is the best way to 
> do that?
> 
> Regards,
> Sourav Gulati
> 
> 
> 
> 
> 
> 
> 
> 
> 
> NOTE: This message may contain information that is confidential, proprietary, 
> privileged or otherwise protected by law. The message is intended solely for 
> the named addressee. If received in error, please destroy and notify the 
> sender. Any use of this email is prohibited when received in error. Impetus 
> does not represent, warrant and/or guarantee, that the integrity of this 
> communication has been maintained nor that the communication is free of 
> errors, virus, interception or interference.

Re: Reg: Nifi Clustering

2016-05-09 Thread Mark Payne

Sourav,

Absolutely, Site-to-Site is a key enabler of NiFi and certainly will continue 
to be supported.
Whereas right now you enter the URL of the NCM, you would simply enter the URL 
of any
of the nodes in the cluster. Your instance would then reach out to that node 
and get a list
of other nodes in the cluster.

This will be similar to how the user will access the application, as well. In 
1.0, since there
will be no NCM to connect to, you will be able to connect to any node in the 
cluster, and it
should be able to perform the necessary functions.

Thanks
-Mark


> On May 9, 2016, at 5:15 AM, Sourav Gulati  wrote:
> 
> Hello Team,
> 
> We are doing some poc with Nifi . IF successful, we may be using it in 
> production some time in future. We are using site-to-site in our poc.  As 
> Joe, in the upcoming 1.0 release work is being done to eliminate this notion 
> of master/slave altogether.
> 
> Will site-to-site be still supported? Reason I am asking this is currently 
> while configuring site-to-site we need to provide URL of NCM in Remote 
> process Group.
> 
> 
> Regards,
> Sourav Gulati
> 
> -Original Message-
> From: Matthew Clarke [mailto:matt.clarke@gmail.com]
> Sent: Tuesday, May 03, 2016 4:41 PM
> To: dev@nifi.apache.org
> Subject: RE: Reg: Nifi Clustering
> 
> Sourav,
>   A single instance of NiFi cannot be both a NCM and a Node at the same time. 
> In order to have both a NCM and a Node on a single server, you will need to 
> install two copies of NiFi. One will be configured to be the NCM while the 
> other is configured to be the Node. As Joe pointed out, these two instance of 
> NiFi cannot share any listening ports (HTTP or S2S ports) in their configs.
> 
> Matt
> On May 3, 2016 2:03 AM, "Sourav Gulati"  wrote:
> 
> Hi Joe,
> 
> This is what I am getting while running both master and node
> 
> IllegalStateException: Application may be configured as a cluster manager or 
> a node, but not both
> 
> 
> 
> Regards,
> Sourav Gulati
> 
> -Original Message-
> From: Joe Witt [mailto:joe.w...@gmail.com]
> Sent: Monday, May 02, 2016 6:45 PM
> To: dev@nifi.apache.org
> Subject: Re: Reg: Nifi Clustering
> 
> Sourav,
> 
> The installation procedure does not differ.  However, you do need to be 
> careful to ensure you're not trying to use the same ports for both the master 
> and the node.
> 
> If you're having trouble go ahead and send some of the configuration details 
> over and someone can try to assist.
> 
> Thanks
> Joe
> 
> On Mon, May 2, 2016 at 9:05 AM, Sourav Gulati 
> wrote:
>> Hello Joe,
>> 
>> Could you please provide steps of creating cluster on single node in
> current version?
>> 
>> Regards,
>> Sourav Gulati
>> 
>> -Original Message-
>> From: Joe Witt [mailto:joe.w...@gmail.com]
>> Sent: Monday, May 02, 2016 6:28 PM
>> To: dev@nifi.apache.org
>> Subject: Re: Reg: Nifi Clustering
>> 
>> Sourav,
>> 
>> Yes.  And in the upcoming 1.0 release work is being done to eliminate
> this notion of master/slave altogether and move to instead that at any time 
> any node can carry the responsibility of being a coordinator for the cluster.
>> 
>> Thanks
>> Joe
>> 
>> On Mon, May 2, 2016 at 8:49 AM, Sourav Gulati
>> 
> wrote:
>>> Hi Team,
>>> 
>>> While creating Nifi Cluster, Can I run master and slave on same node?
>>> 
>>> Regards,
>>> Sourav Gulati
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is 
> intended solely for the named addressee. If received in error, please destroy 
> and notify the sender. Any use of this email is prohibited when received in 
> error. Impetus does not represent, warrant and/or guarantee, that the 
> integrity of this communication has been maintained nor that the 
> communication is free of errors, virus, interception or interference.
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is 
> intended solely for the named addressee. If received in error, please destroy 
> and notify the sender. Any use of this email is prohibited when received in 
> error. Impetus does not represent, warrant and/or guarantee, that the 
> integrity of this communication has been maintained nor that the 
> communication is free of errors, virus, interception or interference.
> 
> 
> 
> 
> 
> 
> 
> 
> NOTE: This message may contain information that is confidential, proprietary, 
> privileged or otherwise protected by law. The message is intended solely for 
> the named addressee. If received in error, please destroy and notify the 
> sender. Any use of this email is prohibited when received in error.

Reg: Get files from ftp

2016-05-09 Thread Sourav Gulati

Hi Team,

I need a suggestion.

I want to get files from ftp server for which GetFtp processor is available. 
However, as I cannot delete files from source, I need to put a check that this 
processor does not pick a file more than once. What is the best way to do that?

Regards,
Sourav Gulati









NOTE: This message may contain information that is confidential, proprietary, 
privileged or otherwise protected by law. The message is intended solely for 
the named addressee. If received in error, please destroy and notify the 
sender. Any use of this email is prohibited when received in error. Impetus 
does not represent, warrant and/or guarantee, that the integrity of this 
communication has been maintained nor that the communication is free of errors, 
virus, interception or interference.

RE: Reg: Nifi Clustering

[GitHub] nifi pull request: [NIFI-1707] upgradeable angular components

[GitHub] nifi pull request: [NIFI-1707] upgradeable angular components

[GitHub] nifi pull request: NIFI-1858 Adding SiteToSiteProvenanceReportingT...

Re: [DISCUSS] Removal of FlowFilePrioritizer as first-class extension point

Re: [DISCUSS] Removal of FlowFilePrioritizer as first-class extension point

Re: [DISCUSS] Removal of FlowFilePrioritizer as first-class extension point

Re: deploying the web layer

[GitHub] nifi pull request: NIFI-1862 User Guide corrections/improvements

[GitHub] nifi pull request: NIFI-1862 User Guide corrections/improvements

[GitHub] nifi pull request: NIFI-1862 Corrections to User Guide

[GitHub] nifi pull request: NIFI-1862 Corrections to User Guide

[GitHub] nifi pull request: NIFI-1858 Adding SiteToSiteProvenanceReportingT...

[GitHub] nifi pull request: NIFI-1858 Adding SiteToSiteProvenanceReportingT...

[GitHub] nifi pull request: [NIFI-1707] upgradeable angular components

[GitHub] nifi pull request: NIFI-1850 - Initial Commit for JSON-to-JSON Sch...

[GitHub] nifi pull request: NIFI-1860 Added ContainerRequestFilter to redir...

[GitHub] nifi pull request: NIFI-1860 Added ContainerRequestFilter to redir...

Re: Reg: Get files from ftp

Re: Reg: Nifi Clustering

Reg: Get files from ftp

21 matches

Site Navigation

Mail list logo

Footer information