Re: Nifi to Titan... How?

[Pat Trainor]

Matt,

Great response, and thanks for writing. I would imagine querying would be
left to Java, Gephi, etc., and a Cassandra query (QueryCassandra) would
seem to do well, or even the REST API in Titan 1.0.0? But then again, I see
where Titan queries would be valuable, with Traversals not being the least
important.

I don't know what Nifi uses on the backend, but the Java API to the Gremin
server has been very simple to incorporate, as in:

public String doTest() {
Graph graph = new TinkerGraph();
Vertex a = graph.addVertex(null);
Vertex b = graph.addVertex(null);
a.setProperty("name", "Fred");
b.setProperty("name", "Frank");
Edge e = graph.addEdge(null, a, b, "knows");
String r = e.getVertex(Direction.OUT).getProperty("name")
+ "--" + e.getLabel()
+ "-->" + e.getVertex(Direction.IN).getProperty("name");
return r;
}

*results in:*
Fred--knows-->Frank

I'm grabbing NEs (Named Entities in Stamford-speak) from scraped html pages
now, and it makes sense for early stuff to make the name of the NE the key.
But that is too far down the rabbit hole for this discussion on this list.

What I will probably end up doing with Nifi to send & receive from parsers
is take advantage of it's ability to send & receive from endpoints (TCP,
UDP, even HTTP) to send Nifi content to a daemon I'll write that is sitting
on that endpoint. It may even be simpler to start another Nifi flow with an
endpoint that is waiting for my daemon to send it the results... Need to
experiment...

I'm just really trying not to load code from disk every time there is text
to be worked with... (sent from Nifi)...

Your idea for a GraphProcessor sounds extremely interesting! I write in
lots of languages, so I'm sitting here wondering if Nifi is doing compiles
on the fly, or...

Thanks to the NSA! But it makes me wonder what they're using NOW... :)

Thanks again...

pat
:)




Thanks!

pat 
( ͡° ͜ʖ ͡°)

LinkedIn 
Hobby/Fun Blog 
Sales Engineering University 



"A wise man can learn more from a foolish *question *than a fool can learn
from a wise *answer*". ~ Bruce Lee.

On Fri, May 20, 2016 at 11:20 PM, Matt Burgess  wrote:

> Pat,
>
> No worries, this discussion is relevant to the devs group :) I
> appreciate and share your interest in getting connected data into
> graph databases in order to get more insight out of the data.
>
> It may be possible to put the data directly into the backing store
> (Cassandra, e.g.) via NiFi, but in my experience that may be a fragile
> and possibly non-future-proof solution. IMHO I think you're on the
> right track with respect to taking graph-ready data and putting it
> directly into a graph DB like Titan.
>
> To that end, and there are some email thread(s) in this (and/or the
> users list) that mention it, I think we need a PutToGraphDatabase
> processor. In my mind, this processor takes GraphSON (or some other
> supported format) and writes it to a Tinkerpop graph DB (to include
> Titan, Neo4J, etc.). Conversion of input data should be possible with
> existing processors in NiFi, and such a Put processor would allow the
> user to pick the destination DB (Titan, Neo4J, Sail, OrientDB, e.g.)
>
> Querying existing graphs is a different animal; in fact, it's so
> complex that a DSL like Gremlin is likely the best play (as mentioned
> in the thread), but certainly a processor that hides the scaffolding
> would be helpful. Maybe we can get a graph bundle with
> PutToGraphDatabase and QueryGraphDatabase processors, if you agree
> (partially or wholly) please feel free to log Jira(s) to add such
> things :)
>
> As it turns out, I may have some free time this weekend (thanks
> Grandma for watching the kids!), I've wanted the PutToGraphDB
> processor for a while, as well as a Tinkerpop-enabled
> Site-to-Site-client to ingest NiFi flow files (containing graph-ready
> data) that writes to graph databases. Stay tuned to this list, if I
> get something useful I will be sure to share it. Also I would be very
> appreciative for any guidance, suggestions, and review you'd like to
> share :)
>
> Cheers,
> Matt
>
>
> On Fri, May 20, 2016 at 10:02 PM, Pat Trainor 
> wrote:
> > Joe Witt & Andre,
> >
> > You guys are very nice, but I thought this was the only mailing list for
> > Nifi... I now realize that there is a user's list, where this kind of
> > question would be more appropriate.
> >
> > Thanks for not tazing me... :)
> >
> > As for the great replies:
> >
> > Joe,
> >
> > I want the output to be used in Titan for analysis (which is why I
> figured
> > a Nifi->Titan connection was needed), but I didn't think of going right
> to
> > the DB store of Titan's. I'm wondering now if doing so will allow Titan
> to
> > do it's thing if graphs are created/modified in this fashion.
> >
> > 

Re: Nifi to Titan... How?

[Matt Burgess]

Pat,

No worries, this discussion is relevant to the devs group :) I
appreciate and share your interest in getting connected data into
graph databases in order to get more insight out of the data.

It may be possible to put the data directly into the backing store
(Cassandra, e.g.) via NiFi, but in my experience that may be a fragile
and possibly non-future-proof solution. IMHO I think you're on the
right track with respect to taking graph-ready data and putting it
directly into a graph DB like Titan.

To that end, and there are some email thread(s) in this (and/or the
users list) that mention it, I think we need a PutToGraphDatabase
processor. In my mind, this processor takes GraphSON (or some other
supported format) and writes it to a Tinkerpop graph DB (to include
Titan, Neo4J, etc.). Conversion of input data should be possible with
existing processors in NiFi, and such a Put processor would allow the
user to pick the destination DB (Titan, Neo4J, Sail, OrientDB, e.g.)

Querying existing graphs is a different animal; in fact, it's so
complex that a DSL like Gremlin is likely the best play (as mentioned
in the thread), but certainly a processor that hides the scaffolding
would be helpful. Maybe we can get a graph bundle with
PutToGraphDatabase and QueryGraphDatabase processors, if you agree
(partially or wholly) please feel free to log Jira(s) to add such
things :)

As it turns out, I may have some free time this weekend (thanks
Grandma for watching the kids!), I've wanted the PutToGraphDB
processor for a while, as well as a Tinkerpop-enabled
Site-to-Site-client to ingest NiFi flow files (containing graph-ready
data) that writes to graph databases. Stay tuned to this list, if I
get something useful I will be sure to share it. Also I would be very
appreciative for any guidance, suggestions, and review you'd like to
share :)

Cheers,
Matt


On Fri, May 20, 2016 at 10:02 PM, Pat Trainor  wrote:
> Joe Witt & Andre,
>
> You guys are very nice, but I thought this was the only mailing list for
> Nifi... I now realize that there is a user's list, where this kind of
> question would be more appropriate.
>
> Thanks for not tazing me... :)
>
> As for the great replies:
>
> Joe,
>
> I want the output to be used in Titan for analysis (which is why I figured
> a Nifi->Titan connection was needed), but I didn't think of going right to
> the DB store of Titan's. I'm wondering now if doing so will allow Titan to
> do it's thing if graphs are created/modified in this fashion.
>
> I'm only into Titan/Cassandra for a month or so now, starting off on
> Hadoop. I will go on the user list and see if anyone has taken this
> approach, and I'll try to figure out which connector (processor) would
> update/insert into Cassandra.
>
> Very insightful!
>
> Andre,
>
> This is promising, but the running of scripts is a way I really don't want
> to go in. I like the speed/performance of the code in memory (like a
> daemon), instead of loading off the HD every time it's needed. It would
> seem to me to be a limiting factor for scaling.
>
> Also, I'm really trying to not add anything more to the zoo I have now. If
> it is inevitable, so be it. I can write code in Java that talks to the
> tinkerpop3 stack and I can see the data in cassandra, but again, I have to
> load up the java program each time it is run.
>
> If a Processor in Nifi could just convert to the needed format/syntax and
> act against either Titan, Cassandra, or an existing, running component
> directly, it would make the flow of data very fast In my mind).
>
> I need to figure out what other users are doing, and I will do that on the
> user's list...
> Maybe like Joe said, I can go from Nifi processor directly to Cassandra...
> Sounds very interesting...
>
> Thanks again for putting up with a non-dev question, folks!
>
> pat
> :)
>
>
>
>
>
> Thanks!
>
> pat 
> ( ͡° ͜ʖ ͡°)
>
> LinkedIn 
> Hobby/Fun Blog 
> Sales Engineering University 
>
>
>
> "A wise man can learn more from a foolish *question *than a fool can learn
> from a wise *answer*". ~ Bruce Lee.
>
> On Fri, May 20, 2016 at 12:22 AM, Joe Witt  wrote:
>
>> Pat
>>
>> It looks like Titan can be backed by Apache Cassandra or Apache HBase.
>> NiFi can deliver to both of those.  Would that take care of what
>> you're looking to do?
>>
>> Thanks
>> Joe
>>
>> On Thu, May 19, 2016 at 11:16 PM, Pat Trainor 
>> wrote:
>> > Thanks to everyone making this now open sourced tool awesome.
>> >
>> > I can't find anything that directly links the 2.
>> >
>> > I want to use Nifi to coordinate page scraping, parsing, and finally
>> throw
>> > Titan data for graphs.
>> >
>> > Do I have to use Kafka or Spark for this?
>> >
>> > I'm looking at the output mechanisms (Processors), and I'm not a JSON
>> > expert, but I can write anything needed in Java...
>> >
>> > I kind of 

[GitHub] nifi pull request: NIFI-1909 Adding ability to process schemaless ...

[ryanpersaud]

Github user ryanpersaud commented on the pull request:

https://github.com/apache/nifi/pull/459#issuecomment-220753130
  
I made the changes that @markap14 suggested and removed unused imports that 
were identified by checkstyle.  I've attempted to run `mvn -Pcontrib-check` on 
3 different machines, but I get failures in tests for the nifi-framework-core 
module that seem unrelated to my modifications.  For example, running the build 
on a Linux machine, I get the following error:

Running TestSuite
Tests run: 172, Failures: 2, Errors: 0, Skipped: 1, Time elapsed: 272.879 
sec <<< FAILURE! - in TestSuite
testStateTooLargeExceptionThrownOnReplace on 
testStateTooLargeExceptionThrownOnReplace(org.apache.nifi.controller.state.providers.zookeeper.TestZooKeeperStateProvider)(org.apache.nifi.controller.state.providers.zookeeper.TestZooKeeperStateProvider)
  Time elapsed: 24.341 sec  <<< FAILURE!
org.junit.runners.model.TestTimedOutException: test timed out after 2 
milliseconds
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at 
org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1342)
at org.apache.zookeeper.ZooKeeper.setData(ZooKeeper.java:1268)
at 
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:318)
at 
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:283)
at 
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:228)
at 
org.apache.nifi.controller.state.providers.zookeeper.TestZooKeeperStateProvider.testStateTooLargeExceptionThrownOnReplace(TestZooKeeperStateProvider.java:176)






---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Nifi to Titan... How?

[Pat Trainor]

Joe Witt & Andre,

You guys are very nice, but I thought this was the only mailing list for
Nifi... I now realize that there is a user's list, where this kind of
question would be more appropriate.

Thanks for not tazing me... :)

As for the great replies:

Joe,

I want the output to be used in Titan for analysis (which is why I figured
a Nifi->Titan connection was needed), but I didn't think of going right to
the DB store of Titan's. I'm wondering now if doing so will allow Titan to
do it's thing if graphs are created/modified in this fashion.

I'm only into Titan/Cassandra for a month or so now, starting off on
Hadoop. I will go on the user list and see if anyone has taken this
approach, and I'll try to figure out which connector (processor) would
update/insert into Cassandra.

Very insightful!

Andre,

This is promising, but the running of scripts is a way I really don't want
to go in. I like the speed/performance of the code in memory (like a
daemon), instead of loading off the HD every time it's needed. It would
seem to me to be a limiting factor for scaling.

Also, I'm really trying to not add anything more to the zoo I have now. If
it is inevitable, so be it. I can write code in Java that talks to the
tinkerpop3 stack and I can see the data in cassandra, but again, I have to
load up the java program each time it is run.

If a Processor in Nifi could just convert to the needed format/syntax and
act against either Titan, Cassandra, or an existing, running component
directly, it would make the flow of data very fast In my mind).

I need to figure out what other users are doing, and I will do that on the
user's list...
Maybe like Joe said, I can go from Nifi processor directly to Cassandra...
Sounds very interesting...

Thanks again for putting up with a non-dev question, folks!

pat
:)





Thanks!

pat 
( ͡° ͜ʖ ͡°)

LinkedIn 
Hobby/Fun Blog 
Sales Engineering University 



"A wise man can learn more from a foolish *question *than a fool can learn
from a wise *answer*". ~ Bruce Lee.

On Fri, May 20, 2016 at 12:22 AM, Joe Witt  wrote:

> Pat
>
> It looks like Titan can be backed by Apache Cassandra or Apache HBase.
> NiFi can deliver to both of those.  Would that take care of what
> you're looking to do?
>
> Thanks
> Joe
>
> On Thu, May 19, 2016 at 11:16 PM, Pat Trainor 
> wrote:
> > Thanks to everyone making this now open sourced tool awesome.
> >
> > I can't find anything that directly links the 2.
> >
> > I want to use Nifi to coordinate page scraping, parsing, and finally
> throw
> > Titan data for graphs.
> >
> > Do I have to use Kafka or Spark for this?
> >
> > I'm looking at the output mechanisms (Processors), and I'm not a JSON
> > expert, but I can write anything needed in Java...
> >
> > I kind of like the elegance of Titan on Cassandra, and am reticent to add
> > more animals to my little ark!
> >
> > Just looking for pointers to the tech that would fit neatly...
> >
> > Thanks
> > in advance for your insights
> > !
> >
> > pat 
> > ( ͡° ͜ʖ ͡°)
>

[GitHub] nifi pull request: NIFI-786 AWS credential refactoring and enhance...

[jskora]

Github user jskora commented on the pull request:

https://github.com/apache/nifi/pull/244#issuecomment-220749583
  
I'm starting to review and test this.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Issues with Regex used with ReplaceTextWithMapping: where am I going wrong?

[idioma]

Hi,
I have come across an issue with my regex associated with
ReplaceTextWithMapping. I have attached input and mapping files together
with a NiFi template. My Json looks like this:

{"field1": "someText",
  "field2": "Text Again",
  "field3": "Text Again"}

I am trying to match the first occurrence of a phrase starting with capital
letter and having a space in between words (in my case "Text Again"). This
is the regex I have ended up with:

("[A-Za-z]+\s[A-Za-z]+")

It should match the first "Text Again" as tested with the online tester
https://regex101.com/ and replace that with the value from the mapping file:

"Text Again" "New Text"

However, it does not pick either and the substitution is not applied. Is my
regex incorrect? Can you help?

Thank you so much for all your help,

I.

3c370620-54fb-4a38-b987-f87cc2f56493.template

  

inputJson.json

  

mapping_file.txt

  




--
View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/Issues-with-Regex-used-with-ReplaceTextWithMapping-where-am-I-going-wrong-tp10592.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

[GitHub] nifi pull request: NIFI-1909 Adding ability to process schemaless ...

[markap14]

Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/459#discussion_r64097658
  
--- Diff: 
nifi-nar-bundles/nifi-avro-bundle/nifi-avro-processors/src/main/java/org/apache/nifi/processors/avro/ConvertAvroToJSON.java
 ---
@@ -128,49 +141,77 @@ public void onTrigger(ProcessContext context, 
ProcessSession session) throws Pro
 // Wrap a single record (inclusive of no records) only when a 
container is being used
 final boolean wrapSingleRecord = 
context.getProperty(WRAP_SINGLE_RECORD).asBoolean() && useContainer;
 
+final String stringSchema = context.getProperty(SCHEMA).getValue();
+final boolean schemaLess = stringSchema != null;
+
 try {
 flowFile = session.write(flowFile, new StreamCallback() {
 @Override
 public void process(final InputStream rawIn, final 
OutputStream rawOut) throws IOException {
-try (final InputStream in = new 
BufferedInputStream(rawIn);
- final OutputStream out = new 
BufferedOutputStream(rawOut);
- final DataFileStream reader = new 
DataFileStream<>(in, new GenericDatumReader())) {
+if (schemaLess) {
+if (schema == null) {
+schema = new 
Schema.Parser().parse(stringSchema);
+}
+try (final InputStream in = new 
BufferedInputStream(rawIn);
+ final OutputStream out = new 
BufferedOutputStream(rawOut)) {
+final DatumReader reader = new 
GenericDatumReader(schema);
+final BinaryDecoder decoder = 
DecoderFactory.get().binaryDecoder(in, null);
+final GenericRecord record = reader.read(null, 
decoder);
+
+// Schemaless records are singletons, so both 
useContainer and wrapSingleRecord
+// need to be true before we wrap it with an 
array
+if (useContainer && wrapSingleRecord) {
+out.write('[');
+}
 
-final GenericData genericData = GenericData.get();
+final byte[] outputBytes = (record == null) ? 
EMPTY_JSON_OBJECT : record.toString().getBytes(StandardCharsets.UTF_8);
--- End diff --

I think we need to keep the use of GenericData here. While 
record.toString() does in fact convert the Avro object to JSON, the toString() 
method is not documented as doing so and could change at any time. The 
GenericData object is documented to convert the object into JSON.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1909 Adding ability to process schemaless ...

[markap14]

Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/459#discussion_r64097258
  
--- Diff: 
nifi-nar-bundles/nifi-avro-bundle/nifi-avro-processors/src/main/java/org/apache/nifi/processors/avro/ConvertAvroToJSON.java
 ---
@@ -92,6 +103,7 @@
 .build();
 
 private List properties;
+private Schema schema = null;
--- End diff --

This could be marked by multiple different threads concurrently, so we need 
to make sure that it is protected. Marking it as 'volatile' should take care of 
this.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1909 Adding ability to process schemaless ...

[ryanpersaud]

GitHub user ryanpersaud opened a pull request:

https://github.com/apache/nifi/pull/459

NIFI-1909 Adding ability to process schemaless Avro records to

NIFI-1909 Adding ability to process schemaless Avro records to 
ConvertAvroToJson.  A new property named schema is added to the processor.  If 
a value is provided for this property, then it is assumed that incoming Avro 
records are schemaless and the value of the schema property is used when 
parsing incoming records.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ryanpersaud/nifi nifi-1909

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/459.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #459


commit dd03d6c573820ba6f8e356c288c0e25dc11f1dd0
Author: Ryan Persaud 
Date:   2016-05-20T19:27:39Z

NIFI-1909 Adding ability to process schemaless Avro records to 
ConvertAvrotoJson.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1568: Add Filter Capability to UnpackConte...

[rickysaltzer]

Github user rickysaltzer commented on the pull request:

https://github.com/apache/nifi/pull/248#issuecomment-220686739
  
@trkurc changed the default value to `.*` 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1568: Add Filter Capability to UnpackConte...

[rickysaltzer]

Github user rickysaltzer commented on a diff in the pull request:

https://github.com/apache/nifi/pull/248#discussion_r64087730
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/UnpackContent.java
 ---
@@ -111,8 +107,18 @@
 .name("Packaging Format")
 .description("The Packaging Format used to create the file")
 .required(true)
-.allowableValues(AUTO_DETECT_FORMAT, TAR_FORMAT, ZIP_FORMAT, 
FLOWFILE_STREAM_FORMAT_V3, FLOWFILE_STREAM_FORMAT_V2, FLOWFILE_TAR_FORMAT)
-.defaultValue(AUTO_DETECT_FORMAT)
+.allowableValues(PackageFormat.AUTO_DETECT_FORMAT.toString(), 
PackageFormat.TAR_FORMAT.toString(),
+PackageFormat.ZIP_FORMAT.toString(), 
PackageFormat.FLOWFILE_STREAM_FORMAT_V3.toString(),
+PackageFormat.FLOWFILE_STREAM_FORMAT_V2.toString(), 
PackageFormat.FLOWFILE_TAR_FORMAT.toString())
+.defaultValue(PackageFormat.AUTO_DETECT_FORMAT.toString())
+.build();
+
+public static final PropertyDescriptor FILE_FILTER = new 
PropertyDescriptor.Builder()
+.name("File Filter")
+.description("Only files whose names match the given regular 
expression will be extracted (tar/zip only)")
+.required(true)
+.defaultValue("[^\\.].*")
--- End diff --

@trkurc - I believe it does, I took it from the `GetFile` processor, but it 
doesn't really make sense here. I think changing this to just `.*` would make 
more sense. I'll do that. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1884 Defining API for Users, Groups, and P...

[jtstorck]

Github user jtstorck commented on a diff in the pull request:

https://github.com/apache/nifi/pull/452#discussion_r64086436
  
--- Diff: 
nifi-api/src/main/java/org/apache/nifi/authorization/AccessPolicy.java ---
@@ -0,0 +1,291 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.authorization;
+
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Objects;
+import java.util.Set;
+
+/**
+ * Defines a policy for a set of entities to perform a set of actions on a 
given resource.
+ */
+public class AccessPolicy {
+
+private final String identifier;
+
+private final Resource resource;
+
+private final Set entities;
+
+private final Set actions;
+
+private AccessPolicy(final AccessPolicyBuilder builder) {
+this.identifier = builder.identifier;
+this.resource = builder.resource;
+
+Set entities = new HashSet<>();
+if (builder.entities != null) {
+entities.addAll(builder.entities);
+}
+this.entities = Collections.unmodifiableSet(entities);
+
+Set actions = new HashSet<>();
+if (builder.actions != null) {
+actions.addAll(builder.actions);
+}
+this.actions = Collections.unmodifiableSet(actions);
+
+if (this.identifier == null || this.identifier.trim().isEmpty()) {
+throw new IllegalArgumentException("Identifier can not be null 
or empty");
+}
+
+if (this.resource == null) {
+throw new IllegalArgumentException("Resource can not be null");
+}
+
+if (this.entities == null || this.entities.isEmpty()) {
+throw new IllegalArgumentException("Entities can not be null 
or empty");
+}
+
+if (this.actions == null || this.actions.isEmpty()) {
+throw new IllegalArgumentException("Actions can not be null or 
empty");
+}
+}
+
+/**
+ * @return the identifier for this policy
+ */
+public String getIdentifier() {
+return identifier;
+}
+
+/**
+ * @return the resource for this policy
+ */
+public Resource getResource() {
+return resource;
+}
+
+/**
+ * @return an unmodifiable set of entity ids for this policy
+ */
+public Set getEntities() {
+return entities;
+}
+
+/**
+ * @return an unmodifiable set of actions for this policy
+ */
+public Set getActions() {
+return actions;
+}
+
+@Override
+public boolean equals(Object obj) {
+if (obj == null) {
+return false;
+}
+if (getClass() != obj.getClass()) {
+return false;
+}
+
+final AccessPolicy other = (AccessPolicy) obj;
+return Objects.equals(this.identifier, other.identifier);
+}
+
+@Override
+public int hashCode() {
+return Objects.hashCode(this.identifier);
+}
+
+@Override
+public String toString() {
+return String.format("identifier[%s], resource[%s], entityId[%s], 
action[%s]",
+getIdentifier(), getResource().getIdentifier(), 
getEntities(), getActions(), ", ");
+}
+
+/**
+ * Builder for Access Policies.
+ */
+public static class AccessPolicyBuilder {
+
+private String identifier;
+private Resource resource;
+private Set entities = new HashSet<>();
+private Set actions = new HashSet<>();
+private final boolean fromPolicy;
+
+/**
+ * Default constructor for building a new AccessPolicy.
+ */
+public AccessPolicyBuilder() {
+this.fromPolicy = false;

[GitHub] nifi pull request: NIFI-1568: Add Filter Capability to UnpackConte...

[trkurc]

Github user trkurc commented on a diff in the pull request:

https://github.com/apache/nifi/pull/248#discussion_r64084293
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/UnpackContent.java
 ---
@@ -111,8 +107,18 @@
 .name("Packaging Format")
 .description("The Packaging Format used to create the file")
 .required(true)
-.allowableValues(AUTO_DETECT_FORMAT, TAR_FORMAT, ZIP_FORMAT, 
FLOWFILE_STREAM_FORMAT_V3, FLOWFILE_STREAM_FORMAT_V2, FLOWFILE_TAR_FORMAT)
-.defaultValue(AUTO_DETECT_FORMAT)
+.allowableValues(PackageFormat.AUTO_DETECT_FORMAT.toString(), 
PackageFormat.TAR_FORMAT.toString(),
+PackageFormat.ZIP_FORMAT.toString(), 
PackageFormat.FLOWFILE_STREAM_FORMAT_V3.toString(),
+PackageFormat.FLOWFILE_STREAM_FORMAT_V2.toString(), 
PackageFormat.FLOWFILE_TAR_FORMAT.toString())
+.defaultValue(PackageFormat.AUTO_DETECT_FORMAT.toString())
+.build();
+
+public static final PropertyDescriptor FILE_FILTER = new 
PropertyDescriptor.Builder()
+.name("File Filter")
+.description("Only files whose names match the given regular 
expression will be extracted (tar/zip only)")
+.required(true)
+.defaultValue("[^\\.].*")
--- End diff --

@rickysaltzer - I'm a bit confused by this default value... is the default 
to filter files starting with "."? Does this change this processor's behavior?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1745: Refactor how revisions are handled a...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/454


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1799 Implements auto-scaling flow layout

[jtstorck]

Github user jtstorck commented on the pull request:

https://github.com/apache/nifi/pull/442#issuecomment-220668197
  
Hey @alopresto, I added a test case to address your previous comment 
regarding the flow file versioning.  The two test specs, PositionScalerSpec, 
and StandardFlowSynchronizerSpec, validate that flows missing an encoding 
version or having one that is less than 1.0 will result in the positions being 
scaled, and that they are scaled properly.  Thoughts?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1884 Defining API for Users, Groups, and P...

[bbende]

Github user bbende commented on a diff in the pull request:

https://github.com/apache/nifi/pull/452#discussion_r64076734
  
--- Diff: 
nifi-api/src/main/java/org/apache/nifi/authorization/AccessPolicy.java ---
@@ -0,0 +1,291 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.authorization;
+
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Objects;
+import java.util.Set;
+
+/**
+ * Defines a policy for a set of entities to perform a set of actions on a 
given resource.
+ */
+public class AccessPolicy {
+
+private final String identifier;
+
+private final Resource resource;
+
+private final Set entities;
+
+private final Set actions;
+
+private AccessPolicy(final AccessPolicyBuilder builder) {
+this.identifier = builder.identifier;
+this.resource = builder.resource;
+
+Set entities = new HashSet<>();
+if (builder.entities != null) {
+entities.addAll(builder.entities);
+}
+this.entities = Collections.unmodifiableSet(entities);
+
+Set actions = new HashSet<>();
+if (builder.actions != null) {
+actions.addAll(builder.actions);
+}
+this.actions = Collections.unmodifiableSet(actions);
+
+if (this.identifier == null || this.identifier.trim().isEmpty()) {
+throw new IllegalArgumentException("Identifier can not be null 
or empty");
+}
+
+if (this.resource == null) {
+throw new IllegalArgumentException("Resource can not be null");
+}
+
+if (this.entities == null || this.entities.isEmpty()) {
+throw new IllegalArgumentException("Entities can not be null 
or empty");
+}
+
+if (this.actions == null || this.actions.isEmpty()) {
+throw new IllegalArgumentException("Actions can not be null or 
empty");
+}
+}
+
+/**
+ * @return the identifier for this policy
+ */
+public String getIdentifier() {
+return identifier;
+}
+
+/**
+ * @return the resource for this policy
+ */
+public Resource getResource() {
+return resource;
+}
+
+/**
+ * @return an unmodifiable set of entity ids for this policy
+ */
+public Set getEntities() {
+return entities;
+}
+
+/**
+ * @return an unmodifiable set of actions for this policy
+ */
+public Set getActions() {
+return actions;
+}
+
+@Override
+public boolean equals(Object obj) {
+if (obj == null) {
+return false;
+}
+if (getClass() != obj.getClass()) {
+return false;
+}
+
+final AccessPolicy other = (AccessPolicy) obj;
+return Objects.equals(this.identifier, other.identifier);
+}
+
+@Override
+public int hashCode() {
+return Objects.hashCode(this.identifier);
+}
+
+@Override
+public String toString() {
+return String.format("identifier[%s], resource[%s], entityId[%s], 
action[%s]",
+getIdentifier(), getResource().getIdentifier(), 
getEntities(), getActions(), ", ");
+}
+
+/**
+ * Builder for Access Policies.
+ */
+public static class AccessPolicyBuilder {
+
+private String identifier;
+private Resource resource;
+private Set entities = new HashSet<>();
+private Set actions = new HashSet<>();
+private final boolean fromPolicy;
+
+/**
+ * Default constructor for building a new AccessPolicy.
+ */
+public AccessPolicyBuilder() {
+this.fromPolicy = false;
+ 

[GitHub] nifi pull request: Nifi 1540 - AWS Kinesis Get and Put Processors

[jvwing]

Github user jvwing commented on the pull request:

https://github.com/apache/nifi/pull/239#issuecomment-220657759
  
@mans2singh, I am a bit behind on this PR, but will try to get up to speed 
and contribute to the review.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi-cpp pull request: completion of C++ framework for MiNi...

[benqiu2016]

Github user benqiu2016 closed the pull request at:

https://github.com/apache/nifi-minifi-cpp/pull/3


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi-cpp pull request: MINIFI-6: Basic C++ implementation f...

[benqiu2016]

GitHub user benqiu2016 opened a pull request:

https://github.com/apache/nifi-minifi-cpp/pull/4

MINIFI-6: Basic C++ implementation for MiNiFi



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/benqiu2016/nifi-minifi-cpp MINIFI-6

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-minifi-cpp/pull/4.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4


commit 9192675cc2fe015b75ba9cf24a47108e922a394f
Author: Bin Qiu 
Date:   2016-05-19T16:46:04Z

MINIFI-6: Basic C++ implementation for MiNiFi




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: Nifi 1540 - AWS Kinesis Get and Put Processors

[mans2singh]

Github user mans2singh commented on the pull request:

https://github.com/apache/nifi/pull/239#issuecomment-220644414
  
Hey @olegz @apiri @jvwing @joewitt - I've tried to address your review 
comments above.  Please let me know if you have any feedback for me.  Thanks 
Mans


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi-cpp pull request: completion of C++ framework for MiNi...

[benqiu2016]

GitHub user benqiu2016 opened a pull request:

https://github.com/apache/nifi-minifi-cpp/pull/3

completion of C++ framework for MiNiFi6



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/nifi-minifi-cpp MINIFI-6

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-minifi-cpp/pull/3.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3


commit 62394f727e48382c982e23b5b29277238de816ad
Author: Bin Qiu 
Date:   2016-04-28T15:46:53Z

MINIFI-6: Initial Checkin for basic C++ MiNiFi framework

commit 9280e814d6633cb4a91e574c66e4b801a1ccad6c
Author: Bin Qiu 
Date:   2016-05-03T21:29:56Z

MINIFI-6: Add ProcessGroup/GenerateFlowFile, etc

commit 31c9a1b61d2e14fa567eda3737c415ba965b14a1
Author: Aldrin Piri 
Date:   2016-05-05T13:06:57Z

Adding ASL header to Makefile.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi-cpp pull request: Minifi 6

[benqiu2016]

Github user benqiu2016 closed the pull request at:

https://github.com/apache/nifi-minifi-cpp/pull/2


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1907 Moving lazy init of SSLContext...

[bbende]

GitHub user bbende opened a pull request:

https://github.com/apache/nifi/pull/457

NIFI-1907 Moving lazy init of SSLContext...

to StandardSiteToSiteClientConfig rather than the builder.

Tested secure SiteToSite still working between two NiFi nodes, and now 
example Storm topology can also connect to a secured NiFi instance.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/bbende/nifi NIFI-1907

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/457.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #457


commit 25bae0fd890b53f2f201d1e5475d5d0e1708580e
Author: Bryan Bende 
Date:   2016-05-20T15:15:36Z

NIFI-1907 Moving lazy init of SSLContext to StandardSiteToSiteClientConfig 
rather than the builder




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Output connection is invalid

[Mark Payne]

Hello,

I believe the confusion here is coming from the fact that Output Ports on the 
Root Group
(the very top level of your canvas) are treated a little bit differently than 
other ports in your graph.
The idea of a port is that it lets data into or out of your process group 
(depending on if it's an input
port or output port).

Ports on the Root Group then let data into or out of your entire NiFi instance. 
This is accomplished
via site-to-site. So if you want to use a port for site-to-site communications, 
that port must be on
the Root Group. From the error message here, we see that the port is a Local 
Port
("LocalPort[name=Data_For_Spark,id=6063528e-6313-4cba-93a7-351c396eaed4]") 
rather than a
Root Group Port.

So you will need to create an Output Port on your root group and then connect 
your current Process Group
to it by dragging a connection from the Process Group to that port.

Does this make sense?

Thanks
-Mark


> On May 20, 2016, at 9:38 AM, pradeepbill  wrote:
> 
> I have a spark application reading from an output port in NIFI, and I have
> the spark application running, but when I try to start the nifi flow, it
> says below error message, where should I define the "Output connection"
> ?.Please advice
> 
> 
> Node sabn6:8282 is unable to fulfill this request due to:
> LocalPort[name=Data_For_Spark,id=6063528e-6313-4cba-93a7-351c396eaed4] is
> not in a valid state: Output connection for port 'Data_For_Spark' is not
> defined
> 
> 
> 
> --
> View this message in context: 
> http://apache-nifi-developer-list.39713.n7.nabble.com/Output-connection-is-invalid-tp10572.html
> Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Output connection is invalid

[pradeepbill]

I have a spark application reading from an output port in NIFI, and I have
the spark application running, but when I try to start the nifi flow, it
says below error message, where should I define the "Output connection"
?.Please advice


Node sabn6:8282 is unable to fulfill this request due to:
LocalPort[name=Data_For_Spark,id=6063528e-6313-4cba-93a7-351c396eaed4] is
not in a valid state: Output connection for port 'Data_For_Spark' is not
defined



--
View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/Output-connection-is-invalid-tp10572.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

[GitHub] nifi pull request: NIFI-1905 enabled ExecuteProcess to terminate p...

[olegz]

Github user olegz commented on a diff in the pull request:

https://github.com/apache/nifi/pull/456#discussion_r64043923
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteProcess.java
 ---
@@ -181,7 +183,14 @@ public Thread newThread(final Runnable r) {
 
 @OnUnscheduled
 public void shutdownExecutor() {
-executor.shutdown();
+try {
+executor.shutdown();
+} finally {
+if (this.externalProcess.isAlive()) {
+this.getLogger().warn("Interrupting external process since 
it did not finished at the time stop Processor was initiated.");
--- End diff --

Fair, i'll change. Let me know when you get a chance to test it in your 
environment.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1745: Refactor how revisions are handled a...

[markap14]

Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/454#discussion_r64043396
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-optimistic-locking/src/main/java/org/apache/nifi/web/revision/NaiveRevisionManager.java
 ---
@@ -423,7 +440,7 @@ public boolean requestWriteLock(final Revision 
proposedRevision) throws ExpiredR
 throw ise;
 }
 
-if (stamp.getClientId() == null || 
stamp.getClientId().equals(proposedRevision.getClientId())) {
+if (stamp.getUser() == null || 
stamp.getUser().equals(user)) {
--- End diff --

@mcgilman good call. I have addressed the issue and updated the PR.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1884 Defining API for Users, Groups, and P...

[jtstorck]

Github user jtstorck commented on the pull request:

https://github.com/apache/nifi/pull/452#issuecomment-220607952
  
+1 looks good to me at this point, though @mcgilman's earlier comment on 
making MutableAuthorizer a base class does sound good, since the idea of the 
MutableAuthorizer was not so much about the authorization, but about the 
management of users/groups/policies from the NiFi UI.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1884 Defining API for Users, Groups, and P...

[jtstorck]

Github user jtstorck commented on a diff in the pull request:

https://github.com/apache/nifi/pull/452#discussion_r64040788
  
--- Diff: 
nifi-api/src/main/java/org/apache/nifi/authorization/AccessPolicy.java ---
@@ -0,0 +1,291 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.authorization;
+
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Objects;
+import java.util.Set;
+
+/**
+ * Defines a policy for a set of entities to perform a set of actions on a 
given resource.
+ */
+public class AccessPolicy {
+
+private final String identifier;
+
+private final Resource resource;
+
+private final Set entities;
+
+private final Set actions;
+
+private AccessPolicy(final AccessPolicyBuilder builder) {
+this.identifier = builder.identifier;
+this.resource = builder.resource;
+
+Set entities = new HashSet<>();
+if (builder.entities != null) {
+entities.addAll(builder.entities);
+}
+this.entities = Collections.unmodifiableSet(entities);
+
+Set actions = new HashSet<>();
+if (builder.actions != null) {
+actions.addAll(builder.actions);
+}
+this.actions = Collections.unmodifiableSet(actions);
+
+if (this.identifier == null || this.identifier.trim().isEmpty()) {
+throw new IllegalArgumentException("Identifier can not be null 
or empty");
+}
+
+if (this.resource == null) {
+throw new IllegalArgumentException("Resource can not be null");
+}
+
+if (this.entities == null || this.entities.isEmpty()) {
+throw new IllegalArgumentException("Entities can not be null 
or empty");
+}
+
+if (this.actions == null || this.actions.isEmpty()) {
+throw new IllegalArgumentException("Actions can not be null or 
empty");
+}
+}
+
+/**
+ * @return the identifier for this policy
+ */
+public String getIdentifier() {
+return identifier;
+}
+
+/**
+ * @return the resource for this policy
+ */
+public Resource getResource() {
+return resource;
+}
+
+/**
+ * @return an unmodifiable set of entity ids for this policy
+ */
+public Set getEntities() {
+return entities;
+}
+
+/**
+ * @return an unmodifiable set of actions for this policy
+ */
+public Set getActions() {
+return actions;
+}
+
+@Override
+public boolean equals(Object obj) {
+if (obj == null) {
+return false;
+}
+if (getClass() != obj.getClass()) {
+return false;
+}
+
+final AccessPolicy other = (AccessPolicy) obj;
+return Objects.equals(this.identifier, other.identifier);
+}
+
+@Override
+public int hashCode() {
+return Objects.hashCode(this.identifier);
+}
+
+@Override
+public String toString() {
+return String.format("identifier[%s], resource[%s], entityId[%s], 
action[%s]",
+getIdentifier(), getResource().getIdentifier(), 
getEntities(), getActions(), ", ");
+}
+
+/**
+ * Builder for Access Policies.
+ */
+public static class AccessPolicyBuilder {
+
+private String identifier;
+private Resource resource;
+private Set entities = new HashSet<>();
+private Set actions = new HashSet<>();
+private final boolean fromPolicy;
+
+/**
+ * Default constructor for building a new AccessPolicy.
+ */
+public AccessPolicyBuilder() {
+this.fromPolicy = false;

[GitHub] nifi pull request: NIFI-1905 enabled ExecuteProcess to terminate p...

[aperepel]

Github user aperepel commented on a diff in the pull request:

https://github.com/apache/nifi/pull/456#discussion_r64040583
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteProcess.java
 ---
@@ -181,7 +183,14 @@ public Thread newThread(final Runnable r) {
 
 @OnUnscheduled
 public void shutdownExecutor() {
-executor.shutdown();
+try {
+executor.shutdown();
+} finally {
+if (this.externalProcess.isAlive()) {
+this.getLogger().warn("Interrupting external process since 
it did not finished at the time stop Processor was initiated.");
--- End diff --

Check for typos and wording, maybe simplify it? How about 'Process hasn't 
terminated on request, forcing the interrupt'? 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request: NIFI-1905 enabled ExecuteProcess to terminate p...

[olegz]

GitHub user olegz opened a pull request:

https://github.com/apache/nifi/pull/456

NIFI-1905 enabled ExecuteProcess to terminate process



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/olegz/nifi NIFI-1905

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/456.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #456


commit 7924933b892d3f6b59741389b53ca50c7781e728
Author: Oleg Zhurakousky 
Date:   2016-05-20T13:05:44Z

NIFI-1905 enabled ExecuteProcess to terminate process




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Contribution ListFTP and FetchSTP processors

[Joris Mees]

Hi,

We (Data Minded) are a company mainly active in consultancy services regarding 
data solutions. We have recently deployed a Nifi instance at a client and 
needed functionality for a ListFTP and FetchFTP processor as described in the 
following Jira issue:

https://issues.apache.org/jira/browse/NIFI-1844?jql=text%20~%20%22listftp%22 


As this Jira ticket was still open we developed our own ListFTP and FecthFTP 
processors based on the already available SFTP processors. It turned out that 
almost all functionality was already there in the FTPTransfer class. Is there 
any particular reason why the FTP processors are not yet part of the standard 
processors as despite of some small changes they are already there?

We are currently testing the processors but for now they seem to work fine. We 
are happy to contribute them to the community, please let us know the following 
steps and requirements to do so.

Kind regards
Joris Mees

Data Minded
joris.m...@dataminded.be 
http://www.dataminded.be  

Re: Nifi to Titan... How?

[Andre]

Pat,

There's a few ways of skinning this cat but the common approach is to make
use of Tinkerpop's Gremlin using Groovy.

I would then suggest tying the Groovy code with NiFi's ExecuteScript
processor. This would ensure your code is reasonably re-usable across
different Graph DBs (as long they implement the usual Tinkerpop stuff).

This comment http://stackoverflow.com/a/33830733/76 and this TinkerPop
wiki https://github.com/tinkerpop/gremlin/wiki/Using-Gremlin-through-Groovy
should shed some light on the Gremlin Groovy syntax.

Cheers



On Fri, May 20, 2016 at 1:16 PM, Pat Trainor  wrote:

> Thanks to everyone making this now open sourced tool awesome.
>
> ​I can't find anything that directly links the 2.
>
> I want to use ​Nifi to coordinate page scraping, parsing, and finally throw
> Titan data for graphs.
>
> ​Do I have to use Kafka or Spark for this?
>
> I'm looking at the output mechanisms (Processors), and I'm not a JSON
> expert, but I can write anything needed in Java...
>
> I kind of like the elegance of Titan on Cassandra, and am reticent to add
> more animals to my little ark!​
>
> ​Just looking for pointers to the tech that would fit neatly...​
>
> Thanks
> ​ in advance for your insights​
> !
>
> pat 
> ( ͡° ͜ʖ ͡°)
>