[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran closed the pull request at:

https://github.com/apache/nifi-minifi/pull/118


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r176297389
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/extension/ExtensionComponent.java
 ---
@@ -0,0 +1,85 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model.extension;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.List;
+import java.util.Set;
+
+/**
+ * A component provided by an extension bundle
+ */
+@ApiModel
+public class ExtensionComponent extends DefinedType {
+
+// TODO, does arch/binary/compiler metadata need to be added here?
--- End diff --

Ok, so here is what I came up with for now:

An extension component is uniquely identified by (bundle, type), where 
bundle = (group, artifact, version) and type is a fully qualified class name. 
Bundle is optional in the case that you have an extension component outside of 
a bundle (common in the C++ case for now), in which case the fully qualified 
type must be globally unique.

An extension component implementation is uniquely identified (bundle, type, 
build) where build = (version, revision, timestamp, target architecture, 
compiler, compiler flags)

In general, a flow designer should only have to target an extension 
component in order to author a flow. Whether or not a flow can run on a given 
agent depends on if the extension component has an available implementation on 
that agent.

It's still a bit hand-wavy (at least for now), but I think this distinction 
should work once we have more dynamic authoring and deployment logic that 
leverages a fully featured device registry and extension registry. For now, we 
will focus on agent class name compatibility, so we will have to assume that 
manifests for classes don't change (or at least, don't remove capabilities) and 
use the intersection of all agent manifest capabilities when designing against 
a an agent class label. 

I need to flesh it out a bit more, but I think it should work. To get a 
sense of what some of this looks like in code, look at the equals/hashcode 
methods for DefinedType and ExtensionComponent after I push my next commit.


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r176286360
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/FlowUri.java
 ---
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+@ApiModel("Uniform Resource Identifier for NiFi Versioned Flows saved to a 
NiFi Registry")
+public class FlowUri {
--- End diff --

Will do!


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[apiri]

Github user apiri commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r176286241
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/FlowUri.java
 ---
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+@ApiModel("Uniform Resource Identifier for NiFi Versioned Flows saved to a 
NiFi Registry")
+public class FlowUri {
--- End diff --

I dig the constructor/toString idea.  Gives us more flexibility and can 
tack things on as needed for whatever reason.


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r176281479
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/FlowUri.java
 ---
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+@ApiModel("Uniform Resource Identifier for NiFi Versioned Flows saved to a 
NiFi Registry")
+public class FlowUri {
--- End diff --

Yes it is {base_url}/buckets/{bucketId}/flows/{flowId}. We could use that 
in place of this object if preferred. or we could use this object but have a 
constructor / toString that is capable of going from/to that form.


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r176280748
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/FlowStatus.java
 ---
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.Map;
+
+/**
+ * Status of the aspects of the flow that are controllable by the C2 
server, ie:
+ *   - Processors that can be started/stopped and their current state
+ *   - Queues that can be cleared and their current state
+ */
+@ApiModel
+public class FlowStatus {
--- End diff --

OK, I got it. The current structure assumes to much and is limiting. Will 
take another look at that, thanks.

Yes, I don't think these DTOs need to worry about concurrent access. 
Official model and state and would be managed at a lower level in the C2 server 
(persisted or cached and accessed through thread safe interfaces). These 
objects represent a snapshot copy for I/O. It should become clearer in my next 
PR that adds some of the service and persistence layer foundations.


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r176279968
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/extension/BundleManifest.java
 ---
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model.extension;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.List;
+
+@ApiModel
+public class BundleManifest {
+
--- End diff --

OK, I like that too. I'll add top level components to the AgentManifest, 
and devs can choose which makes more sense for them. Flow Designer can adapt to 
both.


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r176278810
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/C2Operation.java
 ---
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import javax.validation.constraints.NotBlank;
+import java.util.Map;
+
+@ApiModel
+public class C2Operation {
+
+private String identifier;
--- End diff --

OK got it, thanks. It seems that a protocol distinction here is that 
operations in the top level requestedOperations list are always executed, but 
nested operations only execute if their parent succeeds?


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r176278082
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/extension/ExtensionComponent.java
 ---
@@ -0,0 +1,85 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model.extension;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.List;
+import java.util.Set;
+
+/**
+ * A component provided by an extension bundle
+ */
+@ApiModel
+public class ExtensionComponent extends DefinedType {
+
+// TODO, does arch/binary/compiler metadata need to be added here?
--- End diff --

> ... would it make this an implementation of N types of components based 
on flags/architectures or a singular object which is encapsulated elsewhere, 
with the necessary information...

Great question, and one I am still grappling with myself. I'd like to 
abstract these details away from the user as well and make it as easy as 
possible for both extension authors and extension users. I think I would lean 
towards something along the lines that an extension should define an interface 
(eg, the metadata in this model) that can be implemented on multiple platforms. 
In otherwords, if I choose to support the JVM, x86, and ARM, I could publish 
that as a single extension (MyCompany, MyExtension, v1.0) and there is some 
metadata that lets the MiNiFi/NiFi ecosystem know about the platform-specific 
binaries. I think this is heading down the path of a full-fledged Extension 
Registry though, which might be biting of a bit to much for this initial 
version of the C2 effort (we aren't even attempting to host binaries or 
facilitate centralized distribution and dynamic installation). That said, If 
anyone can think of a minimal version of that concept that gets us on the right 
path,
  I'll be happy to include it!


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r176273652
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/extension/ExtensionComponent.java
 ---
@@ -0,0 +1,85 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model.extension;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.List;
+import java.util.Set;
+
+/**
+ * A component provided by an extension bundle
+ */
+@ApiModel
+public class ExtensionComponent extends DefinedType {
+
+// TODO, does arch/binary/compiler metadata need to be added here?
--- End diff --

There may be additional toolchain formation we may need to carry over. I'm 
not 100% certain at this time if we would need all of that. It comes down to 
making a decision based on the compile time flags to know if an extension or 
implementation of an extension is translatable to an agent's architecture. I'm 
a little fuzzy on how this model views ExtensionComponent as it relates to an 
Extension of architectures types. In that case would it make this an 
implementation of N types of components based on flags/architectures or a 
singular object which is encapsulated elsewhere, with the necessary 
information...


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r176272024
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/extension/ExtensionComponent.java
 ---
@@ -0,0 +1,85 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model.extension;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.List;
+import java.util.Set;
+
+/**
+ * A component provided by an extension bundle
+ */
+@ApiModel
+public class ExtensionComponent extends DefinedType {
+
+// TODO, does arch/binary/compiler metadata need to be added here?
--- End diff --

For extensions we probably need to carry the type of built binary, to 
include arch built, compiler flags that could limit cross platform 
compatibility, and anything else that could possibly limit movement of that 
object. Ideally this is all abstracted from the user, so whether that lives 
here or elsewhere is unclear; however we would need to be certain a user isn't 
getting x86 shared object when they're running on a pi. 


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r176271235
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/FlowStatus.java
 ---
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.Map;
+
+/**
+ * Status of the aspects of the flow that are controllable by the C2 
server, ie:
+ *   - Processors that can be started/stopped and their current state
+ *   - Queues that can be cleared and their current state
+ */
+@ApiModel
+public class FlowStatus {
--- End diff --

Perhaps it is the naming, but the name  FlowComponentStatus implies that it 
is part of the flow, but there may be components that run that are flow 
independent. For example think of things that may run a procedure. That 
component could be tracked in the FlowComponentStatus map components, but 
perhaps not. 

Additionally, not sure I mentioned this already but I see that tracking 
maps are not concurrent map impls. Should I assume that state is 
monitored/controlled elsewhere? 


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r176270398
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/C2Operation.java
 ---
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import javax.validation.constraints.NotBlank;
+import java.util.Map;
+
+@ApiModel
+public class C2Operation {
+
+private String identifier;
--- End diff --

Think of it like this 
OpA {
  OpB {
 OpC
  }
  OpD{}}
}
OpC would not execute if B fails


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[apiri]

Github user apiri commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r176268253
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/Device.java
 ---
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+@ApiModel
+public class Device extends DeviceInfo {
+
+private String name;
+private String firstSeen;  // TODO, decide on standard JSON format for 
timestamps in the C2 REST API
--- End diff --

My inclination would be to just do this in epoch millis UTC  and munge it 
however needed.


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[apiri]

Github user apiri commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r176267863
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/FlowUri.java
 ---
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+@ApiModel("Uniform Resource Identifier for NiFi Versioned Flows saved to a 
NiFi Registry")
+public class FlowUri {
--- End diff --

Not sure of how this is represented by Registry but is there an actual URI 
form?

https://registry.myorganization.org/nifi-registry//?


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[apiri]

Github user apiri commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r176267101
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/extension/BundleManifest.java
 ---
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model.extension;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.List;
+
+@ApiModel
+public class BundleManifest {
+
--- End diff --

I think I like the idea of the direct access to add components.  Bundles 
are nice for the Java side of the house but don't necessarily map well to C++.  
From the standpoint of the flow designer, the notions of bundles should be 
minimal but some defined tuple (class, version, etc) would be all that matters 
as a keyed extension (which has a fuller model as outlined with the other work 
that has gone on in this PR). 


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175938847
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/AgentRepositoryStatus.java
 ---
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+@ApiModel
+public class AgentRepositoryStatus {
+
+private Long size;
+private Long sizeMax;
+private Long count;
+private Long countMax;
--- End diff --

For now I went with these field names:

private Long size;
private Long sizeMax;
private Long dataSize;
private Long dataSizeMax;

Thoughts?


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175844957
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/extension/BundleManifest.java
 ---
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model.extension;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.List;
+
+@ApiModel
+public class BundleManifest {
+
--- End diff --

::sigh:: I typed a long response that I think got lost before I hit the 
button.

The gist of it is -> I need to think about this. Currently the data model 
requires defining an extension in the context of a bundle. If there is no 
actual bundle, you could use a default bundle (I even included a constructor 
`defaultBundle()` to create one) that indicates there is no actual bundle used. 

Alternatively, we could add ExtensionComponents directly to AgentManifest 
so you could include them without creating fake bundles to act as containers. 
Thoughts?


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175843712
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/extension/DefinedType.java
 ---
@@ -0,0 +1,69 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model.extension;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+/**
+ * A reference to a defined type identified by bundle and fully qualified 
class type identifiers
+ */
+@ApiModel
+public class DefinedType {
+
+private String group;
+private String artifact;
+private String version;
+private String type;
--- End diff --

I think in practice it will just be a class that is deployable, i.e., via a 
flow config.yaml. So basically a processor, controller service, or reporting 
task.

But it could be more than that. For example, in NiFi, some of the core 
framework interface implementations are actually built as NARs and deployed by 
including them in the /lib dir. So givent that the model in the 
`org.apache.nifi.minifi.c2.model.extension` package is designed to be a 
prototype for a model that could eventually live in an Extension Registry, 
these could be used in the NiFi case as well. But for our purposes, just the 
types that would be used as Flow Components


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175813531
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/FlowStatus.java
 ---
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.Map;
+
+/**
+ * Status of the aspects of the flow that are controllable by the C2 
server, ie:
+ *   - Processors that can be started/stopped and their current state
+ *   - Queues that can be cleared and their current state
+ */
+@ApiModel
+public class FlowStatus {
--- End diff --

Not sure I follow. Do you have an example of something that might not into 
the proposed model?


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175812152
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/C2Heartbeat.java
 ---
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.Map;
+
+@ApiModel
+public class C2Heartbeat {
+
+// TODO, timestamp?
+private DeviceInfo deviceInfo;
+private AgentInfo agentInfo;
+private FlowInfo flowInfo;
+private Map metrics;
--- End diff --

Typed metrics have been incorporated as nested objects of the top-level 
DeviceInfo, AgentInfo, and FlowInfo objects.

The thinking of the additional `Map metrics` fields was 
that it might be useful for agents to include some custom measurements in a 
heartbeat that are not "core" to the MiNiFi agent, but are relevant to the 
particular deployment use case or scenario. For example:

"metrics": {
"com.example.device.battery.health": "0.8",
"com.example.device.battery.cycle-count": "334",
"com.example.device.battery.charge": "0.2"
}

I was thinking that if we built such a framework, all metrics would be 
produced as string values (to make it easy to recieve adhoc metrics without 
typing metadata), so the it would be the responsibility of the consumer, 
presumably customize, to cast the metrics to the desired type. The idea was 
that the C2 server would just be an input point for these adhoc metrics that 
would then get forwarded to some other time-series metrics store such as Ambari 
Metrics System, Graphite, Datadog, etc. Those storage systems usually provide 
their own type conversion capabilities, so I was thinking it could keep the 
requirements for of defining these customized metrics lightweight and pass the 
interpretation logic downstream.

I don't think there is a need/use for this field yet, so I'll remove this 
for this PR. At a minimum, it requires work on the agent side to enable a way 
to add custom metrics to a heartbeat, so we don't need to officially put it 
into the C2 protocol until we have a need and end-to-end design to support it. 
There also might be a different way to solve this problem (I could think of 
ways to do it without C2 just using site-to-site, or maybe C2 heartbeats is the 
right approach, but accurate typing is needed) We can discuss it as a feature 
proposal and re-introduce it if people have a need for a way to add custom 
metrics to be collected by the MiNiFi agent to be reported in the heartbeat to 
the C2 Server.


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175588599
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/C2Operation.java
 ---
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import javax.validation.constraints.NotBlank;
+import java.util.Map;
+
+@ApiModel
+public class C2Operation {
+
+private String identifier;
--- End diff --

Interesting. I guess I missed that. Can you provide an example?


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175587496
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/BuildInfo.java
 ---
@@ -0,0 +1,77 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+@ApiModel
+public class BuildInfo {
--- End diff --

Yeah, good point. We probably need it at the extensioncomponent level as 
well. See my comment below.


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175587202
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/extension/ExtensionComponent.java
 ---
@@ -0,0 +1,85 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model.extension;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.List;
+import java.util.Set;
+
+/**
+ * A component provided by an extension bundle
+ */
+@ApiModel
+public class ExtensionComponent extends DefinedType {
+
+// TODO, does arch/binary/compiler metadata need to be added here?
+
+private List providedApiImplementations;
+
+private String description;
+private Set tags;
+
+private Boolean deprecated;
+private String deprecationReason;
+
+@ApiModelProperty("If this type represents a provider for an 
interface, this lists the APIs it implements")
+public List getProvidedApiImplementations() {
+return providedApiImplementations;
+}
+
+public void setProvidedApiImplementations(List 
providedApiImplementations) {
+this.providedApiImplementations = providedApiImplementations;
--- End diff --

The intention of this field is to allow a component to declare that it 
provides an implementation of an interface defined by another DefinedType. 

The goal (the details of which can change) is to allow bundles to declare 
interfaces/headers as DefinedTypes that other components can provide 
implementations for.  

This is used in the NiFi UI, so will also likely be useful in the [Flow 
Designer](https://lists.apache.org/thread.html/b720ff7b6e4e88fb23163b279f2d71eb125df18d1be8a7fca1c9b28d@%3Cdev.nifi.apache.org%3E).
 In NiFi, it is used so that property descriptors can mark a property as 
requiring an API implementation selection as its value, so that those options 
can be presented in a drop down selection. That is one thought of how this will 
be used in this context.

Another possible use case of this is defining an interface in version 1.0 
of and extension bundle and letting implementations of that interface 
explicitly declare support for in future versions of the bundle. For example, 
you could have:

MyBundle v1.0.0
apis: [org::apache:nifi:minifi::DatabaseService]
controllerServices: org::apache:nifi:minifi::DatabaseServiceImpl
provides: 
[MyBundle/v1.0.0/org::apache:nifi:minifi::DatabaseService]

MyBundle v1.0.1
controllerServices: org::apache:nifi:minifi::DatabaseServiceImpl
provides: 
[MyBundle/v1.0.0/org::apache:nifi:minifi::DatabaseService]

MyBundle v1.1.0
apis: [org::apache:nifi:minifi::DatabaseService]
controllerServices: org::apache:nifi:minifi::DatabaseServiceImpl
provides: 
[MyBundle/v1.0.0/org::apache:nifi:minifi::DatabaseService, 
   
MyBundle/v1.1.0/org::apache:nifi:minifi::DatabaseService]

MyBundle v2.0.0
apis: [org::apache:nifi:minifi::DatabaseService]
controllerServices: org::apache:nifi:minifi::DatabaseServiceImpl
provides: 
[MyBundle/v2.0.0/org::apache:nifi:minifi::DatabaseService]

In this example, an API is defined in a 1.0, improved in a patch release, 
evolved in a minor release, and then ultimately rethought with breaking changes 
in a 2.0. The implementation of the ControllerService API, which happens to be 
in the same bundle in this example, can declare compatibility with current or 
previous versions.



---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175587026
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/extension/DefinedType.java
 ---
@@ -0,0 +1,69 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model.extension;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+/**
+ * A reference to a defined type identified by bundle and fully qualified 
class type identifiers
+ */
+@ApiModel
+public class DefinedType {
+
+private String group;
+private String artifact;
+private String version;
+private String type;
+
+@ApiModelProperty("The group name of the bundle that provides the 
referenced type.")
+public String getGroup() {
+return group;
+}
+
+public void setGroup(String group) {
+this.group = group;
+}
+
+@ApiModelProperty("The artifact name of the bundle that provides the 
referenced type.")
+public String getArtifact() {
+return artifact;
+}
+
+public void setArtifact(String artifact) {
+this.artifact = artifact;
+}
+
+@ApiModelProperty("The version of the bundle that provides the 
referenced type.")
+public String getVersion() {
+return version;
+}
+
+public void setVersion(String version) {
+this.version = version;
+}
+
+@ApiModelProperty(
+value = "The fully-qualified class type",
+required = true,
+notes = "For example, 'org.apache.nifi.GetFile' or 
'org::apache:nifi::minifi::GetFile'")
--- End diff --

See above, it's really only intended to declare that a bundle contains an 
API definition, not the API definition itself. Presumable the interface type 
would reference a fully qualified class interface defined in a .h or .java file.


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175586309
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/extension/PropertyDescriptor.java
 ---
@@ -0,0 +1,123 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model.extension;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.List;
+
+@ApiModel
+public class PropertyDescriptor {
+
+private String name;
+private String displayName;
+private String description;
+private List allowableValues;
+private String defaultValue;
+private Boolean required;
+private Boolean sensitive;
+private Boolean supportsEl;
+private DefinedType identifiesControllerServiceApi;
--- End diff --

No, but I can see how the field name is misleading. The intention of this 
field is to indicate if the property value identifies a Controller Service API, 
and if so, which DefinedType that API is. This is used in NiFi to present a 
drop down menu of any ExtensionComponents that implement that API.

See the comment about which is related to this field.

If this API definition+implementation metadata complexity is not needed in 
this initial version of this workflow, or if it is too difficult to provide, we 
could leave it out and reintroduce it in a future version of C2 server.


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175573944
  
--- Diff: 
minifi-c2/minifi-c2-web-api/src/main/java/org/apache/nifi/minifi/c2/web/api/AgentClassResource.java
 ---
@@ -0,0 +1,132 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.web.api;
+
+import io.swagger.annotations.Api;
+import io.swagger.annotations.ApiOperation;
+import io.swagger.annotations.ApiParam;
+import org.apache.nifi.minifi.c2.model.AgentClass;
+import org.springframework.stereotype.Component;
+
+import javax.ws.rs.Consumes;
+import javax.ws.rs.DELETE;
+import javax.ws.rs.GET;
+import javax.ws.rs.NotSupportedException;
+import javax.ws.rs.PATCH;
+import javax.ws.rs.POST;
+import javax.ws.rs.PUT;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response;
+
+@Component
+@Path("/agent-classes")
+@Api(value = "Agent Classes", description = "Register and manage agent 
class definitions")
+public class AgentClassResource {
+
+@POST
+@Consumes(MediaType.APPLICATION_JSON)
+@Produces(MediaType.APPLICATION_JSON)
+@ApiOperation(
+value = "Register a MiNiFi agent class with this C2 server",
+notes = "This can also be done with a heartbeat, which will 
register a MiNiFi agent class the first time it is seen in a heartbeat.",
+response = AgentClass.class
+)
+public Response createAgentClass(
+@ApiParam(value = "The class to create", required = true)
+AgentClass agentClass) {
+throw new NotSupportedException("This method is not yet 
implemented for this resource.");
+
+}
+
+@GET
+@Produces(MediaType.APPLICATION_JSON)
+@ApiOperation(
+value = "Get all MiNiFi agent classes that are registered with 
this C2 server",
+response = AgentClass.class,
+responseContainer = "List"
+)
+public Response getAgentClasses() {
--- End diff --

- [ ] Add ReadOnly NiFi Registry URL, Bucket, and Flow ID metadata when 
retrieving Agent classes (so that the flow designer knows where to read/save a 
flow)


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r17010
  
--- Diff: 
minifi-c2/minifi-c2-web-api/src/main/java/org/apache/nifi/minifi/c2/web/api/C2ProtocolResource.java
 ---
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.web.api;
+
+import io.swagger.annotations.Api;
+import io.swagger.annotations.ApiOperation;
+import io.swagger.annotations.ApiParam;
+import io.swagger.annotations.ApiResponse;
+import io.swagger.annotations.ApiResponses;
+import org.apache.nifi.minifi.c2.model.C2Heartbeat;
+import org.apache.nifi.minifi.c2.model.C2HeartbeatResponse;
+import org.apache.nifi.minifi.c2.model.C2OperationAck;
+import org.springframework.stereotype.Component;
+
+import javax.ws.rs.Consumes;
+import javax.ws.rs.POST;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response;
+
+@Component
+@Path("/c2-protocol")
--- End diff --

The internal architecture of the C2 Server will be based on these layered:

Web/Network Protocols (REST, MQTT, etc)
Services
Providers (e.g., database or persistence providers)

The bulk of the C2 Protocol implementation logic will live in the service 
layer, which will expose Java APIs for the Web protocol layers to interact 
will. The Web protocol implementations will provide very minimal/lightweight 
adapters, such as deserializing whatever wire-format they use into the 
in-memory objects that are required for calling the service layer API.  
External clients, such as MiNiFi agents, will only interact with the web 
protocols we choose to define, expose, and support. Everything else at the 
service level and below is internal to the C2 server implementation.


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175548117
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/extension/ExtensionComponent.java
 ---
@@ -0,0 +1,85 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model.extension;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.List;
+import java.util.Set;
+
+/**
+ * A component provided by an extension bundle
+ */
+@ApiModel
+public class ExtensionComponent extends DefinedType {
+
+// TODO, does arch/binary/compiler metadata need to be added here?
--- End diff --

Noted. I'll add it. What do you think are the required/optional fields?


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175534416
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/extension/Bundle.java
 ---
@@ -0,0 +1,106 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model.extension;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.Objects;
+
+@ApiModel
+public class Bundle {
+
+public static final String DEFAULT_GROUP = "default";
+public static final String DEFAULT_ARTIFACT = "unknown";
+public static final String DEFAULT_VERSION = "unversioned";
+
+private String group;
+private String artifact;
+private String version;
+// TODO md5sum?
+private BundleManifest bundleManifest;
+
+public Bundle() {}
+
+public Bundle(String group, String artifact, String version) {
+this.group = group;
+this.artifact = artifact;
+this.version = version;
+}
+
+public static Bundle defaultBundle() {
+return new Bundle(DEFAULT_GROUP, DEFAULT_ARTIFACT, 
DEFAULT_VERSION);
+}
+
+@ApiModelProperty(
+value = "The group id of the bundle",
+notes = "A globally unique group namespace, e.g., 
org.apache.nifi",
+required = true)
+public String getGroup() {
+return group;
+}
+
+public void setGroup(String group) {
+this.group = group;
+}
+
+@ApiModelProperty(
+value = "The artifact id of the bundle",
+notes = "Unique within the group",
+required = true)
+public String getArtifact() {
+return artifact;
+}
+
+public void setArtifact(String artifact) {
+this.artifact = artifact;
+}
+
+@ApiModelProperty("The version of the bundle artifact")
+public String getVersion() {
+return version;
+}
+
+public void setVersion(String version) {
+this.version = version;
+}
+
+@ApiModelProperty(value = "The full specification of the bundle 
contents",
+notes = "This is optional, as the group, artifact, and version 
are " +
+"also enough to reference a bundle in the case the 
bundle " +
+"specification has been published to a registry.")
+public BundleManifest getBundleManifest() {
+return bundleManifest;
+}
+
+public void setBundleManifest(BundleManifest bundleManifest) {
+this.bundleManifest = bundleManifest;
+}
+
+@Override
+public boolean equals(Object o) {
+if (this == o) return true;
--- End diff --

- [ ] Change equals method implementations to utilize commons-lang equals 
builder.


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175531893
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/OperationRequest.java
 ---
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+@ApiModel
+public class OperationRequest {
--- End diff --

This is C2 server specific for now. I wanted to try to formulate a basic 
mechanism that someone could manually submit an operation request to the C2 
server using it's rest api, which would then be queued in the C2 Server and 
eventually sent to the targeted MiNiFi agent over the C2 protocol (e.g., in a 
heartbeat response). I don't think there is a need to push this into the C2 
protocol yet, but I could see how that would be useful eventually, so if you 
have an idea for how to do that I'd be happy to implement support in the C2 
server.


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175529769
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/Device.java
 ---
@@ -0,0 +1,48 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+public class Device extends DeviceInfo {
+
+private String name;
+private String firstSeen;
--- End diff --

Yes, they should be timestamps. I meant to ask this question, so I'll just 
ask this here: what should the preferred JSON format of timestamps be for the 
C2 server REST API? We can go with Epoch time or another standard 
representation such as [RFC 3339](https://tools.ietf.org/html/rfc3339), which 
is a stricter version of ISO 8601, e.g., 2018-03-19T17:51:00+00:00. If Epoch 
time, this field would change to a long of course, but should the units be 
seconds or milliseconds since epoch?


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175519408
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/AgentStatus.java
 ---
@@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.Map;
+
+/**
+ * Status of the aspects of the agent, including any agent components that 
are controllable by the C2 server, ie:
+ *   - Repositories that can be cleared and their current state
+ */
+@ApiModel
+public class AgentStatus {
+
+private long uptime;
+private Map repositories;
--- End diff --

No good reason. You're right, the repos are fixed. Good catch!

- [ ] Replace map for AgentRepositoryStatus with fields for each specific 
repo.


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175517256
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/AgentRepositoryStatus.java
 ---
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+@ApiModel
+public class AgentRepositoryStatus {
+
+private Long size;
+private Long sizeMax;
+private Long count;
+private Long countMax;
--- End diff --

I'm not tied to the names of these fields. I agree that storageMax is 
better than sizeMax and that 'size' in general is ambiguous without reading the 
description. Better to avoid confusion.

I want to make this change so that the names stay consistent with the 
FlowQueueStatus fields if possible. My understanding (correct me if this is 
wrong) is that both queues and repositories have two important metrics: 

1. number of items they currently contain
2. the data/storage footprint of those items

And that both of these metrics can number configurable caps (i.e., the "max 
value") for both queues and repositories.

If that understanding is not correct, let me know as I'll need to correct 
these model classes. If it is correct, what would be the best names to use that 
would be consistent with how these values are labeled in MiNiFi's configuration?



---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175511730
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/AgentRepositoryStatus.java
 ---
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+@ApiModel
+public class AgentRepositoryStatus {
+
+private Long size;
--- End diff --

Excellent point. Will review this and update

- [ ] For numeric primitive types, should document how it will be 
interpreted so that serialized values are valid and interpreted correctly when 
deserialized by the Java C2 (e.g., signed vs unsigned)


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175509670
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/AgentManifest.java
 ---
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+import org.apache.nifi.minifi.c2.model.extension.Bundle;
+
+import java.util.List;
+
+@ApiModel
+public class AgentManifest {
+
+private String identifier;
+private String agentType;
+private String version;
+private BuildInfo buildInfo;
+private List bundles;
+
+@ApiModelProperty(
+value = "A unique identifier for the manifest",
+readOnly = true)
+public String getIdentifier() {
+return identifier;
+}
+
+public void setIdentifier(String identifier) {
+this.identifier = identifier;
+}
+
+@ApiModelProperty(
+value = "The type of the agent binary, e.g., 'minifi-java' or 
'minifi-cpp'",
+notes = "Usually set when the agent is built.")
+public String getAgentType() {
+return agentType;
+}
+
+public void setAgentType(String agentType) {
+this.agentType = agentType;
+}
+
+@ApiModelProperty(
+value = "The version of the agent binary, e.g., '1.0.1'",
+notes = "Usually set when the agent is built.")
+public String getVersion() {
+return version;
+}
+
+public void setVersion(String version) {
+this.version = version;
+}
+
+@ApiModelProperty("Build summary for this agent binary")
+public BuildInfo getBuildInfo() {
+return buildInfo;
+}
+
+public void setBuildInfo(BuildInfo buildInfo) {
+this.buildInfo = buildInfo;
+}
+
+@ApiModelProperty("All extension bundles included with this agent")
+public List getBundles() {
+return bundles;
--- End diff --

Yes, will do.


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175509585
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/AgentInfo.java
 ---
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.Collection;
+
+@ApiModel
+public class AgentInfo {
+
+private String identifier;
+// TODO, do we also need identity. e.g., cert DN
+private String agentClass;
--- End diff --

_[Some background to those who are new to the terminology that's starting 
to formulate around these features: In this context, class != OO type. Rather, 
it's intended as a category/purpose label for a MiNiFi agent that could be used 
for a variety of purposes from a C2 server: e.g., seeing summary views of a 
class, showing all agents of a particular class, scheduling a bulk operation 
(such as deploying flow update) to all agents that match a particular class.]_

In the data model, I do have an AgentClass type that could be used here, 
but I think that will really just be metadata that one can associate with a 
class in the C2 server. We'll have to enforce agent class name uniqueness, so 
that in contexts where the full agent class metadata is not needed, just the 
class name can be used.


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175506629
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/AgentInfo.java
 ---
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.Collection;
+
+@ApiModel
+public class AgentInfo {
+
+private String identifier;
+// TODO, do we also need identity. e.g., cert DN
+private String agentClass;
+private Collection tags;
--- End diff --

I was thinking it would be nice for an operator to be able to tag agents 
with arbitrary labels that they can then use as filters in future queries. I'm 
not sure those tags would be just on the C2 server in the agents it is 
tracking, or if they could also be set on agents when they are provisioned and 
deployed, similar to the agent class labels. 

Thanks for calling this out. Now that I think about it, I'd like to drop 
this field for now so that we can properly discuss (on the mailing list, JIRA, 
etc) if this is valuable and if so  how to implement it.


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175503881
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/AgentManifest.java
 ---
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+import org.apache.nifi.minifi.c2.model.extension.Bundle;
+
+import java.util.List;
+
+@ApiModel
+public class AgentManifest {
+
+private String identifier;
--- End diff --

No, this is a new identifier that will only be used by the C2 server. It 
assumes the agent manifests are going to be persistable and reusable. It 
probably would not be present when an agent manifest is being uploaded to a C2 
server (e.g., in a C2 heartbeat from a MiNiFi agent or as the output of a 
MiNiFi agent build that some build script uploads), but when the C2 server 
persists it, it will serve as a unique ID that can be used to retrieve that 
agent manifest again.

The primary use case would be when using a Flow Designer in NiFi, the user 
chooses an agent class name from a list. That selection then returns one or 
more agent manifests that are associated with that class name. So the way I am 
picturing it the mapping of agent to agent manifest depends on context: from 
the perspecitive of an agent instance, it's going to be 1-to-1 where the 
manifest is always what it currently consists of. from the c2 server's 
perspective, it will be 1-to-many, where it is keeping track of all manifests 
for a given class name based on what has been uploaded to it.

These are still very new concepts, so if you have an input to help 
formulate them that would be welcome! This is just an initial idea and I tried 
to represent it the way that will support this use case. I'm sure it will 
change a bit before the minifi-c2-server branch could be considered "done" and 
merged to master.


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175459214
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/AgentStatus.java
 ---
@@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.Map;
+
+/**
+ * Status of the aspects of the agent, including any agent components that 
are controllable by the C2 server, ie:
+ *   - Repositories that can be cleared and their current state
+ */
+@ApiModel
+public class AgentStatus {
+
+private long uptime;
+private Map repositories;
+
+@ApiModelProperty("The number of milliseconds since the agent 
started.")
+public long getUptime() {
+return uptime;
+}
+
+public void setUptime(long uptime) {
+this.uptime = uptime;
+}
+
+@ApiModelProperty("Status and metrics for each repository")
+public Map getRepositories() {
+return repositories;
--- End diff --

Yep will do.


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

Github user kevdoran commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175459177
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/AgentInfo.java
 ---
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.Collection;
+
+@ApiModel
+public class AgentInfo {
+
+private String identifier;
+// TODO, do we also need identity. e.g., cert DN
+private String agentClass;
+private Collection tags;
+private AgentManifest agentManifest;
+private AgentStatus status;
+// private Map configProperties;  // TODO we should 
add this information eventually, but we need to handle the best way to handle 
sharing sensitive properties.
+
+@ApiModelProperty(
+value = "A unique identifier for the Agent",
+notes = "Usually set when the agent is provisioned and 
deployed")
+public String getIdentifier() {
+return identifier;
+}
+
+public void setIdentifier(String identifier) {
+this.identifier = identifier;
+}
+
+@ApiModelProperty(
+value = "The class or category label of the agent, e.g., 
'sensor-collector'",
+notes = "Usually set when the agent is provisioned and 
deployed")
+public String getAgentClass() {
+return agentClass;
+}
+
+public void setAgentClass(String agentClass) {
+this.agentClass = agentClass;
+}
+
+@ApiModelProperty("Arbitrary tags that describe the agent in addition 
to the class")
+public Collection getTags() {
+return tags;
--- End diff --

Good idea, will do. I'll review all getters to see which ones should return 
immutable collections.


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175205415
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/OperationRequest.java
 ---
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+@ApiModel
+public class OperationRequest {
--- End diff --

This looks to be C2 server specific, is that true? If there is the need to 
push this further to a quorum of agents we will need a source identifier and 
then what's referred to as a request path. 


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175203879
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/C2Operation.java
 ---
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import javax.validation.constraints.NotBlank;
+import java.util.Map;
+
+@ApiModel
+public class C2Operation {
+
+private String identifier;
--- End diff --

The cpp agent has a nested operations. we will likely need to keep that. 


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175204853
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/FlowStatus.java
 ---
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.Map;
+
+/**
+ * Status of the aspects of the flow that are controllable by the C2 
server, ie:
+ *   - Processors that can be started/stopped and their current state
+ *   - Queues that can be cleared and their current state
+ */
+@ApiModel
+public class FlowStatus {
--- End diff --

Do we intent to have an analog for non flow component statuses? 


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175206733
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/extension/ExtensionComponent.java
 ---
@@ -0,0 +1,85 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model.extension;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.List;
+import java.util.Set;
+
+/**
+ * A component provided by an extension bundle
+ */
+@ApiModel
+public class ExtensionComponent extends DefinedType {
+
+// TODO, does arch/binary/compiler metadata need to be added here?
--- End diff --

I'm afraid if we don't include this we will likely not have deployable C++ 
agents across platforms. 


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175207121
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/extension/ExtensionComponent.java
 ---
@@ -0,0 +1,85 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model.extension;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.List;
+import java.util.Set;
+
+/**
+ * A component provided by an extension bundle
+ */
+@ApiModel
+public class ExtensionComponent extends DefinedType {
+
+// TODO, does arch/binary/compiler metadata need to be added here?
+
+private List providedApiImplementations;
+
+private String description;
+private Set tags;
+
+private Boolean deprecated;
+private String deprecationReason;
+
+@ApiModelProperty("If this type represents a provider for an 
interface, this lists the APIs it implements")
+public List getProvidedApiImplementations() {
+return providedApiImplementations;
+}
+
+public void setProvidedApiImplementations(List 
providedApiImplementations) {
+this.providedApiImplementations = providedApiImplementations;
--- End diff --

How is this "API" implementation used? What if api implementations change 
will we need to require that an agreed upon versioning is used ? Is this a 
public API? 


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175208131
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/C2OperationAck.java
 ---
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+@ApiModel
+public class C2OperationAck {
+
+private String operationId;
+// TODO, result, outcome, or status of the operation? eg SUCCESS/FAIL?
--- End diff --

yeah agreed. probably need something to provide more information. 


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175200116
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/AgentInfo.java
 ---
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.Collection;
+
+@ApiModel
+public class AgentInfo {
+
+private String identifier;
+// TODO, do we also need identity. e.g., cert DN
+private String agentClass;
+private Collection tags;
--- End diff --

What are tags?


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175207806
  
--- Diff: 
minifi-c2/minifi-c2-web-api/src/main/java/org/apache/nifi/minifi/c2/web/api/C2ProtocolResource.java
 ---
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.web.api;
+
+import io.swagger.annotations.Api;
+import io.swagger.annotations.ApiOperation;
+import io.swagger.annotations.ApiParam;
+import io.swagger.annotations.ApiResponse;
+import io.swagger.annotations.ApiResponses;
+import org.apache.nifi.minifi.c2.model.C2Heartbeat;
+import org.apache.nifi.minifi.c2.model.C2HeartbeatResponse;
+import org.apache.nifi.minifi.c2.model.C2OperationAck;
+import org.springframework.stereotype.Component;
+
+import javax.ws.rs.Consumes;
+import javax.ws.rs.POST;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response;
+
+@Component
+@Path("/c2-protocol")
--- End diff --

Is there going to be a transformation layer for different protocol types? 
Will that imply live in the protocol implementation? 


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175208409
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/extension/DefinedType.java
 ---
@@ -0,0 +1,69 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model.extension;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+/**
+ * A reference to a defined type identified by bundle and fully qualified 
class type identifiers
+ */
+@ApiModel
+public class DefinedType {
+
+private String group;
+private String artifact;
+private String version;
+private String type;
+
+@ApiModelProperty("The group name of the bundle that provides the 
referenced type.")
+public String getGroup() {
+return group;
+}
+
+public void setGroup(String group) {
+this.group = group;
+}
+
+@ApiModelProperty("The artifact name of the bundle that provides the 
referenced type.")
+public String getArtifact() {
+return artifact;
+}
+
+public void setArtifact(String artifact) {
+this.artifact = artifact;
+}
+
+@ApiModelProperty("The version of the bundle that provides the 
referenced type.")
+public String getVersion() {
+return version;
+}
+
+public void setVersion(String version) {
+this.version = version;
+}
+
+@ApiModelProperty(
+value = "The fully-qualified class type",
+required = true,
+notes = "For example, 'org.apache.nifi.GetFile' or 
'org::apache:nifi::minifi::GetFile'")
--- End diff --

How will this be used to define an API? 


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175207453
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/extension/PropertyDescriptor.java
 ---
@@ -0,0 +1,123 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model.extension;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.List;
+
+@ApiModel
+public class PropertyDescriptor {
+
+private String name;
+private String displayName;
+private String description;
+private List allowableValues;
+private String defaultValue;
+private Boolean required;
+private Boolean sensitive;
+private Boolean supportsEl;
+private DefinedType identifiesControllerServiceApi;
--- End diff --

Should this be a boolean? 


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175200073
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/AgentInfo.java
 ---
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.Collection;
+
+@ApiModel
+public class AgentInfo {
+
+private String identifier;
+// TODO, do we also need identity. e.g., cert DN
+private String agentClass;
--- End diff --

Any reason why this is not more strongly typed? 


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175205789
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/extension/Bundle.java
 ---
@@ -0,0 +1,106 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model.extension;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.Objects;
+
+@ApiModel
+public class Bundle {
+
+public static final String DEFAULT_GROUP = "default";
+public static final String DEFAULT_ARTIFACT = "unknown";
+public static final String DEFAULT_VERSION = "unversioned";
+
+private String group;
+private String artifact;
+private String version;
+// TODO md5sum?
+private BundleManifest bundleManifest;
+
+public Bundle() {}
+
+public Bundle(String group, String artifact, String version) {
+this.group = group;
+this.artifact = artifact;
+this.version = version;
+}
+
+public static Bundle defaultBundle() {
+return new Bundle(DEFAULT_GROUP, DEFAULT_ARTIFACT, 
DEFAULT_VERSION);
+}
+
+@ApiModelProperty(
+value = "The group id of the bundle",
+notes = "A globally unique group namespace, e.g., 
org.apache.nifi",
+required = true)
+public String getGroup() {
+return group;
+}
+
+public void setGroup(String group) {
+this.group = group;
+}
+
+@ApiModelProperty(
+value = "The artifact id of the bundle",
+notes = "Unique within the group",
+required = true)
+public String getArtifact() {
+return artifact;
+}
+
+public void setArtifact(String artifact) {
+this.artifact = artifact;
+}
+
+@ApiModelProperty("The version of the bundle artifact")
+public String getVersion() {
+return version;
+}
+
+public void setVersion(String version) {
+this.version = version;
+}
+
+@ApiModelProperty(value = "The full specification of the bundle 
contents",
+notes = "This is optional, as the group, artifact, and version 
are " +
+"also enough to reference a bundle in the case the 
bundle " +
+"specification has been published to a registry.")
+public BundleManifest getBundleManifest() {
+return bundleManifest;
+}
+
+public void setBundleManifest(BundleManifest bundleManifest) {
+this.bundleManifest = bundleManifest;
+}
+
+@Override
+public boolean equals(Object o) {
+if (this == o) return true;
--- End diff --

Equals builder can make this a little cleaner. I see that commons-lang is 
in the root pom. 


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175201509
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/AgentRepositoryStatus.java
 ---
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+@ApiModel
+public class AgentRepositoryStatus {
+
+private Long size;
--- End diff --

keep in mind that java is a 64-bit two's complement so you could get values 
here that exceed the max. Should either comment re this or use Long's unsigned 
comparator methods. 


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175202125
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/AgentStatus.java
 ---
@@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.Map;
+
+/**
+ * Status of the aspects of the agent, including any agent components that 
are controllable by the C2 server, ie:
+ *   - Repositories that can be cleared and their current state
+ */
+@ApiModel
+public class AgentStatus {
+
+private long uptime;
+private Map repositories;
--- End diff --

why a map versus calling out the specific repos?


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175206927
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/extension/DefinedType.java
 ---
@@ -0,0 +1,69 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model.extension;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+/**
+ * A reference to a defined type identified by bundle and fully qualified 
class type identifiers
+ */
+@ApiModel
+public class DefinedType {
+
+private String group;
+private String artifact;
+private String version;
+private String type;
--- End diff --

A defined type is effectively a version for a class that's deployable or 
any class in the software stack? 


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175201914
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/AgentRepositoryStatus.java
 ---
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+@ApiModel
+public class AgentRepositoryStatus {
+
+private Long size;
+private Long sizeMax;
+private Long count;
+private Long countMax;
--- End diff --

Count max seems confusing. I imagine it's not the max count seen. Realize 
you set a description in the model prop annotation, but you probably should 
tend toward a different name. Similarly sizemax and countmax seem to have 
overlap. Would storageMax and  size max better reflect their intent? 


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175202384
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/AgentInfo.java
 ---
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.Collection;
+
+@ApiModel
+public class AgentInfo {
+
+private String identifier;
+// TODO, do we also need identity. e.g., cert DN
+private String agentClass;
+private Collection tags;
+private AgentManifest agentManifest;
+private AgentStatus status;
+// private Map configProperties;  // TODO we should 
add this information eventually, but we need to handle the best way to handle 
sharing sensitive properties.
+
+@ApiModelProperty(
+value = "A unique identifier for the Agent",
+notes = "Usually set when the agent is provisioned and 
deployed")
+public String getIdentifier() {
+return identifier;
+}
+
+public void setIdentifier(String identifier) {
+this.identifier = identifier;
+}
+
+@ApiModelProperty(
+value = "The class or category label of the agent, e.g., 
'sensor-collector'",
+notes = "Usually set when the agent is provisioned and 
deployed")
+public String getAgentClass() {
+return agentClass;
+}
+
+public void setAgentClass(String agentClass) {
+this.agentClass = agentClass;
+}
+
+@ApiModelProperty("Arbitrary tags that describe the agent in addition 
to the class")
+public Collection getTags() {
+return tags;
--- End diff --

can this be an immutable collection ? Preferably a set since I imagine 
similar tags have no real meaning?


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175202203
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/AgentManifest.java
 ---
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+import org.apache.nifi.minifi.c2.model.extension.Bundle;
+
+import java.util.List;
+
+@ApiModel
+public class AgentManifest {
+
+private String identifier;
+private String agentType;
+private String version;
+private BuildInfo buildInfo;
+private List bundles;
+
+@ApiModelProperty(
+value = "A unique identifier for the manifest",
+readOnly = true)
+public String getIdentifier() {
+return identifier;
+}
+
+public void setIdentifier(String identifier) {
+this.identifier = identifier;
+}
+
+@ApiModelProperty(
+value = "The type of the agent binary, e.g., 'minifi-java' or 
'minifi-cpp'",
+notes = "Usually set when the agent is built.")
+public String getAgentType() {
+return agentType;
+}
+
+public void setAgentType(String agentType) {
+this.agentType = agentType;
+}
+
+@ApiModelProperty(
+value = "The version of the agent binary, e.g., '1.0.1'",
+notes = "Usually set when the agent is built.")
+public String getVersion() {
+return version;
+}
+
+public void setVersion(String version) {
+this.version = version;
+}
+
+@ApiModelProperty("Build summary for this agent binary")
+public BuildInfo getBuildInfo() {
+return buildInfo;
+}
+
+public void setBuildInfo(BuildInfo buildInfo) {
+this.buildInfo = buildInfo;
+}
+
+@ApiModelProperty("All extension bundles included with this agent")
+public List getBundles() {
+return bundles;
--- End diff --

can we return an immutable list here?


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175203025
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/C2Heartbeat.java
 ---
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.Map;
+
+@ApiModel
+public class C2Heartbeat {
+
+// TODO, timestamp?
+private DeviceInfo deviceInfo;
+private AgentInfo agentInfo;
+private FlowInfo flowInfo;
+private Map metrics;
--- End diff --

Metrics are currently typed, is there a need for bag of words? can we have 
that be metadata, imo that has a better meaning long term. 


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175204137
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/Device.java
 ---
@@ -0,0 +1,48 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+public class Device extends DeviceInfo {
+
+private String name;
+private String firstSeen;
--- End diff --

what do first seen and last seen mean in this context? I assume they are 
timestamps? 


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175202507
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/AgentStatus.java
 ---
@@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.Map;
+
+/**
+ * Status of the aspects of the agent, including any agent components that 
are controllable by the C2 server, ie:
+ *   - Repositories that can be cleared and their current state
+ */
+@ApiModel
+public class AgentStatus {
+
+private long uptime;
+private Map repositories;
+
+@ApiModelProperty("The number of milliseconds since the agent 
started.")
+public long getUptime() {
+return uptime;
+}
+
+public void setUptime(long uptime) {
+this.uptime = uptime;
+}
+
+@ApiModelProperty("Status and metrics for each repository")
+public Map getRepositories() {
+return repositories;
--- End diff --

immutable map? 


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175202823
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/BuildInfo.java
 ---
@@ -0,0 +1,77 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+@ApiModel
+public class BuildInfo {
--- End diff --

Is this an agent level class? The definition below makes me wonder if we're 
modeling extensions differently than agent information? That model doesn't seem 
to fit what we can/will need to actually do. 


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175205532
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/SystemInfo.java
 ---
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+@ApiModel
+public class SystemInfo {
+
+private String machineArch;
+private long physicalMem;
+private int vCores;
+// TODO timezone / UTC offset?
--- End diff --

configured timezone isn't a bad idea but this can change. 


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175204543
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/FlowQueueStatus.java
 ---
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+@ApiModel
+public class FlowQueueStatus {
+
+private Long size;
--- End diff --

same here regarding sizes. I think the queues are less likely to be an 
issue but if they are set to zero then they will be system long max.


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175200297
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/AgentManifest.java
 ---
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+import org.apache.nifi.minifi.c2.model.extension.Bundle;
+
+import java.util.List;
+
+@ApiModel
+public class AgentManifest {
+
+private String identifier;
--- End diff --

is this referential to the identifiers above?


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/118#discussion_r175206480
  
--- Diff: 
minifi-c2/minifi-c2-commons/src/main/java/org/apache/nifi/minifi/c2/model/extension/BundleManifest.java
 ---
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.model.extension;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+
+import java.util.List;
+
+@ApiModel
+public class BundleManifest {
+
--- End diff --

How would one define an extension outside of the confines of a manifest? 


---

[GitHub] nifi-minifi pull request #118: MINIFI-444 C2 Data Model and REST API

[kevdoran]

GitHub user kevdoran opened a pull request:

https://github.com/apache/nifi-minifi/pull/118

MINIFI-444 C2 Data Model and REST API

Adds the foundation of a data model and REST API that would support
discussed features such as a MiNiFi Agent C2 Protocol and
MiNiFi Flow Designer.

Thank you for submitting a contribution to Apache NiFi - MiNiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with MINIFI- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi-minifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under minifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under minifi-assembly?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kevdoran/nifi-minifi MINIFI-444-c2-data-model

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-minifi/pull/118.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #118


commit b765407b29a8dc8876cff340275f1937e28633c8
Author: Kevin Doran 
Date:   2018-02-27T19:09:50Z

MINIFI-444 C2 Data Model and REST API

The foundation of a data model and REST API that would support
discussed features such as a MiNiFi Agent C2 Protocol and
MiNiFi Flow Designer.




---