[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361196#comment-16361196
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-minifi-cpp/pull/191


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>Priority: Major
> Fix For: 0.5.0
>
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361000#comment-16361000
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user phrocker commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/191
  
@calebj I submitted a PR, though it has the full history. Did you ever 
create a benchmark. While @achristianson linked the benchmark's on github, I've 
seen slowdowns of 1-2% when parsing our payloads since they are significantly 
smaller. 


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>Priority: Major
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360833#comment-16360833
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user phrocker commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/191
  
@calebj I've rebased this PR so that it's up to date with master and have 
addressed some issues. I'll try to submit a PR against PR shortly after I 
finish some final testing. 


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>Priority: Major
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335977#comment-16335977
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user apiri commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/191
  
Hey @calebj @phrocker,

Looks like we have a bit of effort to wrap this up remaining so would like 
to slide this to 0.5.0.  Would like for 0.4.0 to be available so we can provide 
the appropriate tooling to interact with the RPG changes in NiFi 1.5.0.  


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>Priority: Major
> Fix For: 0.4.0
>
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329764#comment-16329764
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user calebj commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/191
  
@phrocker Honestly, I just reimplemented the JSON handling in RapidJSON as 
well as I could, and fixed the things that broke the build and tests. Help 
would be greatly appreciated.


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>Priority: Major
> Fix For: 0.4.0
>
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327659#comment-16327659
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user phrocker commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/191
  
@calebj HTTP Site to Site and C2 don't seem to work. Do you need help in 
testing these? 


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>Priority: Major
> Fix For: 0.4.0
>
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327510#comment-16327510
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user achristianson commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/191
  
Suggest if we are going to merge it, a good time would be the very start of 
0.5.0 so we have plenty of time to test and fix any issues that crop up by the 
time the next release comes around.


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>Priority: Major
> Fix For: 0.4.0
>
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16318543#comment-16318543
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user phrocker commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/191
  
I'm currently not a +1 on this PR. 


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
> Fix For: 0.4.0
>
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16316629#comment-16316629
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user achristianson commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/191
  
We should probably make a decision on this soon because this type of change 
can get out of date fairly easily and JSON touches a lot of things in the 
codebase.


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
> Fix For: 0.4.0
>
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16260273#comment-16260273
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user calebj commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r152182652
  
--- Diff: extensions/libarchive/UnfocusArchiveEntry.cpp ---
@@ -67,55 +66,41 @@ void 
UnfocusArchiveEntry::onTrigger(core::ProcessContext *context, core::Process
   }
 
   fileutils::FileManager file_man;
-
-  // Get lens stack from attribute
   ArchiveMetadata lensArchiveMetadata;
 
-  Json::Value lensStack;
-  Json::Reader reader;
-
-  std::string existingLensStack;
-
-  if (flowFile->getAttribute("lens.archive.stack", existingLensStack)) {
-logger_->log_info("UnfocusArchiveEntry loading existing lens context");
-
-// TODO(calebj) handle any exceptions that might arise from working 
with JSON data
-if (!reader.parse(existingLensStack, lensStack)) {
-  logger_->log_error("UnfocusArchiveEntry JSON parse error: %s", 
reader.getFormattedErrorMessages());
-  context->yield();
-  return;
+  // Get lens stack from attribute
+  {
+ArchiveStack archiveStack;
+{
+  std::string existingLensStack;
+
+  if (flowFile->getAttribute("lens.archive.stack", existingLensStack)) 
{
+logger_->log_info("FocusArchiveEntry loading existing lens 
context");
+
+try {
+  archiveStack.loadJsonString(existingLensStack);
--- End diff --

Done in 00282396e2b2fa3f0f450a7916d692b52c749a1d


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16260256#comment-16260256
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user calebj commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r152180661
  
--- Diff: libminifi/src/RemoteProcessorGroupPort.cpp ---
@@ -264,20 +264,27 @@ void 
RemoteProcessorGroupPort::refreshRemoteSite2SiteInfo() {
 if (!response_body.empty()) {
   std::string controller = std::string(response_body.begin(), 
response_body.end());
   logger_->log_debug("controller config %s", controller.c_str());
-  Json::Value value;
-  Json::Reader reader;
-  bool parsingSuccessful = reader.parse(controller, value);
-  if (parsingSuccessful && !value.empty()) {
-Json::Value controllerValue = value["controller"];
-if (!controllerValue.empty()) {
-  Json::Value port = controllerValue["remoteSiteListeningPort"];
-  if (client_type_ == sitetosite::CLIENT_TYPE::RAW && 
!port.empty())
-this->site2site_port_ = port.asInt();
+
--- End diff --

All I did here was translate the jsoncpp code to rapidjson. I didn't test 
it.


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257387#comment-16257387
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user achristianson commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/191
  
@calebj we're 100% golden on the licence.

We have:
- RapidJSON itself under MIT (Apache Category A)
- msinttypes r29 under BSD (Apache Category A)
- JSON_checker under JSON (Recently banned/put into to Apache Category X)

Upon closer inspection of the JSON_checker, the RapidJSON license says this:

"To avoid the problematic JSON license in your own projects, it's 
sufficient to exclude the bin/jsonchecker/ directory, as it's the only code 
under the JSON license."

So, strip out the bin/jsonchecker/ dir and we're fully in Category A 
territory.


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257117#comment-16257117
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user calebj commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/191
  
I know that it's MIT licensed, but their license.txt comes with others 
tacked on for subcomponents. If you think they're all compatible, I'll copy all 
of them in.


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257018#comment-16257018
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user achristianson commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/191
  
@calebj with regard to the license, I think @phrocker is just referring to 
adding RapidJSON's license text to our LICENSE file. If you look in there, for 
every project we include in our thirdparty/ dir, we mention that we did that 
and what the license of that project is.

RapidJSON is MIT license which is fully compatible. A good reference is 
https://www.apache.org/legal/resolved.html

So I think we're good if you copy their LICENSE 
(https://github.com/Tencent/rapidjson/blob/master/license.txt) into our LICENSE 
using the same format as the others.


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257006#comment-16257006
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user achristianson commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r151692781
  
--- Diff: extensions/http-curl/protocols/RESTProtocol.cpp ---
@@ -81,48 +93,117 @@ const C2Payload RESTProtocol::parseJsonResponse(const 
C2Payload , const
   return std::move(C2Payload(payload.getOperation(), 
state::UpdateState::READ_ERROR, true));
 }
 
-Json::Value RESTProtocol::serializeJsonPayload(Json::Value _root, 
const C2Payload ) {
-  // get the name from the content
-  Json::Value json_payload;
-  std::map children;
-  for (const auto _payload : payload.getNestedPayloads()) {
-Json::Value child_payload = serializeJsonPayload(json_payload, 
nested_payload);
-children[nested_payload.getLabel()].push_back(child_payload);
-  }
-  for (auto child_vector : children) {
-if (child_vector.second.size() > 1) {
-  Json::Value children_json(Json::arrayValue);
-  for (auto child : child_vector.second) {
-json_payload[child_vector.first] = child;
-  }
-} else {
-  if (child_vector.second.size() == 1) {
-if (child_vector.second.at(0).isMember(child_vector.first)) {
-  json_payload[child_vector.first] = 
child_vector.second.at(0)[child_vector.first];
-} else {
-  json_payload[child_vector.first] = child_vector.second.at(0);
-}
-  }
-}
-  }
+void setJsonStr(const std::string& key, const std::string& value, 
rapidjson::Value& parent, rapidjson::Document::AllocatorType& alloc) { // NOLINT
+  rapidjson::Value keyVal;
+  rapidjson::Value valueVal;
+  const char* c_key = key.c_str();
+  const char* c_val = value.c_str();
 
+  keyVal.SetString(c_key, key.length()), alloc;
+  valueVal.SetString(c_val, value.length(), alloc);
+
+  parent.AddMember(keyVal, valueVal, alloc);
+}
+
+rapidjson::Value getStringValue(const std::string& value, 
rapidjson::Document::AllocatorType& alloc) { // NOLINT
+  rapidjson::Value Val;
--- End diff --

IMO we should only use macros where really necessary, so the code can be 
clearer and easier to read.


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256966#comment-16256966
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user calebj commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/191
  
@achristianson something in the master branch is failing. Looks like 
something in the commit before the latest one, 
9b9c3354330d525cfdd50f656ae42fc3da80764c, broke it.

Regarding the LICENSE, there are a handful of them. IANAL, so can someone 
look over [rapidjson's 
license.txt](https://github.com/Tencent/rapidjson/blob/master/license.txt) and 
see what might have to be left out, if anything?


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256418#comment-16256418
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user achristianson commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/191
  
s2s integration tests for this are passing (after a few docker int test 
fixes --see other PRs).


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256307#comment-16256307
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user achristianson commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/191
  
Looks like a test failure in the travis xcode:

```
The following tests FAILED:
 38 - CompressContentTests (Failed)
```


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256261#comment-16256261
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user calebj commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r151581577
  
--- Diff: extensions/http-curl/protocols/RESTReceiver.cpp ---
@@ -65,38 +65,8 @@ void RESTReceiver::initialize(const 
std::shared_ptr Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256256#comment-16256256
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user calebj commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r151581091
  
--- Diff: extensions/http-curl/protocols/RESTProtocol.cpp ---
@@ -81,48 +93,117 @@ const C2Payload RESTProtocol::parseJsonResponse(const 
C2Payload , const
   return std::move(C2Payload(payload.getOperation(), 
state::UpdateState::READ_ERROR, true));
 }
 
-Json::Value RESTProtocol::serializeJsonPayload(Json::Value _root, 
const C2Payload ) {
-  // get the name from the content
-  Json::Value json_payload;
-  std::map children;
-  for (const auto _payload : payload.getNestedPayloads()) {
-Json::Value child_payload = serializeJsonPayload(json_payload, 
nested_payload);
-children[nested_payload.getLabel()].push_back(child_payload);
-  }
-  for (auto child_vector : children) {
-if (child_vector.second.size() > 1) {
-  Json::Value children_json(Json::arrayValue);
-  for (auto child : child_vector.second) {
-json_payload[child_vector.first] = child;
-  }
-} else {
-  if (child_vector.second.size() == 1) {
-if (child_vector.second.at(0).isMember(child_vector.first)) {
-  json_payload[child_vector.first] = 
child_vector.second.at(0)[child_vector.first];
-} else {
-  json_payload[child_vector.first] = child_vector.second.at(0);
-}
-  }
-}
-  }
+void setJsonStr(const std::string& key, const std::string& value, 
rapidjson::Value& parent, rapidjson::Document::AllocatorType& alloc) { // NOLINT
--- End diff --

The linter wants a constant ref passed, but it doesn't compile due to 
rapidjson's semantics.


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256250#comment-16256250
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user calebj commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r151580773
  
--- Diff: extensions/http-curl/protocols/RESTProtocol.cpp ---
@@ -81,48 +93,117 @@ const C2Payload RESTProtocol::parseJsonResponse(const 
C2Payload , const
   return std::move(C2Payload(payload.getOperation(), 
state::UpdateState::READ_ERROR, true));
 }
 
-Json::Value RESTProtocol::serializeJsonPayload(Json::Value _root, 
const C2Payload ) {
-  // get the name from the content
-  Json::Value json_payload;
-  std::map children;
-  for (const auto _payload : payload.getNestedPayloads()) {
-Json::Value child_payload = serializeJsonPayload(json_payload, 
nested_payload);
-children[nested_payload.getLabel()].push_back(child_payload);
-  }
-  for (auto child_vector : children) {
-if (child_vector.second.size() > 1) {
-  Json::Value children_json(Json::arrayValue);
-  for (auto child : child_vector.second) {
-json_payload[child_vector.first] = child;
-  }
-} else {
-  if (child_vector.second.size() == 1) {
-if (child_vector.second.at(0).isMember(child_vector.first)) {
-  json_payload[child_vector.first] = 
child_vector.second.at(0)[child_vector.first];
-} else {
-  json_payload[child_vector.first] = child_vector.second.at(0);
-}
-  }
-}
-  }
+void setJsonStr(const std::string& key, const std::string& value, 
rapidjson::Value& parent, rapidjson::Document::AllocatorType& alloc) { // NOLINT
+  rapidjson::Value keyVal;
+  rapidjson::Value valueVal;
+  const char* c_key = key.c_str();
+  const char* c_val = value.c_str();
 
+  keyVal.SetString(c_key, key.length()), alloc;
+  valueVal.SetString(c_val, value.length(), alloc);
+
+  parent.AddMember(keyVal, valueVal, alloc);
+}
+
+rapidjson::Value getStringValue(const std::string& value, 
rapidjson::Document::AllocatorType& alloc) { // NOLINT
+  rapidjson::Value Val;
+  Val.SetString(value.c_str(), value.length(), alloc);
+  return Val;
+}
+
+void RESTProtocol::mergePayloadContent(rapidjson::Value , const 
C2Payload , rapidjson::Document::AllocatorType ) {
   const std::vector  = payload.getContent();
+
   for (const auto _content : content) {
-Json::Value payload_content_values;
+rapidjson::Value payload_content_values(rapidjson::kObjectType);
 bool use_sub_option = true;
+
 if (payload_content.op == payload.getOperation()) {
   for (auto content : payload_content.operation_arguments) {
 if (payload_content.operation_arguments.size() == 1 && 
payload_content.name == content.first) {
-  json_payload[payload_content.name] = content.second;
+  setJsonStr(payload_content.name, content.second, target, alloc);
   use_sub_option = false;
 } else {
-  payload_content_values[content.first] = content.second;
+  setJsonStr(content.first, content.second, 
payload_content_values, alloc);
 }
   }
 }
-if (use_sub_option)
-  json_payload[payload_content.name] = payload_content_values;
+if (use_sub_option) {
+  rapidjson::Value sub_key = getStringValue(payload_content.name, 
alloc);
+  target.AddMember(sub_key, payload_content_values, alloc);
+}
+  }
+}
+
+std::string RESTProtocol::serializeJsonRootPayload(const C2Payload& 
payload) {
+  rapidjson::Document json_payload(rapidjson::kObjectType);
+  rapidjson::Document::AllocatorType  = json_payload.GetAllocator();
--- End diff --

I believe it's supposed to be per-document.


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256248#comment-16256248
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user calebj commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r151580663
  
--- Diff: 
libminifi/src/core/reporting/SiteToSiteProvenanceReportingTask.cpp ---
@@ -53,54 +56,92 @@ void SiteToSiteProvenanceReportingTask::initialize() {
   RemoteProcessorGroupPort::initialize();
 }
 
+void setJsonStr(const std::string& key, const std::string& value, 
rapidjson::Value& parent, rapidjson::Document::AllocatorType& alloc) { // NOLINT
--- End diff --

It's just a shortcut wrapper for adding members from strings. I ran into a 
lot of issues with strings being read as garbage. I'll move them to a central 
location.


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256245#comment-16256245
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user calebj commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r151580153
  
--- Diff: extensions/http-curl/protocols/RESTProtocol.cpp ---
@@ -81,48 +93,117 @@ const C2Payload RESTProtocol::parseJsonResponse(const 
C2Payload , const
   return std::move(C2Payload(payload.getOperation(), 
state::UpdateState::READ_ERROR, true));
 }
 
-Json::Value RESTProtocol::serializeJsonPayload(Json::Value _root, 
const C2Payload ) {
-  // get the name from the content
-  Json::Value json_payload;
-  std::map children;
-  for (const auto _payload : payload.getNestedPayloads()) {
-Json::Value child_payload = serializeJsonPayload(json_payload, 
nested_payload);
-children[nested_payload.getLabel()].push_back(child_payload);
-  }
-  for (auto child_vector : children) {
-if (child_vector.second.size() > 1) {
-  Json::Value children_json(Json::arrayValue);
-  for (auto child : child_vector.second) {
-json_payload[child_vector.first] = child;
-  }
-} else {
-  if (child_vector.second.size() == 1) {
-if (child_vector.second.at(0).isMember(child_vector.first)) {
-  json_payload[child_vector.first] = 
child_vector.second.at(0)[child_vector.first];
-} else {
-  json_payload[child_vector.first] = child_vector.second.at(0);
-}
-  }
-}
-  }
+void setJsonStr(const std::string& key, const std::string& value, 
rapidjson::Value& parent, rapidjson::Document::AllocatorType& alloc) { // NOLINT
+  rapidjson::Value keyVal;
+  rapidjson::Value valueVal;
+  const char* c_key = key.c_str();
+  const char* c_val = value.c_str();
 
+  keyVal.SetString(c_key, key.length()), alloc;
+  valueVal.SetString(c_val, value.length(), alloc);
+
+  parent.AddMember(keyVal, valueVal, alloc);
+}
+
+rapidjson::Value getStringValue(const std::string& value, 
rapidjson::Document::AllocatorType& alloc) { // NOLINT
+  rapidjson::Value Val;
+  Val.SetString(value.c_str(), value.length(), alloc);
+  return Val;
+}
+
+void RESTProtocol::mergePayloadContent(rapidjson::Value , const 
C2Payload , rapidjson::Document::AllocatorType ) {
   const std::vector  = payload.getContent();
+
   for (const auto _content : content) {
-Json::Value payload_content_values;
+rapidjson::Value payload_content_values(rapidjson::kObjectType);
 bool use_sub_option = true;
+
 if (payload_content.op == payload.getOperation()) {
   for (auto content : payload_content.operation_arguments) {
 if (payload_content.operation_arguments.size() == 1 && 
payload_content.name == content.first) {
-  json_payload[payload_content.name] = content.second;
+  setJsonStr(payload_content.name, content.second, target, alloc);
   use_sub_option = false;
 } else {
-  payload_content_values[content.first] = content.second;
+  setJsonStr(content.first, content.second, 
payload_content_values, alloc);
 }
   }
 }
-if (use_sub_option)
-  json_payload[payload_content.name] = payload_content_values;
+if (use_sub_option) {
+  rapidjson::Value sub_key = getStringValue(payload_content.name, 
alloc);
+  target.AddMember(sub_key, payload_content_values, alloc);
+}
+  }
+}
+
+std::string RESTProtocol::serializeJsonRootPayload(const C2Payload& 
payload) {
+  rapidjson::Document json_payload(rapidjson::kObjectType);
+  rapidjson::Document::AllocatorType  = json_payload.GetAllocator();
+
+  rapidjson::Value opReqStrVal;
+  std::string operation_request_str = getOperation(payload);
+  opReqStrVal.SetString(operation_request_str.c_str(), 
operation_request_str.length(), alloc);
+  json_payload.AddMember("operation", opReqStrVal, alloc);
+
+  std::string operationid = payload.getIdentifier();
+  if (operationid.length() > 0) {
+rapidjson::Value operationIdVal = getStringValue(operationid, alloc);
+json_payload.AddMember("operationid", operationIdVal, alloc);
+  }
+
+  mergePayloadContent(json_payload, payload, alloc);
+
+  for (const auto _payload : payload.getNestedPayloads()) {
+rapidjson::Value np_key = getStringValue(nested_payload.getLabel(), 
alloc);
+rapidjson::Value np_value = serializeJsonPayload(nested_payload, 
alloc);
+json_payload.AddMember(np_key, np_value, alloc);
+  }
+
+  

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256243#comment-16256243
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user achristianson commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r151580011
  
--- Diff: extensions/rocksdb-repos/CMakeLists.txt ---
@@ -23,7 +23,7 @@ cmake_minimum_required(VERSION 2.6)
 set(CMAKE_EXE_LINKER_FLAGS "-Wl,--export-all-symbols")
 set(CMAKE_SHARED_LINKER_FLAGS "-Wl,--export-symbols")
 
-include_directories(../../libminifi/include  ../../libminifi/include/core  
../../thirdparty/spdlog-20170710/include ../../thirdparty/concurrentqueue 
../../thirdparty/yaml-cpp-yaml-cpp-0.5.3/include 
../../thirdparty/civetweb-1.9.1/include ../../thirdparty/jsoncpp/include 
../../thirdparty/rocksdb/include  ../../thirdparty/) 
--- End diff --

@phrocker 1000x we need some standard includes/functions for the cmake 
files.


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256173#comment-16256173
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r151569480
  
--- Diff: extensions/rocksdb-repos/CMakeLists.txt ---
@@ -23,7 +23,7 @@ cmake_minimum_required(VERSION 2.6)
 set(CMAKE_EXE_LINKER_FLAGS "-Wl,--export-all-symbols")
 set(CMAKE_SHARED_LINKER_FLAGS "-Wl,--export-symbols")
 
-include_directories(../../libminifi/include  ../../libminifi/include/core  
../../thirdparty/spdlog-20170710/include ../../thirdparty/concurrentqueue 
../../thirdparty/yaml-cpp-yaml-cpp-0.5.3/include 
../../thirdparty/civetweb-1.9.1/include ../../thirdparty/jsoncpp/include 
../../thirdparty/rocksdb/include  ../../thirdparty/) 
--- End diff --

thanks for catching this. We've all been copying and pasting these so at 
some point we need to clean them up and I'm glad you caught this. 


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256170#comment-16256170
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r151569308
  
--- Diff: extensions/libarchive/FocusArchiveEntry.cpp ---
@@ -295,7 +260,9 @@ int64_t 
FocusArchiveEntry::ReadCallback::process(std::shared_ptr
   return nlen;
 }
 
-FocusArchiveEntry::ReadCallback::ReadCallback(core::Processor 
*processor,fileutils::FileManager *file_man, ArchiveMetadata *archiveMetadata)
--- End diff --

Our line length is 200. We haven't merged your changes to lint the 
extensions but you may want to limit these types of changes to reduce later 
changes. 


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256168#comment-16256168
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r151569175
  
--- Diff: extensions/libarchive/FocusArchiveEntry.cpp ---
@@ -112,77 +109,45 @@ void 
FocusArchiveEntry::onTrigger(core::ProcessContext *context, core::ProcessSe
   if (targetEntryStashKey != "") {
 session->restore(targetEntryStashKey, flowFile);
   } else {
-logger_->log_warn("FocusArchiveEntry failed to locate target entry: 
%s", targetEntry.c_str());
+logger_->log_warn("FocusArchiveEntry failed to locate target entry: 
%s",
+  archiveMetadata.focusedEntry.c_str());
   }
 
   // Set new/updated lens stack to attribute
   {
-Json::Value lensStack;
-Json::Reader reader;
+ArchiveStack archiveStack;
 
 std::string existingLensStack;
 
 if (flowFile->getAttribute("lens.archive.stack", existingLensStack)) {
   logger_->log_info("FocusArchiveEntry loading existing lens context");
-  if (!reader.parse(existingLensStack, lensStack)) {
-logger_->log_error("FocusArchiveEntry JSON parse error: %s", 
reader.getFormattedErrorMessages());
+  try {
+archiveStack.loadJsonString(existingLensStack);
--- End diff --

Same here. Is this something we want to rollback or transfer to failure? 


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256164#comment-16256164
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r151568780
  
--- Diff: libminifi/src/RemoteProcessorGroupPort.cpp ---
@@ -264,20 +264,27 @@ void 
RemoteProcessorGroupPort::refreshRemoteSite2SiteInfo() {
 if (!response_body.empty()) {
   std::string controller = std::string(response_body.begin(), 
response_body.end());
   logger_->log_debug("controller config %s", controller.c_str());
-  Json::Value value;
-  Json::Reader reader;
-  bool parsingSuccessful = reader.parse(controller, value);
-  if (parsingSuccessful && !value.empty()) {
-Json::Value controllerValue = value["controller"];
-if (!controllerValue.empty()) {
-  Json::Value port = controllerValue["remoteSiteListeningPort"];
-  if (client_type_ == sitetosite::CLIENT_TYPE::RAW && 
!port.empty())
-this->site2site_port_ = port.asInt();
+
--- End diff --

If we're going to change the JSON code we should decouple it from this 
class. 


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256165#comment-16256165
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r151568839
  
--- Diff: 
libminifi/src/core/reporting/SiteToSiteProvenanceReportingTask.cpp ---
@@ -53,54 +56,92 @@ void SiteToSiteProvenanceReportingTask::initialize() {
   RemoteProcessorGroupPort::initialize();
 }
 
+void setJsonStr(const std::string& key, const std::string& value, 
rapidjson::Value& parent, rapidjson::Document::AllocatorType& alloc) { // NOLINT
--- End diff --

why is this here too?


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256163#comment-16256163
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r151568632
  
--- Diff: extensions/libarchive/UnfocusArchiveEntry.cpp ---
@@ -67,55 +66,41 @@ void 
UnfocusArchiveEntry::onTrigger(core::ProcessContext *context, core::Process
   }
 
   fileutils::FileManager file_man;
-
-  // Get lens stack from attribute
   ArchiveMetadata lensArchiveMetadata;
 
-  Json::Value lensStack;
-  Json::Reader reader;
-
-  std::string existingLensStack;
-
-  if (flowFile->getAttribute("lens.archive.stack", existingLensStack)) {
-logger_->log_info("UnfocusArchiveEntry loading existing lens context");
-
-// TODO(calebj) handle any exceptions that might arise from working 
with JSON data
-if (!reader.parse(existingLensStack, lensStack)) {
-  logger_->log_error("UnfocusArchiveEntry JSON parse error: %s", 
reader.getFormattedErrorMessages());
-  context->yield();
-  return;
+  // Get lens stack from attribute
+  {
+ArchiveStack archiveStack;
+{
+  std::string existingLensStack;
+
+  if (flowFile->getAttribute("lens.archive.stack", existingLensStack)) 
{
+logger_->log_info("FocusArchiveEntry loading existing lens 
context");
+
+try {
+  archiveStack.loadJsonString(existingLensStack);
--- End diff --

Should we not give the option to transfer this to failure? 


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256162#comment-16256162
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r151568175
  
--- Diff: extensions/http-curl/sitetosite/PeersEntity.h ---
@@ -35,73 +41,82 @@ class PeersEntity {
  public:
 
   static bool parse(const std::shared_ptr , const 
std::string , uuid_t id, std::vector _statuses) {
-
-Json::Reader reader;
-Json::Value root;
 try {
-  if (reader.parse(entity, root)) {
-if (root.isMember("peers") && root["peers"].size() > 0) {
-  for (const auto  : root["peers"]) {
+  rapidjson::Document root;
+  rapidjson::ParseResult ok = root.Parse(entity.c_str());
 
-std::string hostname;
-int port = 0, flowFileCount = 0;
-bool secure = false;
+  if (!ok) {
+  std::stringstream ss;
+  ss << "Failed to parse archive lens stack from JSON string with 
reason: "
+ << rapidjson::GetParseError_En(ok.Code())
+ << " at offset " << ok.Offset();
+  std::string exception_msg = ss.str();
+  
+  throw Exception(ExceptionType::GENERAL_EXCEPTION, 
exception_msg.c_str());
--- End diff --

Thanks for doing this. I was noticing this a few days ago.


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256158#comment-16256158
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r151568015
  
--- Diff: extensions/http-curl/protocols/RESTReceiver.cpp ---
@@ -65,38 +65,8 @@ void RESTReceiver::initialize(const 
std::shared_ptr Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256155#comment-16256155
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r151567480
  
--- Diff: extensions/http-curl/protocols/RESTProtocol.cpp ---
@@ -81,48 +93,117 @@ const C2Payload RESTProtocol::parseJsonResponse(const 
C2Payload , const
   return std::move(C2Payload(payload.getOperation(), 
state::UpdateState::READ_ERROR, true));
 }
 
-Json::Value RESTProtocol::serializeJsonPayload(Json::Value _root, 
const C2Payload ) {
-  // get the name from the content
-  Json::Value json_payload;
-  std::map children;
-  for (const auto _payload : payload.getNestedPayloads()) {
-Json::Value child_payload = serializeJsonPayload(json_payload, 
nested_payload);
-children[nested_payload.getLabel()].push_back(child_payload);
-  }
-  for (auto child_vector : children) {
-if (child_vector.second.size() > 1) {
-  Json::Value children_json(Json::arrayValue);
-  for (auto child : child_vector.second) {
-json_payload[child_vector.first] = child;
-  }
-} else {
-  if (child_vector.second.size() == 1) {
-if (child_vector.second.at(0).isMember(child_vector.first)) {
-  json_payload[child_vector.first] = 
child_vector.second.at(0)[child_vector.first];
-} else {
-  json_payload[child_vector.first] = child_vector.second.at(0);
-}
-  }
-}
-  }
+void setJsonStr(const std::string& key, const std::string& value, 
rapidjson::Value& parent, rapidjson::Document::AllocatorType& alloc) { // NOLINT
+  rapidjson::Value keyVal;
+  rapidjson::Value valueVal;
+  const char* c_key = key.c_str();
+  const char* c_val = value.c_str();
 
+  keyVal.SetString(c_key, key.length()), alloc;
+  valueVal.SetString(c_val, value.length(), alloc);
+
+  parent.AddMember(keyVal, valueVal, alloc);
+}
+
+rapidjson::Value getStringValue(const std::string& value, 
rapidjson::Document::AllocatorType& alloc) { // NOLINT
+  rapidjson::Value Val;
+  Val.SetString(value.c_str(), value.length(), alloc);
+  return Val;
+}
+
+void RESTProtocol::mergePayloadContent(rapidjson::Value , const 
C2Payload , rapidjson::Document::AllocatorType ) {
   const std::vector  = payload.getContent();
+
   for (const auto _content : content) {
-Json::Value payload_content_values;
+rapidjson::Value payload_content_values(rapidjson::kObjectType);
 bool use_sub_option = true;
+
 if (payload_content.op == payload.getOperation()) {
   for (auto content : payload_content.operation_arguments) {
 if (payload_content.operation_arguments.size() == 1 && 
payload_content.name == content.first) {
-  json_payload[payload_content.name] = content.second;
+  setJsonStr(payload_content.name, content.second, target, alloc);
   use_sub_option = false;
 } else {
-  payload_content_values[content.first] = content.second;
+  setJsonStr(content.first, content.second, 
payload_content_values, alloc);
 }
   }
 }
-if (use_sub_option)
-  json_payload[payload_content.name] = payload_content_values;
+if (use_sub_option) {
+  rapidjson::Value sub_key = getStringValue(payload_content.name, 
alloc);
+  target.AddMember(sub_key, payload_content_values, alloc);
+}
+  }
+}
+
+std::string RESTProtocol::serializeJsonRootPayload(const C2Payload& 
payload) {
+  rapidjson::Document json_payload(rapidjson::kObjectType);
+  rapidjson::Document::AllocatorType  = json_payload.GetAllocator();
+
+  rapidjson::Value opReqStrVal;
+  std::string operation_request_str = getOperation(payload);
+  opReqStrVal.SetString(operation_request_str.c_str(), 
operation_request_str.length(), alloc);
+  json_payload.AddMember("operation", opReqStrVal, alloc);
+
+  std::string operationid = payload.getIdentifier();
+  if (operationid.length() > 0) {
+rapidjson::Value operationIdVal = getStringValue(operationid, alloc);
+json_payload.AddMember("operationid", operationIdVal, alloc);
+  }
+
+  mergePayloadContent(json_payload, payload, alloc);
+
+  for (const auto _payload : payload.getNestedPayloads()) {
+rapidjson::Value np_key = getStringValue(nested_payload.getLabel(), 
alloc);
+rapidjson::Value np_value = serializeJsonPayload(nested_payload, 
alloc);
+json_payload.AddMember(np_key, np_value, alloc);
+  }
+
+  

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256152#comment-16256152
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r151567342
  
--- Diff: extensions/http-curl/protocols/RESTProtocol.cpp ---
@@ -81,48 +93,117 @@ const C2Payload RESTProtocol::parseJsonResponse(const 
C2Payload , const
   return std::move(C2Payload(payload.getOperation(), 
state::UpdateState::READ_ERROR, true));
 }
 
-Json::Value RESTProtocol::serializeJsonPayload(Json::Value _root, 
const C2Payload ) {
-  // get the name from the content
-  Json::Value json_payload;
-  std::map children;
-  for (const auto _payload : payload.getNestedPayloads()) {
-Json::Value child_payload = serializeJsonPayload(json_payload, 
nested_payload);
-children[nested_payload.getLabel()].push_back(child_payload);
-  }
-  for (auto child_vector : children) {
-if (child_vector.second.size() > 1) {
-  Json::Value children_json(Json::arrayValue);
-  for (auto child : child_vector.second) {
-json_payload[child_vector.first] = child;
-  }
-} else {
-  if (child_vector.second.size() == 1) {
-if (child_vector.second.at(0).isMember(child_vector.first)) {
-  json_payload[child_vector.first] = 
child_vector.second.at(0)[child_vector.first];
-} else {
-  json_payload[child_vector.first] = child_vector.second.at(0);
-}
-  }
-}
-  }
+void setJsonStr(const std::string& key, const std::string& value, 
rapidjson::Value& parent, rapidjson::Document::AllocatorType& alloc) { // NOLINT
+  rapidjson::Value keyVal;
+  rapidjson::Value valueVal;
+  const char* c_key = key.c_str();
+  const char* c_val = value.c_str();
 
+  keyVal.SetString(c_key, key.length()), alloc;
+  valueVal.SetString(c_val, value.length(), alloc);
+
+  parent.AddMember(keyVal, valueVal, alloc);
+}
+
+rapidjson::Value getStringValue(const std::string& value, 
rapidjson::Document::AllocatorType& alloc) { // NOLINT
+  rapidjson::Value Val;
+  Val.SetString(value.c_str(), value.length(), alloc);
+  return Val;
+}
+
+void RESTProtocol::mergePayloadContent(rapidjson::Value , const 
C2Payload , rapidjson::Document::AllocatorType ) {
   const std::vector  = payload.getContent();
+
   for (const auto _content : content) {
-Json::Value payload_content_values;
+rapidjson::Value payload_content_values(rapidjson::kObjectType);
 bool use_sub_option = true;
+
 if (payload_content.op == payload.getOperation()) {
   for (auto content : payload_content.operation_arguments) {
 if (payload_content.operation_arguments.size() == 1 && 
payload_content.name == content.first) {
-  json_payload[payload_content.name] = content.second;
+  setJsonStr(payload_content.name, content.second, target, alloc);
   use_sub_option = false;
 } else {
-  payload_content_values[content.first] = content.second;
+  setJsonStr(content.first, content.second, 
payload_content_values, alloc);
 }
   }
 }
-if (use_sub_option)
-  json_payload[payload_content.name] = payload_content_values;
+if (use_sub_option) {
+  rapidjson::Value sub_key = getStringValue(payload_content.name, 
alloc);
+  target.AddMember(sub_key, payload_content_values, alloc);
+}
+  }
+}
+
+std::string RESTProtocol::serializeJsonRootPayload(const C2Payload& 
payload) {
+  rapidjson::Document json_payload(rapidjson::kObjectType);
+  rapidjson::Document::AllocatorType  = json_payload.GetAllocator();
--- End diff --

Can the allocator be saved for future payloads?


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256150#comment-16256150
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user phrocker commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/191
  
@calebj Thanks for the contribution, but before this can be merged can you 
provide us some tractable benchmarks? Are we making a library change just to 
make a library change based on evidence? https://github.com/google/benchmark is 
a great tool that will help. 


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256147#comment-16256147
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r151566881
  
--- Diff: extensions/http-curl/protocols/RESTProtocol.cpp ---
@@ -81,48 +93,117 @@ const C2Payload RESTProtocol::parseJsonResponse(const 
C2Payload , const
   return std::move(C2Payload(payload.getOperation(), 
state::UpdateState::READ_ERROR, true));
 }
 
-Json::Value RESTProtocol::serializeJsonPayload(Json::Value _root, 
const C2Payload ) {
-  // get the name from the content
-  Json::Value json_payload;
-  std::map children;
-  for (const auto _payload : payload.getNestedPayloads()) {
-Json::Value child_payload = serializeJsonPayload(json_payload, 
nested_payload);
-children[nested_payload.getLabel()].push_back(child_payload);
-  }
-  for (auto child_vector : children) {
-if (child_vector.second.size() > 1) {
-  Json::Value children_json(Json::arrayValue);
-  for (auto child : child_vector.second) {
-json_payload[child_vector.first] = child;
-  }
-} else {
-  if (child_vector.second.size() == 1) {
-if (child_vector.second.at(0).isMember(child_vector.first)) {
-  json_payload[child_vector.first] = 
child_vector.second.at(0)[child_vector.first];
-} else {
-  json_payload[child_vector.first] = child_vector.second.at(0);
-}
-  }
-}
-  }
+void setJsonStr(const std::string& key, const std::string& value, 
rapidjson::Value& parent, rapidjson::Document::AllocatorType& alloc) { // NOLINT
+  rapidjson::Value keyVal;
+  rapidjson::Value valueVal;
+  const char* c_key = key.c_str();
+  const char* c_val = value.c_str();
 
+  keyVal.SetString(c_key, key.length()), alloc;
+  valueVal.SetString(c_val, value.length(), alloc);
+
+  parent.AddMember(keyVal, valueVal, alloc);
+}
+
+rapidjson::Value getStringValue(const std::string& value, 
rapidjson::Document::AllocatorType& alloc) { // NOLINT
+  rapidjson::Value Val;
--- End diff --

This function seems almost unnecessary. Why not use a macro so that it's 
always inlined? 


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256145#comment-16256145
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r151566718
  
--- Diff: extensions/http-curl/protocols/RESTProtocol.cpp ---
@@ -81,48 +93,117 @@ const C2Payload RESTProtocol::parseJsonResponse(const 
C2Payload , const
   return std::move(C2Payload(payload.getOperation(), 
state::UpdateState::READ_ERROR, true));
 }
 
-Json::Value RESTProtocol::serializeJsonPayload(Json::Value _root, 
const C2Payload ) {
-  // get the name from the content
-  Json::Value json_payload;
-  std::map children;
-  for (const auto _payload : payload.getNestedPayloads()) {
-Json::Value child_payload = serializeJsonPayload(json_payload, 
nested_payload);
-children[nested_payload.getLabel()].push_back(child_payload);
-  }
-  for (auto child_vector : children) {
-if (child_vector.second.size() > 1) {
-  Json::Value children_json(Json::arrayValue);
-  for (auto child : child_vector.second) {
-json_payload[child_vector.first] = child;
-  }
-} else {
-  if (child_vector.second.size() == 1) {
-if (child_vector.second.at(0).isMember(child_vector.first)) {
-  json_payload[child_vector.first] = 
child_vector.second.at(0)[child_vector.first];
-} else {
-  json_payload[child_vector.first] = child_vector.second.at(0);
-}
-  }
-}
-  }
+void setJsonStr(const std::string& key, const std::string& value, 
rapidjson::Value& parent, rapidjson::Document::AllocatorType& alloc) { // NOLINT
--- End diff --

Why is setJsonStr here without any scope and should it be elsewhere as a 
result?


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256144#comment-16256144
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r151566381
  
--- Diff: extensions/http-curl/protocols/RESTProtocol.cpp ---
@@ -81,48 +93,117 @@ const C2Payload RESTProtocol::parseJsonResponse(const 
C2Payload , const
   return std::move(C2Payload(payload.getOperation(), 
state::UpdateState::READ_ERROR, true));
 }
 
-Json::Value RESTProtocol::serializeJsonPayload(Json::Value _root, 
const C2Payload ) {
-  // get the name from the content
-  Json::Value json_payload;
-  std::map children;
-  for (const auto _payload : payload.getNestedPayloads()) {
-Json::Value child_payload = serializeJsonPayload(json_payload, 
nested_payload);
-children[nested_payload.getLabel()].push_back(child_payload);
-  }
-  for (auto child_vector : children) {
-if (child_vector.second.size() > 1) {
-  Json::Value children_json(Json::arrayValue);
-  for (auto child : child_vector.second) {
-json_payload[child_vector.first] = child;
-  }
-} else {
-  if (child_vector.second.size() == 1) {
-if (child_vector.second.at(0).isMember(child_vector.first)) {
-  json_payload[child_vector.first] = 
child_vector.second.at(0)[child_vector.first];
-} else {
-  json_payload[child_vector.first] = child_vector.second.at(0);
-}
-  }
-}
-  }
+void setJsonStr(const std::string& key, const std::string& value, 
rapidjson::Value& parent, rapidjson::Document::AllocatorType& alloc) { // NOLINT
--- End diff --

why is there no linter here? We want this linted. 


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256138#comment-16256138
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r151566198
  
--- Diff: extensions/http-curl/protocols/RESTProtocol.cpp ---
@@ -32,46 +33,57 @@ namespace minifi {
 namespace c2 {
 
 const C2Payload RESTProtocol::parseJsonResponse(const C2Payload , 
const std::vector ) {
-  Json::Reader reader;
-  Json::Value root;
+  rapidjson::Document root;
+
   try {
-if (reader.parse(std::string(response.data(), response.size()), root)) 
{
+rapidjson::ParseResult ok = root.Parse(response.data(), 
response.size());
+
+if (ok) {
   std::string requested_operation = getOperation(payload);
 
   std::string identifier;
-  if (root.isMember("operationid")) {
-identifier = root["operationid"].asString();
+  if (root.HasMember("operationid")) {
+identifier = root["operationid"].GetString();
   }
-  if (root["operation"].asString() == requested_operation) {
-if (root["requested_operations"].size() == 0) {
+
+  if (root["operation"].GetString() == requested_operation) {
+if (root["requested_operations"].Size() == 0)
   return std::move(C2Payload(payload.getOperation(), 
state::UpdateState::READ_COMPLETE, true));
-}
+
 C2Payload new_payload(payload.getOperation(), 
state::UpdateState::NESTED, true);
 
 new_payload.setIdentifier(identifier);
 
-for (const Json::Value& request : root["requested_operations"]) {
-  Operation newOp = 
stringToOperation(request["operation"].asString());
+for (const rapidjson::Value& request : 
root["requested_operations"].GetArray()) {
+  Operation newOp = 
stringToOperation(request["operation"].GetString());
   C2Payload nested_payload(newOp, 
state::UpdateState::READ_COMPLETE, true);
   C2ContentResponse new_command(newOp);
   new_command.delay = 0;
   new_command.required = true;
   new_command.ttl = -1;
+
   // set the identifier if one exists
-  if (request.isMember("operationid")) {
-new_command.ident = request["operationid"].asString();
+  if (request.HasMember("operationid")) {
+if (request["operationid"].IsNumber())
+  new_command.ident = 
std::to_string(request["operationid"].GetInt64());
+else if (request["operationid"].IsString())
+  new_command.ident = request["operationid"].GetString();
+else
+  throw(Exception(SITE2SITE_EXCEPTION, "Invalid type for 
operationid"));
--- End diff --

This exception should not be here. 


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256060#comment-16256060
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

Github user achristianson commented on the issue:

https://github.com/apache/nifi-minifi-cpp/pull/191
  
@calebj whoa, nice!!! This is something I have been thinking about for a 
while, but was really back burner.

This is great, will check it out and let you know if I find anything.


> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256059#comment-16256059
 ] 

ASF GitHub Bot commented on MINIFICPP-114:
--

GitHub user calebj opened a pull request:

https://github.com/apache/nifi-minifi-cpp/pull/191

MINIFICPP-114 Consolidate JSON API use to RapidJSON

Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
 in the commit message?

- [x] Does your PR title start with MINIFI- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [x] Is your initial contribution a single, squashed commit?

### For code changes:
- [x] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [x] If applicable, have you updated the LICENSE file?
- [x] If applicable, have you updated the NOTICE file?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/NiFiLocal/nifi-minifi-cpp MINIFICPP-114

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-minifi-cpp/pull/191.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #191


commit cfe9421098f1ab3b09528b56f12482a3147db9cb
Author: Caleb Johnson 
Date:   2017-11-15T12:38:54Z

MINIFICPP-114 Consolidate JSON API use to RapidJSON




> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MINIFICPP-114) Consolidate JSON API use to RapidJSON

[Caleb Johnson (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16237662#comment-16237662
 ] 

Caleb Johnson commented on MINIFICPP-114:
-

Is this a release priority? The following components currently depend on 
jsoncpp:
* Un/FocusArchive
* RemoteProcessorGroupPort
* Site-to-site provenance reporting

I can work on this if John doesn't want to.

> Consolidate JSON API use to RapidJSON
> -
>
> Key: MINIFICPP-114
> URL: https://issues.apache.org/jira/browse/MINIFICPP-114
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: John Reynolds
>Assignee: John Reynolds
>Priority: Major
>
> MiNiFi was updated with new dependencies to jsoncpp: 
> SiteToSiteProvenanceReportingTask
> Performance observations indicate significant improvements with RapidJSON vs. 
> jsoncpp.  Convert all use of jsoncpp to RapidJSON where applicable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)