[GitHub] flink pull request: [FLINK-1434] [FLINK-1401] Streaming support ad...
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/334 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1434] [FLINK-1401] Streaming support ad...
Github user gyfora commented on a diff in the pull request:
https://github.com/apache/flink/pull/334#discussion_r23503616
--- Diff: flink-addons/flink-streaming/flink-streaming-core/pom.xml ---
@@ -48,6 +48,12 @@ under the License.
version${project.version}/version
scopetest/scope
/dependency
+
+dependency
+groupIdorg.apache.sling/groupId
+artifactIdorg.apache.sling.commons.json/artifactId
+version2.0.6/version
+/dependency
--- End diff --
This is already a dependency of the flink-connectors.
It might be good at some point to refactor the batch JSON plan generator to
use some actual JSON parser to make it look less messy
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1434] [FLINK-1401] Streaming support ad...
Github user gyfora commented on a diff in the pull request:
https://github.com/apache/flink/pull/334#discussion_r23503611
--- Diff:
flink-addons/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/StreamGraph.java
---
@@ -536,4 +549,79 @@ public long getIterationTimeout(String vertexName) {
return iterationTimeouts.get(vertexName);
}
+ public String getOperatorName(String vertexName) {
+ return operatorNames.get(vertexName);
+ }
+
+ @Override
+ public String getStreamingPlanAsJSON() {
+
+ try {
+ JSONObject json = new JSONObject();
+ JSONArray nodes = new JSONArray();
+
+ json.put(nodes, nodes);
+
+ for (String id : operatorNames.keySet()) {
+ JSONObject node = new JSONObject();
+ nodes.put(node);
+
+ node.put(id, Integer.valueOf(id));
+ node.put(type, getOperatorName(id));
+
+ if (sources.contains(id)) {
+ node.put(pact, Data Source);
+ } else {
+ node.put(pact, Data Stream);
+ }
+
+ node.put(contents, getOperatorName(id) + at
+ +
getInvokable(id).getUserFunction().getClass().getSimpleName());
+ node.put(parallelism, getParallelism(id));
+
+ int numIn = getInEdges(id).size();
+ if (numIn 0) {
+
+ JSONArray inputs = new JSONArray();
+ node.put(predecessors, inputs);
+
+ for (int i = 0; i numIn; i++) {
+
+ String inID =
getInEdges(id).get(i);
+
+ JSONObject input = new
JSONObject();
+ inputs.put(input);
+
+ input.put(id,
Integer.valueOf(inID));
+ input.put(ship_strategy,
getOutPartitioner(inID, id).getStrategy());
+ if (i == 0) {
+ input.put(side,
first);
+ } else if (i == 1) {
+ input.put(side,
second);
+ }
+ }
+ }
+
+ }
+ return json.toString();
+ } catch (Exception e) {
--- End diff --
Fixed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1434] [FLINK-1401] Streaming support ad...
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/334#discussion_r23503538
--- Diff: flink-addons/flink-streaming/flink-streaming-core/pom.xml ---
@@ -48,6 +48,12 @@ under the License.
version${project.version}/version
scopetest/scope
/dependency
+
+dependency
+groupIdorg.apache.sling/groupId
+artifactIdorg.apache.sling.commons.json/artifactId
+version2.0.6/version
+/dependency
--- End diff --
how many new / and which transitive dependencies are added by this
dependency?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1434] [FLINK-1401] Streaming support ad...
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/334#discussion_r23503540
--- Diff:
flink-addons/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/StreamGraph.java
---
@@ -536,4 +549,79 @@ public long getIterationTimeout(String vertexName) {
return iterationTimeouts.get(vertexName);
}
+ public String getOperatorName(String vertexName) {
+ return operatorNames.get(vertexName);
+ }
+
+ @Override
+ public String getStreamingPlanAsJSON() {
+
+ try {
+ JSONObject json = new JSONObject();
+ JSONArray nodes = new JSONArray();
+
+ json.put(nodes, nodes);
+
+ for (String id : operatorNames.keySet()) {
+ JSONObject node = new JSONObject();
+ nodes.put(node);
+
+ node.put(id, Integer.valueOf(id));
+ node.put(type, getOperatorName(id));
+
+ if (sources.contains(id)) {
+ node.put(pact, Data Source);
+ } else {
+ node.put(pact, Data Stream);
+ }
+
+ node.put(contents, getOperatorName(id) + at
+ +
getInvokable(id).getUserFunction().getClass().getSimpleName());
+ node.put(parallelism, getParallelism(id));
+
+ int numIn = getInEdges(id).size();
+ if (numIn 0) {
+
+ JSONArray inputs = new JSONArray();
+ node.put(predecessors, inputs);
+
+ for (int i = 0; i numIn; i++) {
+
+ String inID =
getInEdges(id).get(i);
+
+ JSONObject input = new
JSONObject();
+ inputs.put(input);
+
+ input.put(id,
Integer.valueOf(inID));
+ input.put(ship_strategy,
getOutPartitioner(inID, id).getStrategy());
+ if (i == 0) {
+ input.put(side,
first);
+ } else if (i == 1) {
+ input.put(side,
second);
+ }
+ }
+ }
+
+ }
+ return json.toString();
+ } catch (Exception e) {
--- End diff --
maybe we should at least LOG.debug() the exception?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
