Re: ClassNotFoundException
Hi Vasia, Thanks a lot for your help! It worked out! Best regards, Olga Best regards, Olga Golovneva On Wed, Sep 28, 2016 at 6:12 PM, Vasiliki Kalavri <vasilikikala...@gmail.com > wrote: > Hi Olga, > > have you copied the flink-gelly jar to your TaskManagers? See the > instructions in [1] on how you can make the gelly classes available during > execution and let us know if you have any problems. > > Best, > -Vasia, > > [1]: > https://ci.apache.org/projects/flink/flink-docs- > master/dev/cluster_execution.html#linking-with-modules-not- > contained-in-the-binary-distribution > > On 28 September 2016 at 20:58, Olga Golovneva <melcha...@gmail.com> wrote: > > > Hi devs, > > > > I'm trying to run Gelly ClusteringCoefficient example on a cluster using > > this command: > > > > flink run -C > > file://home/ubuntu/flink/flink-libraries/flink-gelly/ > > target/flink-gelly_2.10-1.2-SNAPSHOT.jar > > -c org.apache.flink.graph.examples.ClusteringCoefficient > > flink-gelly-examples_2.10-1.2-SNAPSHOT.jar > > > > and get the following ClassNotFoundException: > > > > > > > > The program finished with the following exception: > > > > > > java.lang.RuntimeException: Could not look up the main(String[]) method > > from the class org.apache.flink.graph.examples.ClusteringCoefficient: > > org/apache/flink/graph/GraphAnalytic > > > > at > > org.apache.flink.client.program.PackagedProgram. > > hasMainMethod(PackagedProgram.java:478) > > > > at > > org.apache.flink.client.program.PackagedProgram. > > (PackagedProgram.java:216) > > > > at org.apache.flink.client.CliFrontend.buildProgram( > CliFrontend.java:829) > > > > at org.apache.flink.client.CliFrontend.run(CliFrontend.java:222) > > > > at > > org.apache.flink.client.CliFrontend.parseParameters( > CliFrontend.java:1002) > > > > at org.apache.flink.client.CliFrontend.main(CliFrontend.java:1045) > > > > Caused by: java.lang.NoClassDefFoundError: > > org/apache/flink/graph/GraphAnalytic > > > > at java.lang.Class.getDeclaredMethods0(Native Method) > > > > at java.lang.Class.privateGetDeclaredMethods(Class.java:2625) > > > > at java.lang.Class.getMethod0(Class.java:2866) > > > > at java.lang.Class.getMethod(Class.java:1676) > > > > at > > org.apache.flink.client.program.PackagedProgram. > > hasMainMethod(PackagedProgram.java:473) > > > > ... 5 more > > > > Caused by: java.lang.ClassNotFoundException: > > org.apache.flink.graph.GraphAnalytic > > > > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > > > > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > > > > at java.security.AccessController.doPrivileged(Native Method) > > > > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > > > > ... 10 more > > > > __ > > > > I got similar Exceptions for other examples in Gelly library, only the > > missing methods are different. Perhaps someone more knowledgeable could > > kindly help? Where should I look for org.apache.flink.graph. > GraphAnalytic? > > I thought it should be in flink-gelly_2.10-1.2-SNAPSHOT.jar, but it does > > not work > > > > Thank you, > > Olga > > >
ClassNotFoundException
Hi devs, I'm trying to run Gelly ClusteringCoefficient example on a cluster using this command: flink run -C file://home/ubuntu/flink/flink-libraries/flink-gelly/target/flink-gelly_2.10-1.2-SNAPSHOT.jar -c org.apache.flink.graph.examples.ClusteringCoefficient flink-gelly-examples_2.10-1.2-SNAPSHOT.jar and get the following ClassNotFoundException: The program finished with the following exception: java.lang.RuntimeException: Could not look up the main(String[]) method from the class org.apache.flink.graph.examples.ClusteringCoefficient: org/apache/flink/graph/GraphAnalytic at org.apache.flink.client.program.PackagedProgram.hasMainMethod(PackagedProgram.java:478) at org.apache.flink.client.program.PackagedProgram.(PackagedProgram.java:216) at org.apache.flink.client.CliFrontend.buildProgram(CliFrontend.java:829) at org.apache.flink.client.CliFrontend.run(CliFrontend.java:222) at org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:1002) at org.apache.flink.client.CliFrontend.main(CliFrontend.java:1045) Caused by: java.lang.NoClassDefFoundError: org/apache/flink/graph/GraphAnalytic at java.lang.Class.getDeclaredMethods0(Native Method) at java.lang.Class.privateGetDeclaredMethods(Class.java:2625) at java.lang.Class.getMethod0(Class.java:2866) at java.lang.Class.getMethod(Class.java:1676) at org.apache.flink.client.program.PackagedProgram.hasMainMethod(PackagedProgram.java:473) ... 5 more Caused by: java.lang.ClassNotFoundException: org.apache.flink.graph.GraphAnalytic at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ... 10 more __ I got similar Exceptions for other examples in Gelly library, only the missing methods are different. Perhaps someone more knowledgeable could kindly help? Where should I look for org.apache.flink.graph.GraphAnalytic? I thought it should be in flink-gelly_2.10-1.2-SNAPSHOT.jar, but it does not work Thank you, Olga
Re: On (FLINK-1526) JIRA issue
Hi Vasia, I have uploaded these tests on github: https://github.com/OlgaGolovneva/MST/tree/master/tests I have also uploaded source code, but I'm still working on it: https://github.com/OlgaGolovneva/MST/tree/master/src >I think you cannot add attachments to the mailing list. Could you upload >your example somewhere and post a link here? I'm actually surprised that >the while-loop works without problems. I have run the program on several simple tests, and I was going to try large datasets in the next few days. Please, let me know if this approach is wrong. Thanks, Olga On Wed, Sep 21, 2016 at 4:55 PM, Vasiliki Kalavri <vasilikikala...@gmail.com > wrote: > Hi Olga, > > On 21 September 2016 at 18:50, Olga Golovneva <melcha...@gmail.com> wrote: > > > Hi devs, > > > > I was working on (FLINK-1526) "Add Minimum Spanning Tree library method > > and example" issue. I've developed (Java) code that implements > distributed > > Boruvka's algorithm in Gelly library. I've run several tests and it seems > > to work fine, although I didn't test it on extremely large input graphs > > yet, and I'm also trying to optimize my code. > > Particularly, I have two main issues: > > > > 1. Nested loops. > > I have to use nested loops, and I do not see the way to avoid them. As > > they are currently not supported, I'm using Bulk Iterations inside a > > "classic" while loop. I've included in attachment simple example > > MyNestedIterationExample that shows this issue. > > > > I think you cannot add attachments to the mailing list. Could you upload > your example somewhere and post a link here? I'm actually surprised that > the while-loop works without problems. > > > > > > 2. For some reason I cannot create class that works with types with > > generic variables in Tuple2(or Tuple3), thus my code does not support > > generic types. I also included simple example MyTuple3Example. Here is > the > > Exception I get: > > "Exception in thread "main" org.apache.flink.api.common.functions. > InvalidTypesException: > > Type of TypeVariable 'EV' in 'class org.apache.flink.graph. > > examples.MyTuple3Example$InitializeEdges' could not be determined. This > > is most likely a type erasure problem. The type extraction currently > > supports types with generic variables only in cases where all variables > in > > the return type can be deduced from the input type(s)." > > > > Can you upload this example and link to it too? > > Thanks, > -Vasia. > > > > > > I would really appreciate if someone could explain me know how to avoid > > this Exception. Otherwise, I could submit my code for testing. > > > > Best regards, > > Olga Golovneva > > >
On (FLINK-1526) JIRA issue
Hi devs, I was working on (FLINK-1526) "Add Minimum Spanning Tree library method and example" issue. I've developed (Java) code that implements distributed Boruvka's algorithm in Gelly library. I've run several tests and it seems to work fine, although I didn't test it on extremely large input graphs yet, and I'm also trying to optimize my code. Particularly, I have two main issues: 1. Nested loops. I have to use nested loops, and I do not see the way to avoid them. As they are currently not supported, I'm using Bulk Iterations inside a "classic" while loop. I've included in attachment simple example MyNestedIterationExample that shows this issue. 2. For some reason I cannot create class that works with types with generic variables in Tuple2(or Tuple3), thus my code does not support generic types. I also included simple example MyTuple3Example. Here is the Exception I get: "Exception in thread "main" org.apache.flink.api.common.functions.InvalidTypesException: Type of TypeVariable 'EV' in 'class org.apache.flink.graph.examples.MyTuple3Example$InitializeEdges' could not be determined. This is most likely a type erasure problem. The type extraction currently supports types with generic variables only in cases where all variables in the return type can be deduced from the input type(s)." I would really appreciate if someone could explain me know how to avoid this Exception. Otherwise, I could submit my code for testing. Best regards, Olga Golovneva
Re: Gelly Library. Need an example
c void main(String [] args) throws Exception {
ExecutionEnvironment env =
ExecutionEnvironment.getExecutionEnvironment();
//Create graph
DataSet<Edge<Long, Double>> edges = getEdgeDataSet(env);
DataSet<Vertex<Long, Long>> vertices = getVertexDataSet(env);
Graph<Long, Long, Double> graph = Graph.fromDataSet(vertices,
edges, env);
//emit input
System.out.println("Executing example with following
inputs:\n"+"Vertices:\n");
vertices.print();
System.out.println("Edges:\n");
edges.print();
Graph<Long, Summarization.VertexValue,
Summarization.EdgeValue> result = graph
.run(new Summarization<Long, Long, Double>());
//now we want to read the output
DataSet<Edge<Long, Summarization.EdgeValue>> edgesOut
= result.getEdges();
DataSet<Vertex<Long, Summarization.VertexValue>>
verticesOut = result.getVertices();
// emit result
System.out.println("Summarized graph:\n"+"Vertices:\n");
verticesOut.print();
System.out.println("Edges:\n");
edgesOut.print();
}
@Override
public String getDescription() {
return "Summarization Example";
}
//Define edges
private static DataSet<Edge<Long, Double>>
getEdgeDataSet(ExecutionEnvironment env) {
Object[][] DEFAULT_EDGES = new Object[][] {
new Object[]{1L, 2L, 1.0},
new Object[]{1L, 4L, 3.0},
new Object[]{2L, 3L, 6.0},
new Object[]{2L, 4L, 5.0},
new Object[]{2L, 5L, 1.0},
new Object[]{3L, 5L, 5.0},
new Object[]{3L, 6L, 2.0},
new Object[]{4L, 5L, 1.0},
new Object[]{5L, 6L, 4.0}
};
List<Edge<Long, Double>> edgeList = new LinkedList<Edge<Long,
Double>>();
for (Object[] edge : DEFAULT_EDGES) {
edgeList.add(new Edge<Long, Double>((Long) edge[0], (Long)
edge[1], (Double) edge[2]));
}
return env.fromCollection(edgeList);
}
//Define vertices
private static DataSet<Vertex<Long, Long>>
getVertexDataSet(ExecutionEnvironment env) {
//We will summarize by = Long
Object[][] DEFAULT_VERTICES = new Object[][] {
new Object[]{1L, 1L},
new Object[]{2L, 1L},
new Object[]{3L, 5L},
new Object[]{4L, 5L},
new Object[]{5L, 5L}
};
List<Vertex<Long, Long>> vertexList = new
LinkedList<Vertex<Long, Long>>();
for (Object[] vertex : DEFAULT_VERTICES) {
vertexList.add(new Vertex<Long, Long>((Long) vertex[0],
(Long) vertex[1]));
}
return env.fromCollection(vertexList);
}
}
_EXAMPLE END_
Best regards,
Olga Golovneva
On Thu, Sep 15, 2016 at 9:16 AM, Till Rohrmann <trohrm...@apache.org> wrote:
> Hi Olga,
>
> can you provide us with a little bit more details about the problem. The
> full stack trace of the exception and the program you're trying to run
> would be helpful.
>
> Cheers,
> Till
>
> On Wed, Sep 14, 2016 at 9:49 PM, Olga Golovneva <melcha...@gmail.com>
> wrote:
>
> > Hi devs,
> >
> > Do you know if there is an example (besides ITCase) of usage of
> > Summarization Library in Gelly? I'm having some problems trying to use it
> > in my code. Particularly, I cannot print output edges ( it throws the
> > following exception: Exception in thread "main"
> > org.apache.flink.runtime.client.JobExecutionException: Job execution
> > failed.), while vertices are printed correctly.
> >
> > Best regards,
> > Olga
> >
>
Gelly Library. Need an example
Hi devs, Do you know if there is an example (besides ITCase) of usage of Summarization Library in Gelly? I'm having some problems trying to use it in my code. Particularly, I cannot print output edges ( it throws the following exception: Exception in thread "main" org.apache.flink.runtime.client.JobExecutionException: Job execution failed.), while vertices are printed correctly. Best regards, Olga
