Hi Folks, I've started development on the graphsink for flume and looking for some initial thoughts on the direction/design. Would appreciate some more comitters feedback on this. Initially I will build an abstraction layer around a generic graph sink that underneath will talk to neo4j or titan, the functionality for the sink event for the first rev could be:
1) populate all the nodes and relationships in a graph 2) delete a set of nodes ore relationships in a graph 3) be able to run algorithms through the graph like dijkstra or A* either using a declarative query language inside the graph store or directly through the java API I'd like to know if there are other things that may be interesting to consider or if there are gaps I should consider. Please make your comments directly on the JIRA ticket. Best Regards > Date: Tue, 7 May 2013 18:17:16 +0000 > From: [email protected] > To: [email protected] > Subject: [jira] [Commented] (FLUME-2035) GraphSink for flume > > > [ > https://issues.apache.org/jira/browse/FLUME-2035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13651134#comment-13651134 > ] > > Israel Ekpo commented on FLUME-2035: > ------------------------------------ > > I think this is going to require having specialized sources and channels for > this sink. > > This will allow for easier implementation of graph-specific logic as the data > flows downstream which could vary based on the data structures involved. > > The logic could depend on if each event will represent just a vertex or a > subgraph (directed or undirected). > > I suspect the events will represent a subgraph where each vertex within the > graph will have a degree of one or greater. > > Most of the sources and channels don't really handle graph data out of the > box. > > I will put some more thought into this and put my feedback later. > > Nevertheless, I think this is a great idea and an excellent intiative! > > > GraphSink for flume > > ------------------- > > > > Key: FLUME-2035 > > URL: https://issues.apache.org/jira/browse/FLUME-2035 > > Project: Flume > > Issue Type: New Feature > > Components: Sinks+Sources > > Reporter: Saikat Kanjilal > > Original Estimate: 168h > > Remaining Estimate: 168h > > > > The goal of this feature is to implement a graph sink for flume , the sink > > will connect to both Titan and neo4j and others through an abstraction > > layer. > > -- > This message is automatically generated by JIRA. > If you think it was sent incorrectly, please contact your JIRA administrators > For more information on JIRA, see: http://www.atlassian.com/software/jira
