[jira] [Comment Edited] (CASSANDRA-6846) Provide standard interface for deep application server integration
[ https://issues.apache.org/jira/browse/CASSANDRA-6846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13937349#comment-13937349 ] Tupshin Harper edited comment on CASSANDRA-6846 at 3/16/14 9:52 PM: So that's what I was getting at above regarding just making it easier to embed Cassandra. Does there really need to be a way of bringing up an additional server when you start Cassandra? Or should the requirement be that to do that, you have to embed Cassandra, which is already done today by projects like Titan. https://github.com/thinkaurelius/titan/wiki/Using-Cassandra Honestly, I don't have any strong thoughts on this. Opinions? I would, however, like to see a future where Cassandra could be easily embeddable with such a small footprint that it would be a single mega-jar, with one external config file, one optional commitlog dir and a data directory, and otherwise totally self contained. was (Author: tupshin): So that's what I was getting at above regarding just making it easier to embed Cassandra. Does there really need to be a way of bringing up an additional server when you start Cassandra? Or should the requirement be that to do that, you have to embed Cassandra, which is already done today by projects like Titan. https://github.com/thinkaurelius/titan/wiki/Using-Cassandra Honestly, I don't have a strong opinion on this. Opinions? > Provide standard interface for deep application server integration > -- > > Key: CASSANDRA-6846 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6846 > Project: Cassandra > Issue Type: New Feature > Components: API, Core >Reporter: Tupshin Harper >Assignee: Tupshin Harper >Priority: Minor > Labels: ponies > > Instead of creating a pluggable interface for Thrift, I'd like to create a > pluggable interface for arbitrary app-server deep integration. > Inspired by both the existence of intravert-ug, as well as there being a long > history of various parties embedding tomcat or jetty servlet engines inside > Cassandra, I'd like to propose the creation an internal somewhat stable > (versioned?) interface that could allow any app server to achieve deep > integration with Cassandra, and as a result, these servers could > 1) host their own apis (REST, for example > 2) extend core functionality by having limited (see triggers and wide row > scanners) access to the internals of cassandra > The hand wavey part comes because while I have been mulling this about for a > while, I have not spent any significant time into looking at the actual > surface area of intravert-ug's integration. But, using it as a model, and > also keeping in minds the general needs of your more traditional servlet/j2ee > containers, I believe we could come up with a reasonable interface to allow > any jvm app server to be integrated and maintained in or out of the Cassandra > tree. > This would satisfy the needs that many of us (Both Ed and I, for example) to > have a much greater degree of control over server side execution, and to be > able to start building much more interestingly (and simply) tiered > applications. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-6846) Provide standard interface for deep application server integration
[ https://issues.apache.org/jira/browse/CASSANDRA-6846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13937349#comment-13937349 ] Tupshin Harper edited comment on CASSANDRA-6846 at 3/16/14 9:49 PM: So that's what I was getting at above regarding just making it easier to embed Cassandra. Does there really need to be a way of bringing up an additional server when you start Cassandra? Or should the requirement be that to do that, you have to embed Cassandra, which is already done today by projects like Titan. https://github.com/thinkaurelius/titan/wiki/Using-Cassandra Honestly, I don't have a strong opinion on this. Opinions? was (Author: tupshin): So that's what I was getting at abov regarding just making it easier to embed Cassandra. Does there really need to be a way of bringing up an additional server when you start Cassandra? Or should the requirement be that to do that, you have to embed Cassandra, which is already done today by projects like Titan. https://github.com/thinkaurelius/titan/wiki/Using-Cassandra Honestly, I don't have a strong opinion on this. Opinions? > Provide standard interface for deep application server integration > -- > > Key: CASSANDRA-6846 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6846 > Project: Cassandra > Issue Type: New Feature > Components: API, Core >Reporter: Tupshin Harper >Assignee: Tupshin Harper >Priority: Minor > Labels: ponies > > Instead of creating a pluggable interface for Thrift, I'd like to create a > pluggable interface for arbitrary app-server deep integration. > Inspired by both the existence of intravert-ug, as well as there being a long > history of various parties embedding tomcat or jetty servlet engines inside > Cassandra, I'd like to propose the creation an internal somewhat stable > (versioned?) interface that could allow any app server to achieve deep > integration with Cassandra, and as a result, these servers could > 1) host their own apis (REST, for example > 2) extend core functionality by having limited (see triggers and wide row > scanners) access to the internals of cassandra > The hand wavey part comes because while I have been mulling this about for a > while, I have not spent any significant time into looking at the actual > surface area of intravert-ug's integration. But, using it as a model, and > also keeping in minds the general needs of your more traditional servlet/j2ee > containers, I believe we could come up with a reasonable interface to allow > any jvm app server to be integrated and maintained in or out of the Cassandra > tree. > This would satisfy the needs that many of us (Both Ed and I, for example) to > have a much greater degree of control over server side execution, and to be > able to start building much more interestingly (and simply) tiered > applications. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-6846) Provide standard interface for deep application server integration
[ https://issues.apache.org/jira/browse/CASSANDRA-6846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13933888#comment-13933888 ] Tupshin Harper edited comment on CASSANDRA-6846 at 3/13/14 6:58 PM: So let's continue to use this ticket to explore what the gap is between what I proposed, and what you see as a minimally viable versions of the need expressed by #2 and #3. FWIW, I do still think that UDFs can fulfill your goals for wide partition scanners, if defined capably enough. was (Author: tupshin): So let's continue to use this ticket to explore what the gap is between what I proposed, and what you see as a minimally viable versions of the need expressed by #2 and #3. > Provide standard interface for deep application server integration > -- > > Key: CASSANDRA-6846 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6846 > Project: Cassandra > Issue Type: New Feature > Components: API, Core >Reporter: Tupshin Harper >Assignee: Tupshin Harper >Priority: Minor > Labels: ponies > > Instead of creating a pluggable interface for Thrift, I'd like to create a > pluggable interface for arbitrary app-server deep integration. > Inspired by both the existence of intravert-ug, as well as there being a long > history of various parties embedding tomcat or jetty servlet engines inside > Cassandra, I'd like to propose the creation an internal somewhat stable > (versioned?) interface that could allow any app server to achieve deep > integration with Cassandra, and as a result, these servers could > 1) host their own apis (REST, for example > 2) extend core functionality by having limited (see triggers and wide row > scanners) access to the internals of cassandra > The hand wavey part comes because while I have been mulling this about for a > while, I have not spent any significant time into looking at the actual > surface area of intravert-ug's integration. But, using it as a model, and > also keeping in minds the general needs of your more traditional servlet/j2ee > containers, I believe we could come up with a reasonable interface to allow > any jvm app server to be integrated and maintained in or out of the Cassandra > tree. > This would satisfy the needs that many of us (Both Ed and I, for example) to > have a much greater degree of control over server side execution, and to be > able to start building much more interestingly (and simply) tiered > applications. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-6846) Provide standard interface for deep application server integration
[ https://issues.apache.org/jira/browse/CASSANDRA-6846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13933619#comment-13933619 ] Tupshin Harper edited comment on CASSANDRA-6846 at 3/13/14 5:49 PM: Consider this comment to be an RFC flipping this ticket on its head, and making it into a meta-ticket for the following: 1) Instead of Cassandra turning into the container, focus on making it easier to embed Cassandra in your own custom application or container. This would become a separate independent JIRA, and could be figured out on its own. 2) Focus additional community efforts on figuring out a good and easy to implement triggers interface to be used as the primary programmatic hook at ingestion time. Another existing or new ticket would be used for that. 3) Focus additional community efforts on defining a proper UDF (use defined function) interface using hive-style UDTFs as primary source of inspiration. This would be the primary programmatic hook for processing results, and could also be used in conjunction with triggers for inbound processing. There are already a few tickets around this area, and we would consolidate on one or create a new one. 4) Define a set of semi-frozen medium level apis (along the lines of what Nate suggested above) that would be stable for different Zs in X.Y.Z releases, but free to change in any larger release. The devil is still in the details of this, but this, too, would be an independent ticket subject to its own debate. As a result, we would be constantly focusing on CQL as the primary (and really only) consumable API coming out of Cassandra (as both triggers and UDFs would be extending CQL). If, however, you needed to do more, that was not possible with just the triggers and UDF interfaces, you would have to instead embed Cassandra and would now be responsible for the life-cycle. With additional power (access to that medium level api), you have incurred the responsibility of being the container, setting up the environment, and managing the overall run-time. That's a serious product design question that should not be taken lightly, but would make sense for a small handful. I would love to hear if any of the above is a non-starter for any reason, or, if it were fully realized, if it would be insufficient for anybody's needs. was (Author: tupshin): Consider this comment to be an RFC flipping this ticket on its head, and making it into a meta-ticket for the following: 1) Instead of Cassandra turning into the container, focus on making it easier to embed Cassandra in your own custom application or container. This would become a separate independent JIRA, and could be figured out on its own. 2) Focus additional community efforts on figuring out a good and easy to implement triggers interface to be used as the primary programmatic hook at ingestion time. Another existing or new ticket would be used for that. 3) Focus additional community efforts on defining a proper UDF (use defined function) interface using hive-style UDTFs as primary source of inspiration. This would be the primary programmatic hook for processing results, and could also be used in conjunction with triggers for inbound processing. There are already a few tickets around this area, and we would consolidate on one or create a new one. 4) Define a set of semi-frozen medium level apis (along the lines of what Nate suggested above) that would be stable for different Zs in X.Y.Z releases, but free to change in any larger release. The devil is still in the details of this, but this, too, would be an independent ticket subject to its own debate. As a result, we would be constantly focusing on CQL as the primary (and really only) consumable API coming out of Cassandra (as both triggers and UTFs would be extending CQL). If, however, you needed to do more, that was not possible with just the triggers and UDF interfaces, you would have to instead embed Cassandra and would now be responsible for the life-cycle. With additional power (access to that medium level api), you have incurred the responsibility of being the container, setting up the environment, and managing the overall run-time. That's a serious product design question that should not be taken lightly, but would make sense for a small handful. I would love to hear if any of the above is a non-starter for any reason, or, if it were fully realized, if it would be insufficient for anybody's needs. > Provide standard interface for deep application server integration > -- > > Key: CASSANDRA-6846 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6846 > Project: Cassandra > Issue Type: New Feature > Components: API, Core >Reporter: Tupshin Harper >
[jira] [Comment Edited] (CASSANDRA-6846) Provide standard interface for deep application server integration
[ https://issues.apache.org/jira/browse/CASSANDRA-6846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13933432#comment-13933432 ] Jonathan Ellis edited comment on CASSANDRA-6846 at 3/13/14 3:48 PM: bq. DSE already does this. (Well at leas this is how brisk worked). Brisk embedded the Hadoop jobtracker and tasktracker for a bunch of reasons that ended up not being worth the trouble. (I'm pretty sure the latest DSE moved it back out of process.) But all the actual functionality was done with the public API: http://github.com/riptano/brisk was (Author: jbellis): bq. DSE already does this. (Well at leas this is how brisk worked). Brisk embedded the Hadoop jobtracker for a bunch of reasons that ended up not being worth the trouble. (I'm pretty sure the latest DSE moved it back out of process.) But all the actual functionality was done with the public API: http://github.com/riptano/brisk > Provide standard interface for deep application server integration > -- > > Key: CASSANDRA-6846 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6846 > Project: Cassandra > Issue Type: New Feature > Components: API, Core >Reporter: Tupshin Harper >Assignee: Tupshin Harper >Priority: Minor > Labels: ponies > > Instead of creating a pluggable interface for Thrift, I'd like to create a > pluggable interface for arbitrary app-server deep integration. > Inspired by both the existence of intravert-ug, as well as there being a long > history of various parties embedding tomcat or jetty servlet engines inside > Cassandra, I'd like to propose the creation an internal somewhat stable > (versioned?) interface that could allow any app server to achieve deep > integration with Cassandra, and as a result, these servers could > 1) host their own apis (REST, for example > 2) extend core functionality by having limited (see triggers and wide row > scanners) access to the internals of cassandra > The hand wavey part comes because while I have been mulling this about for a > while, I have not spent any significant time into looking at the actual > surface area of intravert-ug's integration. But, using it as a model, and > also keeping in minds the general needs of your more traditional servlet/j2ee > containers, I believe we could come up with a reasonable interface to allow > any jvm app server to be integrated and maintained in or out of the Cassandra > tree. > This would satisfy the needs that many of us (Both Ed and I, for example) to > have a much greater degree of control over server side execution, and to be > able to start building much more interestingly (and simply) tiered > applications. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-6846) Provide standard interface for deep application server integration
[ https://issues.apache.org/jira/browse/CASSANDRA-6846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13931951#comment-13931951 ] Russell Bradberry edited comment on CASSANDRA-6846 at 3/12/14 4:34 PM: --- +1 I'd like to take it one step further and even make parts of Cassandra, like CQL, use the interface as well. Something of eating one's own dog food. That way the interface will grow with the features that are added to things like CQL and it won't be a constant battle of "Feature X was added to CQL can we please get it exposed in the interface" was (Author: devdazed): :+1: I'd like to take it one step further and even make parts of Cassandra, like CQL, use the interface as well. Something of eating one's own dog food. That way the interface will grow with the features that are added to things like CQL and it won't be a constant battle of "Feature X was added to CQL can we please get it exposed in the interface" > Provide standard interface for deep application server integration > -- > > Key: CASSANDRA-6846 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6846 > Project: Cassandra > Issue Type: New Feature > Components: API, Core >Reporter: Tupshin Harper >Assignee: Tupshin Harper > Fix For: 3.0 > > > Instead of creating a pluggable interface for Thrift, I'd like to create a > pluggable interface for arbitrary app-server deep integration. > Inspired by both the existence of intravert-ug, as well as there being a long > history of various parties embedding tomcat or jetty servlet engines inside > Cassandra, I'd like to propose the creation an internal somewhat stable > (versioned?) interface that could allow any app server to achieve deep > integration with Cassandra, and as a result, these servers could > 1) host their own apis (REST, for example > 2) extend core functionality by having limited (see triggers and wide row > scanners) access to the internals of cassandra > The hand wavey part comes because while I have been mulling this about for a > while, I have not spent any significant time into looking at the actual > surface area of intravert-ug's integration. But, using it as a model, and > also keeping in minds the general needs of your more traditional servlet/j2ee > containers, I believe we could come up with a reasonable interface to allow > any jvm app server to be integrated and maintained in or out of the Cassandra > tree. > This would satisfy the needs that many of us (Both Ed and I, for example) to > have a much greater degree of control over server side execution, and to be > able to start building much more interestingly (and simply) tiered > applications. -- This message was sent by Atlassian JIRA (v6.2#6252)