Not yet Chen. I was outside of U.S and resumed work past Monday. I have some catching up to do. I think, I can get to steal some cycles tomorrow or early next week.
I will keep you posted. Regards, Raman On Thu, Nov 12, 2015 at 4:38 PM, Chen Li <[email protected]> wrote: > @Raman: did you have a chance to look into test cases of using AQL UDF > in data feed? > > Chen > > On Mon, Nov 2, 2015 at 9:25 AM, Jianfeng Jia <[email protected]> > wrote: > > Great, here is the cc log: > > > > Thank you! > > > > On Nov 2, 2015, at 5:54 AM, Raman Grover <[email protected]> > wrote: > > > > Hi, > > > > I should be able to look at this one in a few days when I return and > resume > > work (I am currently on a vacation till 5th Nov). > > > > However, the exception suggests that building the secondary feed pipeline > > encountered an exception. Can you share the logs so that I can get a > better > > understanding of the sequence of steps that happened as your statement > > executed. > > > > I have intermittent access to net, but should be able to revert with some > > delays. > > On Nov 2, 2015 2:32 AM, "Jianfeng Jia" <[email protected]> wrote: > > > > I’ve tried the “apply AQLfunction” idea, but got an NullPointException: > > java.lang.NullPointerException > > at > > > org.apache.asterix.aql.translator.AqlTranslator.getFeedJointKey(AqlTranslator.java:2268) > > at > > > org.apache.asterix.aql.translator.AqlTranslator.getFeedConnectionRequest(AqlTranslator.java:2214) > > at > > > org.apache.asterix.aql.translator.AqlTranslator.handleConnectFeedStatement(AqlTranslator.java:2130) > > at > > > org.apache.asterix.aql.translator.AqlTranslator.compileAndExecute(AqlTranslator.java:362) > > at > > > org.apache.asterix.api.http.servlet.APIServlet.doPost(APIServlet.java:114) > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) > > at > > org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:546) > > at > > > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:483) > > at > > > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) > > at > > > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:970) > > at > > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:411) > > at > > > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192) > > at > > > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:904) > > at > > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) > > at > > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110) > > at org.eclipse.jetty.server.Server.handle(Server.java:347) > > at > > > org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:439) > > at > > > org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:924) > > at > org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:781) > > at > > org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:220) > > at > > > org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:43) > > at > > > org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:545) > > at > > > org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:43) > > at > > > org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:529) > > at java.lang.Thread.run(Thread.java:745) > > > > The ddl that I was using is here: > > > https://gist.githubusercontent.com/JavierJia/ca596df82ffdd456001f/raw/ea86f9ad1531a68c3ecf9036ef5b69976893149d/feed-ddl.aql > > < > > > https://gist.githubusercontent.com/JavierJia/ca596df82ffdd456001f/raw/ea86f9ad1531a68c3ecf9036ef5b69976893149d/feed-ddl.aql > > > > > > Anyone has any idea? Thank you! > > > > On Oct 30, 2015, at 12:30 AM, Heri Ramampiaro <[email protected]> wrote: > > > > Although I haven’t tested this (I have mostly used & created java-based > > > > UDFs), > > > > one can execute AQL calls from within a UDF. Feeds allows functions to > > execute arbitrary AQL statements (DDL, DMLs etc). I.e., I believe what > > you are trying to do is possible. > > > > For example you could do: > > (Given that you have a rec. type called Tweet, and a dataset > > > > ProcessedTweets > > > > > > create feed CoordTwitterFeed if not exists > > using “push_twitter" (("type-name"="Tweet”)) > > apply function find-intersection; > > > > (Here "find-intersection” is an AQL function that does the step > > > > specified under 2)). > > > > > > To do nr. 3 the easiest way is to have a modified version of the > > > > “hashTag” > > > > (let’s call this “tweetlocator”). Java-based UDF (see my previous > > > > message with the > > > > TweetLib example). You can then connect this as a secondary feed, > > > > connected to > > > > “CoordTwitterFeed”. > > > > E.g.: > > create secondary feed ProcessedTwitterFeed from feed CoordTwitterFeed > > apply function “tweelib#tweetlocator"; > > > > Thereafter you can connect the feeds to appropriate datasets. > > > > (Since the find-intersection is an AQL function, you can also call > > > > “tweelib#tweetlocator” > > > > inside this function). > > > > Best, > > -heri > > > > On Oct 30, 2015, at 12:18 AM, Young-Seok Kim <[email protected]> wrote: > > > > I'm not sure whether the following UDF is possible or not, but > > > > hopefully it > > > > is. > > What we're trying to do is to have the following UDF. > > > > The UDF > > 1) accepts an incoming tweet record through a feed job as an input and > > > > then > > > > 2) takes a field, more specifically, coordinate field value from the > > > > tweet > > > > record and sends a spatial-intersect query using the coordinate in > > > > order to > > > > find out the corresponding county of the coordinates (we have created > > AsterixDB instance which stores US county shapes records into a dataset > > > > and > > > > created R-tree index on the polygon field of it, so the query will > > > > return > > > > the county effectively using the R-tree index) > > 3) creates a new tweet record consisting of the original record's > > > > fields + > > > > the returned county value > > 4) ingests to a tweet dataset in the AsterixDB instance. > > > > Can we have such an UDF? > > > > Best, > > Young-Seok > > > > > > On Thu, Oct 29, 2015 at 3:53 PM, Heri Ramampiaro <[email protected]> > > > > wrote: > > > > > > > > 2. Can we use AQL function in those kind of feed UDFs? > > > > > > Can you give an example of what you are trying to do? > > I.e do you want to run an AQL inside a UDF or use an AQL function as a > > UDF connected to a running feed? > > > > -heri > > > > > > > > > > > > Best, > > > > Jianfeng Jia > > PhD Candidate of Computer Science > > University of California, Irvine > > > > > > > > > > Best, > > > > Jianfeng Jia > > PhD Candidate of Computer Science > > University of California, Irvine > > > > > -- Raman
