I should have an update on this in next couple of days. Sorry for the delay.
Regards, Raman On Wed, Nov 18, 2015 at 3:27 PM, Mike Carey <[email protected]> wrote: > Any update on this? > > > On 11/2/15 5:54 AM, Raman Grover wrote: > >> Hi, >> >> I should be able to look at this one in a few days when I return and >> resume >> work (I am currently on a vacation till 5th Nov). >> >> However, the exception suggests that building the secondary feed pipeline >> encountered an exception. Can you share the logs so that I can get a >> better >> understanding of the sequence of steps that happened as your statement >> executed. >> >> I have intermittent access to net, but should be able to revert with some >> delays. >> On Nov 2, 2015 2:32 AM, "Jianfeng Jia" <[email protected]> wrote: >> >> I’ve tried the “apply AQLfunction” idea, but got an NullPointException: >>> java.lang.NullPointerException >>> at >>> >>> org.apache.asterix.aql.translator.AqlTranslator.getFeedJointKey(AqlTranslator.java:2268) >>> at >>> >>> org.apache.asterix.aql.translator.AqlTranslator.getFeedConnectionRequest(AqlTranslator.java:2214) >>> at >>> >>> org.apache.asterix.aql.translator.AqlTranslator.handleConnectFeedStatement(AqlTranslator.java:2130) >>> at >>> >>> org.apache.asterix.aql.translator.AqlTranslator.compileAndExecute(AqlTranslator.java:362) >>> at >>> >>> org.apache.asterix.api.http.servlet.APIServlet.doPost(APIServlet.java:114) >>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) >>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) >>> at >>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:546) >>> at >>> >>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:483) >>> at >>> >>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) >>> at >>> >>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:970) >>> at >>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:411) >>> at >>> >>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192) >>> at >>> >>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:904) >>> at >>> >>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) >>> at >>> >>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110) >>> at org.eclipse.jetty.server.Server.handle(Server.java:347) >>> at >>> >>> org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:439) >>> at >>> >>> org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:924) >>> at >>> org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:781) >>> at >>> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:220) >>> at >>> >>> org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:43) >>> at >>> >>> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:545) >>> at >>> >>> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:43) >>> at >>> >>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:529) >>> at java.lang.Thread.run(Thread.java:745) >>> >>> The ddl that I was using is here: >>> >>> https://gist.githubusercontent.com/JavierJia/ca596df82ffdd456001f/raw/ea86f9ad1531a68c3ecf9036ef5b69976893149d/feed-ddl.aql >>> < >>> >>> https://gist.githubusercontent.com/JavierJia/ca596df82ffdd456001f/raw/ea86f9ad1531a68c3ecf9036ef5b69976893149d/feed-ddl.aql >>> Anyone has any idea? Thank you! >>> >>> On Oct 30, 2015, at 12:30 AM, Heri Ramampiaro <[email protected]> wrote: >>>> >>>> Although I haven’t tested this (I have mostly used & created java-based >>>> >>> UDFs), >>> >>>> one can execute AQL calls from within a UDF. Feeds allows functions to >>>> execute arbitrary AQL statements (DDL, DMLs etc). I.e., I believe what >>>> you are trying to do is possible. >>>> >>>> For example you could do: >>>> (Given that you have a rec. type called Tweet, and a dataset >>>> >>> ProcessedTweets >>> >>>> create feed CoordTwitterFeed if not exists >>>> using “push_twitter" (("type-name"="Tweet”)) >>>> apply function find-intersection; >>>> >>>> (Here "find-intersection” is an AQL function that does the step >>>> >>> specified under 2)). >>> >>>> To do nr. 3 the easiest way is to have a modified version of the >>>> >>> “hashTag” >>> >>>> (let’s call this “tweetlocator”). Java-based UDF (see my previous >>>> >>> message with the >>> >>>> TweetLib example). You can then connect this as a secondary feed, >>>> >>> connected to >>> >>>> “CoordTwitterFeed”. >>>> >>>> E.g.: >>>> create secondary feed ProcessedTwitterFeed from feed CoordTwitterFeed >>>> apply function “tweelib#tweetlocator"; >>>> >>>> Thereafter you can connect the feeds to appropriate datasets. >>>> >>>> (Since the find-intersection is an AQL function, you can also call >>>> >>> “tweelib#tweetlocator” >>> >>>> inside this function). >>>> >>>> Best, >>>> -heri >>>> >>>> On Oct 30, 2015, at 12:18 AM, Young-Seok Kim <[email protected]> wrote: >>>>> >>>>> I'm not sure whether the following UDF is possible or not, but >>>>> >>>> hopefully it >>> >>>> is. >>>>> What we're trying to do is to have the following UDF. >>>>> >>>>> The UDF >>>>> 1) accepts an incoming tweet record through a feed job as an input and >>>>> >>>> then >>> >>>> 2) takes a field, more specifically, coordinate field value from the >>>>> >>>> tweet >>> >>>> record and sends a spatial-intersect query using the coordinate in >>>>> >>>> order to >>> >>>> find out the corresponding county of the coordinates (we have created >>>>> AsterixDB instance which stores US county shapes records into a dataset >>>>> >>>> and >>> >>>> created R-tree index on the polygon field of it, so the query will >>>>> >>>> return >>> >>>> the county effectively using the R-tree index) >>>>> 3) creates a new tweet record consisting of the original record's >>>>> >>>> fields + >>> >>>> the returned county value >>>>> 4) ingests to a tweet dataset in the AsterixDB instance. >>>>> >>>>> Can we have such an UDF? >>>>> >>>>> Best, >>>>> Young-Seok >>>>> >>>>> >>>>> On Thu, Oct 29, 2015 at 3:53 PM, Heri Ramampiaro <[email protected]> >>>>> >>>> wrote: >>> >>>> 2. Can we use AQL function in those kind of feed UDFs? >>>>>>> >>>>>> Can you give an example of what you are trying to do? >>>>>> I.e do you want to run an AQL inside a UDF or use an AQL function as a >>>>>> UDF connected to a running feed? >>>>>> >>>>>> -heri >>>>>> >>>>> >>> >>> Best, >>> >>> Jianfeng Jia >>> PhD Candidate of Computer Science >>> University of California, Irvine >>> >>> >>> > -- Raman
