Hi Ikai, That's a fantastic news!
btw wondering if "processing pipeline" architecture is somewhere down the line in the roadmap? I have been working with FAST and OpenPipe, to name a couple, for document processing... and it will be great to have such a feature incorporated. Is something in roadmap yet? Or any visbility on the same? -Gaurav www.mastergaurav.com On Oct 20, 5:36 am, "Ikai Lan (Google)" <[email protected]> wrote: > That's my fault, here's a working link: > > https://spreadsheets.google.com/a/google.com/viewform?hl=en&formkey=d... > > And yes, Java support is coming. > > -- > Ikai Lan > Developer Programs Engineer, Google App Engine > Blogger:http://googleappengine.blogspot.com > Reddit:http://www.reddit.com/r/appengine > Twitter:http://twitter.com/app_engine > > On Tue, Oct 19, 2010 at 5:29 PM, Nickolas Daskalou <[email protected]>wrote: > > > Hi Ikai, > > > I've tried accessing the trusted tester list but I get this permission > > error from Google Docs: > > > We're sorry, <my email address> does not have permission to access this > > spreadsheet. > > > You are signed in as <my email address>, but that email address doesn't > > have permission to access this spreadsheet. (Sign in as a different user or > > request access to this document) > > > where <my email address> is this email address I'm sending from now (it's > > my Google Apps + Google Account email address). > > > Nick > > > On 20 October 2010 11:10, Ikai Lan (Google) > > <[email protected]<ikai.l%[email protected]> > > > wrote: > > >> Hey everyone, > > >> I wanted to announce that we are accepting signups for trusted testers for > >> the Python Matcher API, which is available for local testing in the 1.3.8 > >> SDK. The Matcher API allows developers to take advantage of Google's high > >> performance matching infrastructure. Developers will be able to register a > >> large number of queries for incoming documents to be matched against. The > >> API will then match these queries against numerical and text properties of > >> incoming data at a very high rate. > > >> To better illustrate what the Matcher API can do, let’s pretend you are > >> building a site that notifies users on stock ticker price changes. That is > >> - > >> a user of the site might sign up and register to receive alerts anytime > >> BRK.A is greater than $500 but lower than $525 (by the way, if Berkshire > >> Hathaway is ever in this price range, sell everything you have and buy. > >> Disclosure: I am not a registered financial advisor). Here’s how this might > >> be implemented on App Engine before: > > >> 1. When a user wants to create a new alert, a new AlertCondition entity is > >> created. This entity records the ticker_symbol, min_price, max_price, and > >> email to notify. > > >> 2. On an incoming notification of a stock price change of BRK.A between > >> $500 and $525, we filter AlertCondition entities. Entities that match are > >> returned, and from these entities, we create offline tasks to email each of > >> the users about the price change. > > >> This works decently, given that we don’t have many stock price changes or > >> many alerts in the system. As the number of AlertConditions go up, we will > >> need to change our application to break the queries into multiple pages, or > >> even move them into task queues. Unfortunately for us, stock prices change > >> very frequently, and (we hope) we will have many users. Fetching tens of > >> thousands of Alert Conditions from the datastore can take on the order of > >> seconds, causing the implementation detailed above to be difficult to scale > >> for our expected usage. > > >> How does the Matcher API help us solve this problem? > >> ------------------- > > >> The Matcher API allows us to register a set of queries, then filter > >> incoming documents against these queries in a scalable, high-performance > >> fashion. The type of problem being solved in the stock price notification > >> example is a good example of how the Matcher API can be used. Here’s what > >> we’d have to do in our application using the Matcher API: > > >> 1. When a user wants to create a new alert, we acquire an instance of a > >> Python matcher object and register queries. > > >> 2. On an incoming stock price change, we run the alert against the matcher > >> and try to find all the queries that matched. As queries are found, the > >> matcher API enqueues tasks to process the results offline. Unlike the > >> implementation using the datastore in the earlier example, the Matcher API > >> performs checks in parallel and runs using a completely different service > >> optimized for this use case. A single price change notification could match > >> against hundreds of thousands of queries in the time span of a few seconds. > > >> Let’s show this example in code (also posted here: > >>http://pastie.org/1234174): > > >> # We’re going to call subscribe. Here’s what we’re passing: > >> # dict - this means we are going to match against a Python dictionary. We > >> can also > >> # pass a db.Model type to match against. For instance, StockPrice > >> # “symbol: GOOG AND price > 500 AND price < 525” - this is our query > >> # “ikai:GOOG” - this is the name of our subscription. We’ll use this to > >> map back to our > >> # User. This must be unique, so we are using the User key and > >> ticket combination > >> matcher.subscribe(dict, “symbol: GOOG AND price > 500 AND price < 525”, > >> “ikai:GOOG”) > > >> # When a new stock price update comes in, we create a Python dictionary > >> representing > >> # all the parts we care about > >> change = { "symbol" : "GOOG", "price" : 515 } > > >> matcher.match(change) > > >> # The code above doesn’t execute right away. It makes an API call to > >> Google’s > >> # matcher service, which, upon completion, begins dispatching matches to a > > >> # task queue at the URI path /_ah/matcher. You’ll need to define the task > >> queue handler: > > >> application = webapp.WSGIApplication( > >> [('/_ah/matcher', ChangeNotificationHandler)]) > > >> # You'd define the handler, a web handler for the results: > > >> class ChangeNotificationHandler(webapp.RequestHandler): > >> def post(self): > >> user_ids = self.request.get_all('id') # Returns > >> ['ikai:GOOG'] > >> results_count = self.request.get('results_count') # Returns the > >> total number of results > >> results_offset = self.request.get('results_offset') # Returns 0 > > >> for id in user_ids: > >> user_id, symbol = id.split(":") > >> # now we have user_id and symbol > >> # we’ll use the user_id to find the User and send them an email! > > >> # Note that subscriptions last, by default, 24 hours, so we'll need to > >> create a > >> # cron job that re-registers them. > > >> What makes Matcher API really powerful are the performance > >> characteristics. We can easily return hundreds of thousands of matches in > >> seconds. > > >> Tip of the iceberg > >> ------------------- > > >> It’s possible to filter on many other types data. Here are a few examples > >> of what this API could be used for: > > >> - matching incoming status updates for specific words or phrases (think > >> Google Alerts or Twitter real-time search updates) > >> - creating a real time notification system for location based services > >> like Google Latitude, allowing users to subscribe to their favorite > >> locations for users matching certain criteria > >> - any kind of notification service with a large number of notifications > >> and incoming data > > >> The full API is much more robust than the stock prices example. You can > >> find more documentation here: > > >>http://code.google.com/p/google-app-engine-samples/wiki/AppEngineMatc... > > >> You'll also want to see the sample application here: > > >>http://code.google.com/p/google-app-engine-samples/source/browse/#svn... > > >> Sounds cool, what do I have to do? > >> ------------------- > > >> 1. Start playing around with the Matcher API in your local SDK! > > >> 2. Add yourself to the trusted tester list here: > > >>https://spreadsheets4.google.com/a/google.com/viewform?formkey=dEc5eF... > > >> Check it out and sign up if this is something you can make use of! If you > >> have any questions about what the API can be used for, let us know and > >> we’ll > >> try to answer any questions to may have. > > >> - Ikai, posted on behalf of Bob, Bartek and the Matcher API team > > >> -- > >> You received this message because you are subscribed to the Google Groups > >> "Google App Engine" group. > >> To post to this group, send email to [email protected]. > >> To unsubscribe from this group, send email to > >> [email protected]<google-appengine%[email protected]> > >> . > >> For more options, visit this group at > >>http://groups.google.com/group/google-appengine?hl=en. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]<google-appengine%[email protected]> > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
