Ah, c'mon now you are giving away the ending! ;) We are looking into this as a possibility. BigQuery has some awesome capabilities and paired with logs its quite effective!
Greg On Wed, Apr 18, 2012 at 10:13 AM, Jason Collins <[email protected]>wrote: > Thanks Greg - it totally makes sense and explains why something that > is even quite recent does not appear on a "high-rate" application. > > Proppy just demo'd a BigQuery-based log analysis in the App Engine > Chat - you should just go straight to that solution under the hood... > > j > > On Apr 18, 12:45 am, "Gregory D'alesandre" <[email protected]> wrote: > > Hey Jason, > > > > Please do fill out an issue but this is indeed a known issue. We don't > do > > currently index for regex and labels so when you do a search across your > > records it will take quite a while and we stop and return whatever > results > > we've found after 10 seconds. We are looking to improve this at some > point > > but it does mean right now you'll get inconsistent results when trying to > > search across a lot of logs. > > > > Hope that makes sense! > > > > Greg > > > > > > > > > > > > > > > > On Tue, Apr 17, 2012 at 9:59 PM, Takashi Matsuo <[email protected]> > wrote: > > > Hi Jason, > > > > > Do you mind filing an issue on the issue tracker with some details? > > > (e.g. app-id, example queries, etc) > > > > > If you mind making the info public, you can also send it directly to > me. > > > > > Thanks, > > > > > -- Takashi > > > > > On Wed, Apr 18, 2012 at 7:33 AM, Jason Collins > > > <[email protected]> wrote: > > > > Is it just me, or does the Log Viewer searching basically not work? > > > > > > I have my log storage configured to give me about 11 days of logs, > but > > > > I can rarely actually find stuff (that I _know_ occurred within my > > > > window) by searching - both regex- and label-based searching. We run > > > > at about 100+QPS, so trying to catch something in action is basically > > > > impossible. > > > > > > I'm guessing the log indexing runs async to the log entries > > > > themselves, and it's just not able to keep pace, or the indexing jobs > > > > themselves are just failing. > > > > > > Does anyone actually have any luck searching the logs in the Log > > > > Viewer on a reasonably high rate application? > > > > > > j > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > Groups "Google App Engine" group. > > > > To post to this group, send email to > [email protected]. > > > > To unsubscribe from this group, send email to > > > [email protected]. > > > > For more options, visit this group at > > >http://groups.google.com/group/google-appengine?hl=en. > > > > > -- > > > Takashi Matsuo | Developer Advocate | [email protected] | > 03-6384-9224 > > > > > -- > > > You received this message because you are subscribed to the Google > Groups > > > "Google App Engine" group. > > > To post to this group, send email to [email protected] > . > > > To unsubscribe from this group, send email to > > > [email protected]. > > > For more options, visit this group at > > >http://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
