Thanks Ryan. We were looking at the Mapreduce code that does the backup and also the Cloud Dataflow code for DatastoreIO. We can do our own custom backups today using a query via dataflow. Only restriction seems to be that the query can't have inequality operators.
However, we are not able to find a way to schedule Dataflow pipelines. Questions to this effect on StackOverflow have gone unanswered. The Dataflow FAQ however has this: Is there a built-in scheduling mechanism to execute pipelines at given time or interval? You can automate pipeline execution through Google App Engine or custom (CRON) job processes on GCE. Future releases of the SDK will support command line options for finer grained control over job management. But there is no documentation on how to automate execution using Appengine or Cron. If you could please help me with this, I'll have a solution. Thanks, Vinuth. On Thu, Jun 18, 2015 at 7:10 PM, Ryan (Cloud Platform Support) < [email protected]> wrote: > I will process your PIT later today. > > Unfortunately there is no way to do incremental backups at this time. > > > On Thursday, June 18, 2015 at 1:02:25 AM UTC-4, Gubbi wrote: >> >> I have created a new feature request here: >> https://code.google.com/p/googleappengine/issues/detail?id=12071 >> >> starred the old feature request as well. >> >> Are there any strategies I can use now for incremental backups? >> >> >> On Wed, Jun 17, 2015 at 12:56 AM, Jason Collins < >> [email protected]> wrote: >> >>> I have an old feature request here if you want to star it: >>> >>> https://code.google.com/p/googleappengine/issues/detail?id=7428 >>> >>> >>> On Tuesday, 16 June 2015 08:37:46 UTC-7, Ryan (Cloud Platform Support) >>> wrote: >>>> >>>> Salutations Gubbi, >>>> >>>> This Feature Request would best be posted in the Public Issue Tracker >>>> <https://code.google.com/p/googleappengine/issues/entry?template=Feature%20request>. >>>> When you fill out the report let me know on this thread and I will process >>>> it for you. >>>> >>>> On Tuesday, June 16, 2015 at 10:53:47 AM UTC-4, Gubbi wrote: >>>>> >>>>> Hi, >>>>> >>>>> The official backup method only supports full backup as of now. I >>>>> don't know when, if at all, it'll support incremental backup. It'll be >>>>> wasteful to take full backups every time. >>>>> >>>>> What strategies are you using for taking incremental backups of your >>>>> datastore? >>>>> >>>>> >>>>> Thanks, >>>>> Vinuth. >>>>> >>>>> >> -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/google-appengine. > To view this discussion on the web visit > https://groups.google.com/d/msgid/google-appengine/06f79b85-be73-4dc4-a774-07dfc6cf9cb0%40googlegroups.com > <https://groups.google.com/d/msgid/google-appengine/06f79b85-be73-4dc4-a774-07dfc6cf9cb0%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/CAEKebv%2BqCR7eH_WupAcorXxN65hNKoZyd75dGY3WqV%2Bdt01_Uw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
