Personal opinion: * Task queues are probably better to use from App Engine Standard. They've existing for a long time, and I suspect the RPC API that is used to call App Engine services is pretty efficient. Using Pub/Sub is likely to be slightly worse in terms of overhead per API call, since it will have to go through the HTTP API via URL Fetch.
* Pub/Sub is probably better to be used from anywhere else. The API works from anywhere. A single Pub/Sub topic has nearly no limits, while task queues top out at around 500 messages / second. You can also "replicate" a topic to multiple subscribers, which is useful for some applications. There are some other minor differences, but IMO those are the ones that I would consider. Hope than helps, Evan On Sunday, September 24, 2017 at 1:22:02 PM UTC-4, Alex Buchanan wrote: > > As of today though, what are the advantages to using Pub/Sub over Pull > Queues? For example, are there any time or scale limits to using Pull > Queues that Pub/Sub resolves? > > Thanks, > Alex > > On Friday, February 3, 2017 at 10:30:54 AM UTC-8, Kim Lewandowski wrote: >> >> Hi Joshua -- >> >> It’s a good question, and it has a complicated answer. Broader >> documentation on when to prefer one over the other is in process, and we >> hope that the feature set we expect to evolve in Cloud Tasks (Alpha >> signup here <https://goo.gl/Ya0AZd>) will make the distinctions clearer. >> >> Much of the answer is historical. Task Queues evolved as part of the App >> Engine developer libraries (back in 2009) and was (internally) slated for >> deprecation and replacement with release of Pub/Sub. As you observed: >> Pub/Sub does seem like a strong generalization of Task Queues. When Pub/Sub >> launched, however, we found that it did not meet the needs of a significant >> number of customers. A number of features that we thought weren’t >> particularly import (de-duplication, scheduled delivery, rate management, >> concurrency management, tagged tasks, configurable retries, etc) turned out >> to be critical and not supported in Pub/Sub. >> >> In response to that feedback, we took a step back and tried to define the >> core differences. The conclusion we arrived at is that Pub/Sub is a >> networking and big data product. It is agnostic to both the content and the >> target and focuses on delivering very high throughput capabilities to >> arbitrary handlers. Task Queues is a managed execution product. It is >> intended for running and managing large numbers of explicit commands. We’re >> working to repackage Task Queues as Cloud Tasks and open it up for broader >> integration with the rest of GCP. For example, we want to make it possible >> to enqueue Datastore operations or Cloud Functions for asynchronous >> execution. The idea is to make it as easy as possible to spin up large >> distributed systems without needing to write handlers and managers. >> >> An example of what we are trying to enable is the App Engine mapreduce >> library (written on top of Datastore and Task Queues). Instructions (in >> this case, map or reduce shards) are explicitly encoded and a user-space >> library can capitalize on Task Queues infrastructure to avoid needing any >> management tools or orchestration services. >> >> I hope that’s helpful and feel free to send me more details on your use >> case and we can provide you with more guidance. >> >> -Kim >> Product Manager of Task Queues >> >> On Thursday, February 2, 2017 at 7:34:00 AM UTC-8, Joshua Fox wrote: >>> >>> This Task Queues >>> <https://cloud.google.com/appengine/docs/java/taskqueue/> page says "Tip: >>> In some cases Google Cloud Pub/Sub >>> <https://cloud.google.com/pubsub/overview> is a good alternative to >>> pull queues." >>> >>> Is there a doc that says more about that? >>> >>> - In what cases are Pub/Sub or Task Queues preferable? >>> - Why is PubSub only an alternative for pull task-queues given that both >>> push- and pull-style subscriptions are supported in Pub/Sub? >>> - Since Pub/Sub appears more general, why do Task Queues exist? Is it >>> just the scheduled push feature? Or the specific support for serialization >>> of Task objects? >>> >>> -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/7af4dae4-13ec-4ead-97c3-6c909644b032%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
