Varnish itself can handle gigabytes of cache, because it uses the hard
disk via virtual memory. But the varnish layer is shared by all heroku
apps, and who knows what exactly Heroku is doing to tweak how it
works.
--
You received this message because you are subscribed to the Google Groups
Oh that seem like a very good approach. The Heroku documentation page
seems pretty light, though. Do you have any idea the size of the
Varnish cache? Would it be able to handle gigabytes of cached
information?
On Sep 10, 12:20 pm, marcel mpoi...@gmail.com wrote:
I think the easiest and cheapest
I think the easiest and cheapest solution is to utilize Heroku's
caching layer, Varnish.
http://docs.heroku.com/http-caching
If you set the max-timeout to 1 week then your app only gets hit once
a week... assuming the sitemaps get hit often enough to keep them in
the cache.
--
You received
Can you store the results in memcache or mongo? How much space do all
the sitemap files consume?
--
You received this message because you are subscribed to the Google Groups
Heroku group.
To post to this group, send email to her...@googlegroups.com.
To unsubscribe from this group, send email to
I just checked, and the sitemaps are even bigger than I expected.
Every 1,000 entries in the sitemap seems to take about a meg...which
means the total size is in the gigabyte range. Now the sitemap
protocol allows for gz compressed sitemaps, which reduces the size by
more than 90%, which means
I have a large number of pages that are stored in my database on my
app, that are only accessible via the search engine under normal
circumstances. This is a perfect ordinary use case for making a
sitemap, which is what I've done.
I have a few million of these pages, so I dynamically generate a
