Re: [Django] #11260: File based cache not very efficient with large amounts of cached files

[Django]

#11260: File based cache not very efficient with large amounts of cached files
-+-
 Reporter:  anteater_sa  |Owner:  josh
 Type:  Uncategorized|   Status:  closed
Component:  Core (Cache system)  |  Version:  1.1
 Severity:  Normal   |   Resolution:  wontfix
 Keywords:  filebased file   | Triage Stage:  Design
  cache  |  decision needed
Has patch:  1|  Needs documentation:  1
  Needs tests:  1|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by matkoniecz):

 https://code.djangoproject.com/ticket/29994#ticket is related issue that
 proposed explicitly documenting this performance issues

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/069.58a4d183268d399f162e998bf36ca47b%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #11260: File based cache not very efficient with large amounts of cached files

[Django]

#11260: File based cache not very efficient with large amounts of cached files
-+-
 Reporter:  anteater_sa  |Owner:  josh
 Type:  Uncategorized|   Status:  closed
Component:  Core (Cache system)  |  Version:  1.1
 Severity:  Normal   |   Resolution:  wontfix
 Keywords:  filebased file   | Triage Stage:  Design
  cache  |  decision needed
Has patch:  1|  Needs documentation:  1
  Needs tests:  1|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by grantjenks):

 * ui_ux:   => 0
 * type:   => Uncategorized
 * severity:   => Normal
 * easy:   => 0


Comment:

 I had the same problem but a couple more requirements. For future readers,
 I want to mention the DiskCache
 (http://www.grantjenks.com/docs/diskcache/) project. DiskCache is an
 Apache2 licensed disk and file backed cache library, written in pure-
 Python, and compatible with Django. There are no dependencies outside the
 standard library (no managing other processes) and it's efficient enough
 to handle cache evictions during the request/response cycle (no cron job
 necessary).

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/069.6a5a590c0169b0a627ee352b3ca67b48%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #11260: File based cache not very efficient with large amounts of cached files

[Django]

#11260: File based cache not very efficient with large amounts of cached files
-+--
  Reporter:  anteater_sa | Owner:  josh 
   
Status:  closed  | Milestone:  1.2  
   
 Component:  Cache system|   Version:  1.1  
   
Resolution:  wontfix |  Keywords:  filebased file 
cache
 Stage:  Design decision needed  | Has_patch:  1
   
Needs_docs:  1   |   Needs_tests:  1
   
Needs_better_patch:  0   |  
-+--
Changes (by anteater_sa):

  * version:  1.0 => 1.1

Comment:

 I have Django sites of tens of thousands of pages running for over 2 years
 using the above patches, so your statements about filesystem caching not a
 serious strategy are irrelevant. Also, filesystem caching is not
 comparable to memcaching, they solve two completely different problems.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-upda...@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #11260: File based cache not very efficient with large amounts of cached files

[Django]

#11260: File based cache not very efficient with large amounts of cached files
-+--
  Reporter:  anteater_sa | Owner:  josh 
   
Status:  closed  | Milestone:  1.2  
   
 Component:  Cache system|   Version:  1.0  
   
Resolution:  wontfix |  Keywords:  filebased file 
cache
 Stage:  Design decision needed  | Has_patch:  1
   
Needs_docs:  1   |   Needs_tests:  1
   
Needs_better_patch:  0   |  
-+--
Comment (by JohnMoylan):

 Are you saying that file based cache is not suitable for production? File
 based cache is more suitable for some scenarios than memcache.

 I use file caching to cache processed JPG's. Memcache is not as capable
 for such a scenario.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-upda...@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #11260: File based cache not very efficient with large amounts of cached files

[Django]

#11260: File based cache not very efficient with large amounts of cached files
-+--
  Reporter:  anteater_sa | Owner:  josh 
   
Status:  closed  | Milestone:  1.2  
   
 Component:  Cache system|   Version:  1.0  
   
Resolution:  wontfix |  Keywords:  filebased file 
cache
 Stage:  Design decision needed  | Has_patch:  1
   
Needs_docs:  1   |   Needs_tests:  1
   
Needs_better_patch:  0   |  
-+--
Changes (by russellm):

  * status:  assigned => closed
  * resolution:  => wontfix

Comment:

 I'm going to wontfix, on the grounds that the filesystem cache is intended
 as an easy way to test caching, not as a serious caching strategy. The
 default cache size and the cull strategy implemented by the file cache
 should make that obvious.

 If you need a cache capable of holding 10 items, I strongly recommend
 you look at memcache. If you insist on using the filesystem as a cache, it
 isn't hard to subclass and extend the existing cache.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-upda...@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #11260: File based cache not very efficient with large amounts of cached files

[Django]

#11260: File based cache not very efficient with large amounts of cached files
-+--
  Reporter:  anteater_sa | Owner:  josh 
   
Status:  assigned| Milestone:  1.2  
   
 Component:  Cache system|   Version:  1.0  
   
Resolution:  |  Keywords:  filebased file 
cache
 Stage:  Design decision needed  | Has_patch:  1
   
Needs_docs:  1   |   Needs_tests:  1
   
Needs_better_patch:  0   |  
-+--
Comment (by josh):

 Just to explain the patch...

 If you pass 'max_entries=0' no culling of the cache will ever occur.
 If number of entries is less than 'max_entries' then culling will be
 performed.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

--

You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-upda...@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #11260: File based cache not very efficient with large amounts of cached files

[Django]

#11260: File based cache not very efficient with large amounts of cached files
-+--
  Reporter:  anteater_sa | Owner:  josh 
   
Status:  assigned| Milestone:  1.2  
   
 Component:  Cache system|   Version:  1.0  
   
Resolution:  |  Keywords:  filebased file 
cache
 Stage:  Design decision needed  | Has_patch:  1
   
Needs_docs:  1   |   Needs_tests:  1
   
Needs_better_patch:  0   |  
-+--
Changes (by josh):

  * owner:  nobody => josh
  * status:  new => assigned
  * has_patch:  0 => 1

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

--

You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-upda...@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #11260: File based cache not very efficient with large amounts of cached files

[Django]

#11260: File based cache not very efficient with large amounts of cached files
-+--
  Reporter:  anteater_sa | Owner:  nobody   
   
Status:  new | Milestone:  1.2  
   
 Component:  Cache system|   Version:  1.0  
   
Resolution:  |  Keywords:  filebased file 
cache
 Stage:  Design decision needed  | Has_patch:  0
   
Needs_docs:  1   |   Needs_tests:  1
   
Needs_better_patch:  0   |  
-+--
Changes (by JohnMoylan):

 * cc: j...@nuatech.net (added)
  * has_patch:  1 => 0

Comment:

 I've been bitten by this also.
 My fix is to return count while it is still 0 and manage the cache myself
 using a daily cron job.

 It would be nice if Django allowed me to disable the filebased cache
 management feature using settings.py.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en
-~--~~~~--~~--~--~---

Re: [Django] #11260: File based cache not very efficient with large amounts of cached files

[Django]

#11260: File based cache not very efficient with large amounts of cached files
-+--
  Reporter:  anteater_sa | Owner:  nobody   
   
Status:  new | Milestone:  1.2  
   
 Component:  Cache system|   Version:  1.0  
   
Resolution:  |  Keywords:  filebased file 
cache
 Stage:  Design decision needed  | Has_patch:  1
   
Needs_docs:  1   |   Needs_tests:  1
   
Needs_better_patch:  0   |  
-+--
Changes (by Alex):

  * stage:  Unreviewed => Design decision needed

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en
-~--~~~~--~~--~--~---

Re: [Django] #11260: File based cache not very efficient with large amounts of cached files

[Django]

#11260: File based cache not very efficient with large amounts of cached files
---+
  Reporter:  anteater_sa   | Owner:  nobody  
Status:  new   | Milestone:  1.2 
 Component:  Cache system  |   Version:  1.0 
Resolution:|  Keywords:  filebased file cache
 Stage:  Unreviewed| Has_patch:  1   
Needs_docs:  1 |   Needs_tests:  1   
Needs_better_patch:  0 |  
---+
Changes (by dc):

  * needs_docs:  0 => 1
  * needs_tests:  0 => 1
  * milestone:  1.1 => 1.2

Comment:

 1.1 is at a feature freeze right now so moving to 1.2.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en
-~--~~~~--~~--~--~---

Re: [Django] #11260: File based cache not very efficient with large amounts of cached files

[Django]

#11260: File based cache not very efficient with large amounts of cached files
---+
  Reporter:  anteater_sa   | Owner:  nobody  
Status:  new   | Milestone:  1.1 
 Component:  Cache system  |   Version:  1.0 
Resolution:|  Keywords:  filebased file cache
 Stage:  Unreviewed| Has_patch:  1   
Needs_docs:  0 |   Needs_tests:  0   
Needs_better_patch:  0 |  
---+
Changes (by anteater_sa):

  * needs_better_patch:  => 0
  * needs_tests:  => 0
  * needs_docs:  => 0

Comment:

 sorry, should be as follows:

 {{{
 def _get_num_entries(self):
 count = 0
 if max_entries == 0: return count # Don't count files if
 max_entries is set to 0
 for _,_,files in os.walk(self._dir):
 count += len(files)
 return count
 _num_entries = property(_get_num_entries)
 }}}

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en
-~--~~~~--~~--~--~---