I did 2 screencasts to show the difference: 1) slow (supervisor/nginx/wsgi): https://www.dropbox.com/s/4cfdvs7wo7buiit/mayan-slow-supervisor.mp4?dl=0 2) fast (runserver): https://www.dropbox.com/s/pdpildht80ihw2v/mayan-runserver.mp4?dl=0
Am Mittwoch, 19. April 2017 18:00:44 UTC+2 schrieb Roberto Rosario: > > Hi Sebastian, > > I'm drawing a blank because it is the same process either way. For example > when running using supervisor and using a background worker you can check > the output of the worker servicing the converter queue and the same output > about `generate_image()` should be displayed. You might need to edit the > supervisor line of the worker to add -l DEBUG instead of -l INFO to > increase the verbosity. This means that the same logic is being applied to > the generation of thumbnails. > > The only thing that comes to mind is that the speed of the message queue > is creating an overhead that makes it looks like the thumbnails are being > generated. > > The process is as follows: > > Frontend will always ask the document model to see if a cached > representation (image + transformation[resize, rotation]) exists. The > thumbnail generation request travels from the frontend to the celery then > to the broker (Redis or Rabbit), then the worker's celery pickups the > message from the broker checks to see if a thumbnail is available, > generates one if not, sends back the filename of the generated of cached > thumbnail back to the worker's celery to the result storage (Redis), the > frontend's celery gets the message, decodes it and passes it to the waiting > frontend display code. > > In DEBUG mode this happens very fast because it is the same process > running both sides of the code (frontend and model+converter, minus the > broker and result storage). > > My guess is that there is a bottleneck in the deployment configuration, > broker getting choked, not enough workers, worker not running in multi > threaded or multi process mode (servicing messages sequentially), uWSGI > process servicinong HTTP requests sequentially and thus showing thumbnails > synchronously. > > Are the thumbnails being displayed in sequence with one displaying only > after the previous one is displayed? Or are they being displayed in random > order only slow? > > On Tuesday, April 18, 2017 at 8:38:57 AM UTC-4, Sebastian Tänzer wrote: >> >> Hello Roberto, >> >> sorry for the late response and thanks for the info. I upgraded my >> virtualenv/pip setup to the latest git development branch and did some >> tests. >> >> When running Mayan in the foreground using runserver and Debug=True it >> shows this: >> >> documents.models <24377> [DEBUG] "generate_image() transformations cache >> filename: >> page-cache-bd7290ba-4f11-4b1f-ad1d-5a69fdd9a619-77-163-1d20f688585d3c09" >> documents.models <24377> [DEBUG] "generate_image() transformations cache >> file >> "page-cache-bd7290ba-4f11-4b1f-ad1d-5a69fdd9a619-77-163-1d20f688585d3c09" >> found" >> >> and thumbnails are beeing generated really fast and then loaded from >> cache the second time. >> >> When running Mayan using supervisor/nginx this is not the case. All >> thumbs are regenerated everytime. >> >> Any ideas why that is? >> >> Best >> Sebastian >> >> Am Montag, 20. März 2017 06:43:56 UTC+1 schrieb Roberto Rosario: >>> >>> Nothing is needed to use the new thumbnail caching. Enable the DEBUG >>> mode by adding this to you settings/local.py file >>> >>> DEBUG=True >>> >>> Go to a view that shows document images and take a look at the debug >>> messages in the console. You should entries like this: >>> >>> documents.models <8408> [DEBUG] "generate_image() transformations cache >>> file >>> "page-cache-bea37f94-57c4-4789-a5b3-f501dc678fcd-24-55-118690ab21ab21be" >>> found" >>> >>> The long filename is the page UUID followed by the transformations if >>> serialized format (zoom, resize, rotation). This means that the page images >>> are being cached in their >>> final resized dimensions. The system still uses a background process to >>> check is a document thumbnail is already available and this causes a slight >>> overhead (not realtime). >>> If runing using the `runserver` command the task broker is disabled and >>> all background tasks run in the same process as the UI, this will cause the >>> appearance that the >>> thumbnails are being generated from scratch. A screen refresh should >>> show that they are not being regenerated and will display faster than the >>> first time they displayed, >>> this should indicate that they are being cached. With this in mind give >>> it another try to see a difference. You can also delete all cached images >>> using the "Clear document image cache" >>> button in the Tools menu or by manually deleting all file in the >>> mayan/media/document_cache folder and doing a view refresh. The time to >>> display the thumbnails should be very >>> different in the first and subsequent refreshes. >>> >>> >>> >>> >>> On Friday, March 17, 2017 at 8:59:04 AM UTC-4, Sebastian Tänzer wrote: >>>> >>>> I managed to perform an upgrade to 2.2b2. >>>> >>>> Is there anything I have to do to use the new thumbnail caching or >>>> isn't that implemented yet? >>>> Thumbnails are still generated with every page call. >>>> >>>> Best >>>> Sebastian >>>> >>>> -- --- You received this message because you are subscribed to the Google Groups "Mayan EDMS" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
