[ 
https://issues.apache.org/jira/browse/AIRAVATA-3284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Christie updated AIRAVATA-3284:
--------------------------------------
    Description: 
Each time the CSS/JS code is built for the Django apps, they get created with a 
checksum extension added. For example 
{{/static/django_airavata_workspace/dist/js/dashboard.567f67c4.js}}.

Overtime this accumulate if these files aren't removed. This issue is to update 
the Ansible deploy to clean these up.

Care has to be taken since during a deploy the browsers of current users may be 
requesting the older checksum of the file.

h5. Proposal
- in addition to the current task that rsyncs the entire django portal checkout 
and build, separately rsync each CSS/JS dist directory with it configured to 
delete files on the destination that aren't there in the source. This will 
clean up multiple checksum files in build directory.
- In addition to the build directory, there is also the static/ directory into 
which the collectstatic task collects the built checksum files. This needs to 
be cleaned up as well. This is a little trickier, as mentioned above, because 
you don't want to delete a file here that is needed by a current user during 
the deploy.  Instead we can add a task to delete any files that haven't been 
accessed in, say, more than 30 days.
-- there's a slight chance this can cause an issue if a CSS/JS file hasn't been 
accessed in more than 30 days simply because there hasn't been any traffic to 
the portal in that time. That's why the delete task should run before 
collectstatic runs since collectstatic only copies files that have changed and 
there may be more than 30 days between some of these files having any changes.

h5. TODO

- [x] restore building JS source maps - these were turned off to save some 
space 
https://github.com/apache/airavata-django-portal/commit/8377d7724cb0b7b6c40b05fccf1ad5ab9e79a914
- [ ] sourcemap files are ending up almost 44MB for a single Django portal. 
Probably we don't need  sourcemap for the chunk-vendors, can we exclude it from 
the deploy?


  was:
Each time the CSS/JS code is built for the Django apps, they get created with a 
checksum extension added. For example 
{{/static/django_airavata_workspace/dist/js/dashboard.567f67c4.js}}.

Overtime this accumulate if these files aren't removed. This issue is to update 
the Ansible deploy to clean these up.

Care has to be taken since during a deploy the browsers of current users may be 
requesting the older checksum of the file.

h5. Proposal
- in addition to the current task that rsyncs the entire django portal checkout 
and build, separately rsync each CSS/JS dist directory with it configured to 
delete files on the destination that aren't there in the source. This will 
clean up multiple checksum files in build directory.
- In addition to the build directory, there is also the static/ directory into 
which the collectstatic task collects the built checksum files. This needs to 
be cleaned up as well. This is a little trickier, as mentioned above, because 
you don't want to delete a file here that is needed by a current user during 
the deploy.  Instead we can add a task to delete any files that haven't been 
accessed in, say, more than 30 days.
-- there's a slight chance this can cause an issue if a CSS/JS file hasn't been 
accessed in more than 30 days simply because there hasn't been any traffic to 
the portal in that time. That's why the delete task should run before 
collectstatic runs since collectstatic only copies files that have changed and 
there may be more than 30 days between some of these files having any changes.

h5. TODO

- [x] restore building JS source maps - these were turned off to save some 
space 
https://github.com/apache/airavata-django-portal/commit/8377d7724cb0b7b6c40b05fccf1ad5ab9e79a914




> Ansible: clean up past built CSS/JS files
> -----------------------------------------
>
>                 Key: AIRAVATA-3284
>                 URL: https://issues.apache.org/jira/browse/AIRAVATA-3284
>             Project: Airavata
>          Issue Type: Bug
>          Components: Django Portal
>            Reporter: Marcus Christie
>            Assignee: Marcus Christie
>            Priority: Major
>
> Each time the CSS/JS code is built for the Django apps, they get created with 
> a checksum extension added. For example 
> {{/static/django_airavata_workspace/dist/js/dashboard.567f67c4.js}}.
> Overtime this accumulate if these files aren't removed. This issue is to 
> update the Ansible deploy to clean these up.
> Care has to be taken since during a deploy the browsers of current users may 
> be requesting the older checksum of the file.
> h5. Proposal
> - in addition to the current task that rsyncs the entire django portal 
> checkout and build, separately rsync each CSS/JS dist directory with it 
> configured to delete files on the destination that aren't there in the 
> source. This will clean up multiple checksum files in build directory.
> - In addition to the build directory, there is also the static/ directory 
> into which the collectstatic task collects the built checksum files. This 
> needs to be cleaned up as well. This is a little trickier, as mentioned 
> above, because you don't want to delete a file here that is needed by a 
> current user during the deploy.  Instead we can add a task to delete any 
> files that haven't been accessed in, say, more than 30 days.
> -- there's a slight chance this can cause an issue if a CSS/JS file hasn't 
> been accessed in more than 30 days simply because there hasn't been any 
> traffic to the portal in that time. That's why the delete task should run 
> before collectstatic runs since collectstatic only copies files that have 
> changed and there may be more than 30 days between some of these files having 
> any changes.
> h5. TODO
> - [x] restore building JS source maps - these were turned off to save some 
> space 
> https://github.com/apache/airavata-django-portal/commit/8377d7724cb0b7b6c40b05fccf1ad5ab9e79a914
> - [ ] sourcemap files are ending up almost 44MB for a single Django portal. 
> Probably we don't need  sourcemap for the chunk-vendors, can we exclude it 
> from the deploy?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Reply via email to