[Mahara-contributors] [Bug 1776364] Re: External file system image infinite loop
Companion update to objectfs to accompany this for testing: https://github.com/catalyst/mahara-module_objectfs/tree/issue-8 -- You received this bug notification because you are a member of Mahara Contributors, which is subscribed to Mahara. Matching subscriptions: Subscription for all Mahara Contributors -- please ask on #mahara-dev or mahara.org forum before editing or unsubscribing it! https://bugs.launchpad.net/bugs/1776364 Title: External file system image infinite loop Status in Mahara: New Bug description: When the objectfs plugin is being used as an external file system, an image is uploaded and that image is no longer locally available the following infinite loop occurs: mahara_1 | PHP message: PHP 34291. module_objectfs\object_file_system->is_file_readable_locally() /siteroot/htdocs/module/objectfs/classes/object_file_system.php:141 mahara_1 | PHP message: PHP 34292. ArtefactTypeImage->get_local_path() /siteroot/htdocs/module/objectfs/classes/object_file_system.php:122 mahara_1 | PHP message: PHP 34293. get_dataroot_image_path() /siteroot/htdocs/artefact/file/lib.php:2391 mahara_1 | PHP message: PHP 34294. ArtefactTypeFile->ensure_local() /siteroot/htdocs/lib/file.php:542 mahara_1 | PHP message: PHP 34295. module_objectfs\mahara_external_filesystem->ensure_local() /siteroot/htdocs/artefact/file/lib.php:1147 mahara_1 | PHP message: PHP 34296. module_objectfs\mahara_external_filesystem->get_file_location_status() /siteroot/htdocs/module/objectfs/classes/mahara_external_filesystem.php:64 mahara_1 | PHP message: PHP 34297. module_objectfs\object_file_system->get_object_location() /siteroot/htdocs/module/objectfs/classes/mahara_external_filesystem.php:87 mahara_1 | PHP message: PHP 34298. module_objectfs\object_file_system->is_file_readable_locally() /siteroot/htdocs/module/objectfs/classes/object_file_system.php:141 It's my opinion this is not an issue with the plugin itself, but the core hooks. It is expected that a FS plugin will call ArtefactTypeImage->get_local_path within its mahara_external_filesystem->ensure_local. Reproduction steps (reproduced with 1804): 1. Fresh install mahara 2. Install https://github.com/catalyst/mahara-module_objectfs and https://github.com/catalyst/mahara-module_aws 3. Configure the plugin with test s3 crendentials. 4. Set: Delete local objects = true Minimum size threshold = 0 Minimum age = 0 Consistancy delay = 0 5. Put this line in config.php: $cfg->externalfilesystem = array( "includefilepath" => "module/objectfs/classes/s3_file_system.php", "class" => "module_objectfs\\s3_file_system" ); 6. Goto /artefact/file/index.php 7. Upload an image 8. php htdocs/module/objectfs/cli/run_pusher.php 9. php htdocs/module/objectfs/cli/run_deleter.php // The file will now be in s3 only 10. Refresh /artefact/file/index.php and view error To manage notifications about this bug go to: https://bugs.launchpad.net/mahara/+bug/1776364/+subscriptions ___ Mailing list: https://launchpad.net/~mahara-contributors Post to : mahara-contributors@lists.launchpad.net Unsubscribe : https://launchpad.net/~mahara-contributors More help : https://help.launchpad.net/ListHelp
[Mahara-contributors] [Bug 1776364] Re: External file system image infinite loop
Hi all, Added a possible patch to fix this. It works by adding a generateifpossible flag to get_local_path which is used within get_dataroot_image_path to gate ensure_local. External FS's can then call get_local_path with this flag set to false to avoid calling any of the generating code. Additionally get_local_path will return the local FS path even if it is not readable in this case. The ideal solution would probably involve decoupling the actual path retrieval and image generation in get_dataroot_image_path, however as a non-mahara developer I was not comfortable with the code footprint that would be required to do so. Thanks, KC -- You received this bug notification because you are a member of Mahara Contributors, which is subscribed to Mahara. Matching subscriptions: Subscription for all Mahara Contributors -- please ask on #mahara-dev or mahara.org forum before editing or unsubscribing it! https://bugs.launchpad.net/bugs/1776364 Title: External file system image infinite loop Status in Mahara: New Bug description: When the objectfs plugin is being used as an external file system, an image is uploaded and that image is no longer locally available the following infinite loop occurs: mahara_1 | PHP message: PHP 34291. module_objectfs\object_file_system->is_file_readable_locally() /siteroot/htdocs/module/objectfs/classes/object_file_system.php:141 mahara_1 | PHP message: PHP 34292. ArtefactTypeImage->get_local_path() /siteroot/htdocs/module/objectfs/classes/object_file_system.php:122 mahara_1 | PHP message: PHP 34293. get_dataroot_image_path() /siteroot/htdocs/artefact/file/lib.php:2391 mahara_1 | PHP message: PHP 34294. ArtefactTypeFile->ensure_local() /siteroot/htdocs/lib/file.php:542 mahara_1 | PHP message: PHP 34295. module_objectfs\mahara_external_filesystem->ensure_local() /siteroot/htdocs/artefact/file/lib.php:1147 mahara_1 | PHP message: PHP 34296. module_objectfs\mahara_external_filesystem->get_file_location_status() /siteroot/htdocs/module/objectfs/classes/mahara_external_filesystem.php:64 mahara_1 | PHP message: PHP 34297. module_objectfs\object_file_system->get_object_location() /siteroot/htdocs/module/objectfs/classes/mahara_external_filesystem.php:87 mahara_1 | PHP message: PHP 34298. module_objectfs\object_file_system->is_file_readable_locally() /siteroot/htdocs/module/objectfs/classes/object_file_system.php:141 It's my opinion this is not an issue with the plugin itself, but the core hooks. It is expected that a FS plugin will call ArtefactTypeImage->get_local_path within its mahara_external_filesystem->ensure_local. Reproduction steps (reproduced with 1804): 1. Fresh install mahara 2. Install https://github.com/catalyst/mahara-module_objectfs and https://github.com/catalyst/mahara-module_aws 3. Configure the plugin with test s3 crendentials. 4. Set: Delete local objects = true Minimum size threshold = 0 Minimum age = 0 Consistancy delay = 0 5. Put this line in config.php: $cfg->externalfilesystem = array( "includefilepath" => "module/objectfs/classes/s3_file_system.php", "class" => "module_objectfs\\s3_file_system" ); 6. Goto /artefact/file/index.php 7. Upload an image 8. php htdocs/module/objectfs/cli/run_pusher.php 9. php htdocs/module/objectfs/cli/run_deleter.php // The file will now be in s3 only 10. Refresh /artefact/file/index.php and view error To manage notifications about this bug go to: https://bugs.launchpad.net/mahara/+bug/1776364/+subscriptions ___ Mailing list: https://launchpad.net/~mahara-contributors Post to : mahara-contributors@lists.launchpad.net Unsubscribe : https://launchpad.net/~mahara-contributors More help : https://help.launchpad.net/ListHelp
[Mahara-contributors] [Bug 1776364] [NEW] External file system image infinite loop
Public bug reported: When the objectfs plugin is being used as an external file system, an image is uploaded and that image is no longer locally available the following infinite loop occurs: mahara_1 | PHP message: PHP 34291. module_objectfs\object_file_system->is_file_readable_locally() /siteroot/htdocs/module/objectfs/classes/object_file_system.php:141 mahara_1 | PHP message: PHP 34292. ArtefactTypeImage->get_local_path() /siteroot/htdocs/module/objectfs/classes/object_file_system.php:122 mahara_1 | PHP message: PHP 34293. get_dataroot_image_path() /siteroot/htdocs/artefact/file/lib.php:2391 mahara_1 | PHP message: PHP 34294. ArtefactTypeFile->ensure_local() /siteroot/htdocs/lib/file.php:542 mahara_1 | PHP message: PHP 34295. module_objectfs\mahara_external_filesystem->ensure_local() /siteroot/htdocs/artefact/file/lib.php:1147 mahara_1 | PHP message: PHP 34296. module_objectfs\mahara_external_filesystem->get_file_location_status() /siteroot/htdocs/module/objectfs/classes/mahara_external_filesystem.php:64 mahara_1 | PHP message: PHP 34297. module_objectfs\object_file_system->get_object_location() /siteroot/htdocs/module/objectfs/classes/mahara_external_filesystem.php:87 mahara_1 | PHP message: PHP 34298. module_objectfs\object_file_system->is_file_readable_locally() /siteroot/htdocs/module/objectfs/classes/object_file_system.php:141 It's my opinion this is not an issue with the plugin itself, but the core hooks. It is expected that a FS plugin will call ArtefactTypeImage->get_local_path within its mahara_external_filesystem->ensure_local. Reproduction steps (reproduced with 1804): 1. Fresh install mahara 2. Install https://github.com/catalyst/mahara-module_objectfs and https://github.com/catalyst/mahara-module_aws 3. Configure the plugin with test s3 crendentials. 4. Set: Delete local objects = true Minimum size threshold = 0 Minimum age = 0 Consistancy delay = 0 5. Put this line in config.php: $cfg->externalfilesystem = array( "includefilepath" => "module/objectfs/classes/s3_file_system.php", "class" => "module_objectfs\\s3_file_system" ); 6. Goto /artefact/file/index.php 7. Upload an image 8. php htdocs/module/objectfs/cli/run_pusher.php 9. php htdocs/module/objectfs/cli/run_deleter.php // The file will now be in s3 only 10. Refresh /artefact/file/index.php and view error ** Affects: mahara Importance: Undecided Status: New -- You received this bug notification because you are a member of Mahara Contributors, which is subscribed to Mahara. Matching subscriptions: Subscription for all Mahara Contributors -- please ask on #mahara-dev or mahara.org forum before editing or unsubscribing it! https://bugs.launchpad.net/bugs/1776364 Title: External file system image infinite loop Status in Mahara: New Bug description: When the objectfs plugin is being used as an external file system, an image is uploaded and that image is no longer locally available the following infinite loop occurs: mahara_1 | PHP message: PHP 34291. module_objectfs\object_file_system->is_file_readable_locally() /siteroot/htdocs/module/objectfs/classes/object_file_system.php:141 mahara_1 | PHP message: PHP 34292. ArtefactTypeImage->get_local_path() /siteroot/htdocs/module/objectfs/classes/object_file_system.php:122 mahara_1 | PHP message: PHP 34293. get_dataroot_image_path() /siteroot/htdocs/artefact/file/lib.php:2391 mahara_1 | PHP message: PHP 34294. ArtefactTypeFile->ensure_local() /siteroot/htdocs/lib/file.php:542 mahara_1 | PHP message: PHP 34295. module_objectfs\mahara_external_filesystem->ensure_local() /siteroot/htdocs/artefact/file/lib.php:1147 mahara_1 | PHP message: PHP 34296. module_objectfs\mahara_external_filesystem->get_file_location_status() /siteroot/htdocs/module/objectfs/classes/mahara_external_filesystem.php:64 mahara_1 | PHP message: PHP 34297. module_objectfs\object_file_system->get_object_location() /siteroot/htdocs/module/objectfs/classes/mahara_external_filesystem.php:87 mahara_1 | PHP message: PHP 34298. module_objectfs\object_file_system->is_file_readable_locally() /siteroot/htdocs/module/objectfs/classes/object_file_system.php:141 It's my opinion this is not an issue with the plugin itself, but the core hooks. It is expected that a FS plugin will call ArtefactTypeImage->get_local_path within its mahara_external_filesystem->ensure_local. Reproduction steps (reproduced with 1804): 1. Fresh install mahara 2. Install https://github.com/catalyst/mahara-module_objectfs and https://github.com/catalyst/mahara-module_aws 3. Configure the plugin with test s3 crendentials. 4. Set: Delete local objects = true Minimum size threshold = 0 Minimum age = 0 Consistancy delay = 0 5. Put this line in config.php: $cfg->externalfilesystem = array( "includefilepath" =>
[Mahara-contributors] [Bug 1680633] [NEW] Cron job metadata
Public bug reported:
Hello,
Currently the meta data stored for cron jobs is fairly limited having only the
following fields:
We are implementing a plugin for Mahara that will mirror the cron check
functionality present in this Moodle plugin:
https://github.com/brendanheywood/moodle-tool_heartbeat.
It would be beneficial if Mahara included two new fields in the cron and
the plugintype_cron tables:
1. lastruntime - the last runtime of the cron job.
Currently on /admin/statistics.php the following sql is used to determine if
cron has run.
record_exists_select('cron', 'nextrun IS NULL OR nextrun < CURRENT_DATE') -
from a monitoring standpoint this is not acceptable as it will give false
positives/negatives depending on the situation. With this change, the max
lastruntime could be used to determine if cron had run recently.
2. faildelay - An Exponentially increasing fail delay that is updated
when the cron job fails (if it throws an exception). This way if there
is a consistently failing cron job its execution is throttled and its
failure is being tracked in some way aside from just logging an error.
The heartbeat plugin above uses this to help us to identify and fix
issues that would otherwise go undetected.
Aside from the benefits from a monitoring perspective, these fields
provide useful information when it comes to debugging anything cron
related.
I'm happy to discuss at any time and more than happy to do the dev on
this. Ill be floating around the Mahara IRC under the kenH handle if you
would like to talk at any time.
Thanks,
Kenneth
** Affects: mahara
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Mahara
Contributors, which is subscribed to Mahara.
Matching subscriptions: Subscription for all Mahara Contributors -- please ask
on #mahara-dev or mahara.org forum before editing or unsubscribing it!
https://bugs.launchpad.net/bugs/1680633
Title:
Cron job metadata
Status in Mahara:
New
Bug description:
Hello,
Currently the meta data stored for cron jobs is fairly limited having only
the following fields:
We are implementing a plugin for Mahara that will mirror the cron
check functionality present in this Moodle plugin:
https://github.com/brendanheywood/moodle-tool_heartbeat.
It would be beneficial if Mahara included two new fields in the cron
and the plugintype_cron tables:
1. lastruntime - the last runtime of the cron job.
Currently on /admin/statistics.php the following sql is used to determine if
cron has run.
record_exists_select('cron', 'nextrun IS NULL OR nextrun < CURRENT_DATE') -
from a monitoring standpoint this is not acceptable as it will give false
positives/negatives depending on the situation. With this change, the max
lastruntime could be used to determine if cron had run recently.
2. faildelay - An Exponentially increasing fail delay that is updated
when the cron job fails (if it throws an exception). This way if there
is a consistently failing cron job its execution is throttled and its
failure is being tracked in some way aside from just logging an error.
The heartbeat plugin above uses this to help us to identify and fix
issues that would otherwise go undetected.
Aside from the benefits from a monitoring perspective, these fields
provide useful information when it comes to debugging anything cron
related.
I'm happy to discuss at any time and more than happy to do the dev on
this. Ill be floating around the Mahara IRC under the kenH handle if
you would like to talk at any time.
Thanks,
Kenneth
To manage notifications about this bug go to:
https://bugs.launchpad.net/mahara/+bug/1680633/+subscriptions
___
Mailing list: https://launchpad.net/~mahara-contributors
Post to : mahara-contributors@lists.launchpad.net
Unsubscribe : https://launchpad.net/~mahara-contributors
More help : https://help.launchpad.net/ListHelp
