To update everyone: this is definitely a bug with this stuff when using FileResource with Tracker (2.23 -> ). I'm currently working on a fix, but it will take some time.
It hopefully won't take long, but as it's a pretty critical issue to have in production you will need to work around it in the mean time. The least intrusive way is to run a cronjob at just before the cleanup task triggers (2:00 server time) and update all fileresource rows in the DB with isassigned=TRUE. Alternately a patch could be applied (comments out a single line to disable the deletion/cleanup task). Let me know if you need any more details. On Mon, Jul 25, 2016 at 2:06 PM, Halvdan Hoem Grelland <[email protected]> wrote: > Ok. > > That task *should* detect *unreferenced* fileResource objects and delete > them. If they are referenced, however (i.e. there is a data value > referencing their UID) they should not be deleted. If the uid in your > example is in fact stored in a data value with a FR data element there is a > bug, if not it works as intended (to prevent orphaned files). > > Could you check if the 'assigned' field of the FR is true after creating a > new event with a FR data elements? If not, there is a problem in the server > side event creation logic. > > There is no way to disable the task (also, you really shouldn't). > > On Mon, Jul 25, 2016 at 1:37 PM, Mike Nelushi <[email protected]> > wrote: > >> Hi Halvdan, >> >> We want the user to upload the file and be able to retrieve it but the >> files are deleted(purged) at 2AM everyday. >> * INFO 2016-07-25 02:00:00,072 'system-process' delete >> org.hisp.dhis.fileresource.FileResource, name: >> PERSONAL_CONSIDERATION_SUBMISSION. Samuel Mur 7910025121081.pdf, uid: >> ko0TqodGcPh (AuditLogUtil.java [taskScheduler-4]) >> * INFO 2016-07-25 02:00:00,082 'system-process' delete >> org.hisp.dhis.fileresource.FileResource, name: Craig Anders Personal >> Consideration.pdf, uid: ShiqhAsXc78 (AuditLogUtil.java [taskScheduler-4]) >> * INFO 2016-07-25 02:00:00,096 'system-process' delete >> org.hisp.dhis.fileresource.FileResource, name: Motivation for Comm - >> Copy.pdf, uid: L1fLHOSqwkM (AuditLogUtil.java [taskScheduler-4]) >> * INFO 2016-07-25 02:00:00,102 'system-process' delete >> org.hisp.dhis.fileresource.FileResource, name: SATA - TOUCH WORLD CUP 2015 >> - SA MIXED TEAM CONFIRMATION.pdf, uid: EWQXKQsCunv (AuditLogUtil.java >> [taskScheduler-4]) >> * INFO 2016-07-25 02:00:00,108 'system-process' delete >> org.hisp.dhis.fileresource.FileResource, name: N_Isaacs_WPTA_July2016.pdf, >> uid: AmoMiCsoSvD (AuditLogUtil.java [taskScheduler-4]) >> *.....* >> >> How do we disable this task from running? >> >> Regards, >> Mike >> >> On Mon, Jul 25, 2016 at 1:27 PM, Halvdan Hoem Grelland <[email protected] >> > wrote: >> >>> This should be fine. Are you experiencing that it doesn't work as >>> expected? (I didn't consider the tracker use case when I wrote 'dataset'. >>> My bad...) >>> >>> Calle: your assumptions are correct. DV to FR is one-to-one. You would >>> need a DE per specific file for your event/dataset, or alternately package >>> multiple files in some archive format and store that instead (you could >>> handle this on the client side). >>> >>> What if you want multiple data value records to reference the same >>>> document [...] >>>> >>> >>> A FR data value is just a data value with a weak reference to a file. >>> Grouping data elements is no different than for straight numeric or text >>> values. That is, there is no 'link' between the fields, but you could use a >>> construct such as a data set (aggr) or programstageinstance/event (tracker) >>> to group DEs. >>> >>> On Mon, Jul 25, 2016 at 1:02 PM, Mike Nelushi <[email protected]> >>> wrote: >>> >>>> Hi Halvdan, >>>> >>>> The following are the steps I'm using: >>>> 1. Save the file on the api/fileResource endpoint, get the UID back, >>>> then save the uid as of a dataValue on the event. Example: >>>> >>>> POST to api/fileResources, get id: “a11111bbbb” back. >>>> 2. PUT the event: >>>> >>>> { >>>> "event" : "<some-event-uid>", >>>> "dataValues" : [ >>>> { >>>> "value" : "a11111bbbb", >>>> "dataElement" : "<some-fileResource-dataelement-uid>" >>>> } >>>> ] >>>> } >>>> >>>> Are these steps correct? >>>> Are the datasets required in these case? If yes, how should I go about >>>> doing this in the tracker module. >>>> >>>> Regards, >>>> Mike >>>> >>>> >>>> >>>> >>>> >>>> On Mon, Jul 25, 2016 at 11:53 AM, Halvdan Hoem Grelland < >>>> [email protected]> wrote: >>>> >>>>> Hi Mike, >>>>> >>>>> As is explained in the docs, the FR must be referenced by a data >>>>> value. If the FR is referenced it will not be deleted. >>>>> >>>>> The FR object is not usable by itself, so the upload to the >>>>> api/*/fileResources is step 1 of 2 in the process of uploading a file (it >>>>> needs to be tied to a DV afterwards). >>>>> >>>>> In practice this means that in order to use FR you need to create data >>>>> elements (and datasets) to contain your files. >>>>> >>>>> Hope that answers your question. If not, please let me know. >>>>> >>>>> On Mon, Jul 25, 2016 at 7:57 AM, Mike Nelushi <[email protected] >>>>> > wrote: >>>>> >>>>>> Hi Halvdan/Morten, >>>>>> >>>>>> There seem to be a cleanup task which is ran at 2AM every 24H which >>>>>> necessary maintenance which removes orphaned files (FileResources) which >>>>>> have been posted to the instance but are marked for deletion as Halvdan >>>>>> explained here >>>>>> <http://dhis-developers-list.1563109.n2.nabble.com/Re-Dhis2-users-Nightly-Batch-Jobs-td7604891.html>. >>>>>> And from File resource constraints explained *here >>>>>> <http://dhis2.github.io/dhis2-docs/2.23/en/developer/html/ch01s12.html#d0e2396> >>>>>> "Any >>>>>> file resources which are in this state and are older than two hours will >>>>>> be >>>>>> marked for deletion and will eventually be purged from the system." * >>>>>> >>>>>> Our new application allows users to upload files and would like to >>>>>> prevent this from happening. >>>>>> How do we stop the system from running this file deletion task or tag >>>>>> the files to not be deleted? >>>>>> >>>>>> Please advice. >>>>>> >>>>>> Regards, >>>>>> Mike >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Halvdan Hoem Grelland >>>>> Software developer, DHIS 2 >>>>> University of Oslo >>>>> http://www.dhis2.org <https://www.dhis2.org/> >>>>> >>>>> >>>> >>> >>> >>> -- >>> Halvdan Hoem Grelland >>> Software developer, DHIS 2 >>> University of Oslo >>> http://www.dhis2.org <https://www.dhis2.org/> >>> >>> >> > > > -- > Halvdan Hoem Grelland > Software developer, DHIS 2 > University of Oslo > http://www.dhis2.org <https://www.dhis2.org/> > > -- Halvdan Hoem Grelland Software developer, DHIS 2 University of Oslo http://www.dhis2.org <https://www.dhis2.org/>
_______________________________________________ Mailing list: https://launchpad.net/~dhis2-devs Post to : [email protected] Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
