[
http://opencast.jira.com/browse/MH-7602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=26235#comment-26235
]
Adam Hochman commented on MH-7602:
----------------------------------
adhocbot: still make sense? akm220 is there an easy way to confirm that
garbage collection works?
[3:17pm] akm220: we could set the value to really high and see if it will clean
up an old capture
[3:17pm] akm220: I think it is a good plan
[3:17pm] michellezee left the chat room.
[3:17pm] akm220: as long as people use our suggested mpegps default it should
be fine
[3:18pm] adhocbot: _micah claims that the audio issue happens regardless
[3:19pm] adhocbot: unfortunately the config for garbage collection is days.
but it clearly didn't work on centrino
[3:22pm] akm220: oh I was thinking the min disk space one moreso
[3:23pm] adhocbot: ohh...
[3:24pm] greg_logan: the diskspace one is finnicky too unfortunately
[3:24pm] greg_logan: there's a volume of the disk that's reserved, and we can't
see that from within java
[3:24pm] greg_logan: so the minimum disk space needs to be set to 5% of the
disk size, plus whatever buffer you want
[3:24pm] greg_logan: assuming you're using the default disk settings
[3:26pm] akm220: we could make is something huge, like the minimum disk space
is the entire disk, then change a capture to be older than it really is and see
if it deletes it
[3:26pm] greg_logan: that should work
[3:26pm] greg_logan: actually, if either criteria is met it should delete it...
[3:27pm] akm220: does it check the schedule for how old it is or the capture
files on the disk?
[3:27pm] greg_logan: dunno, let's check!
[3:28pm] greg_logan: long age = theRec.getLastCheckinTime();
[3:28pm] greg_logan: long currentTime = System.currentTimeMillis();
[3:28pm] greg_logan: if (currentTime - age > maxArchivalDays *
DAY_LENGTH_MILLIS) {
[3:28pm] greg_logan: so the last time something changed in the recording's state
[3:28pm] greg_logan: sorry
[3:28pm] greg_logan: there *are* unit tests for these things
[3:28pm] greg_logan: and they pass and work last I checked
[3:34pm] adhocbot: so greg_logan akm220 is there anything I can do? what is
the plan for testing this?
[3:36pm] greg_logan: adhocbot: there are two things to test here
[3:36pm] greg_logan: the age, and the disk space checks
[3:37pm] greg_logan: so to check the age I would take a capture, shut down the
agent and then move the clock forward say... a year.
[3:37pm] greg_logan: then bring the agent back up and see if it deletes the
capture
[3:37pm] greg_logan: disk space checking is simpler in that you just need to
change the parameter in the config file and then restart the agent
[3:42pm] adhocbot: so centrino's min diskspace is
capture.cleaner.mindiskspace=536870912. the avail disk space is 63G. So I
should set it close to 63 GB to see if it cleans up the recording?
[3:43pm] kenneth__ left the chat room. (Quit: Leaving)
[3:44pm] greg_logan: akm220: how big is the disk on centrino supposed to be?
[3:44pm] greg_logan: I'm only seeing a 71GB disk on it
[3:44pm] akm220: really, thought it would be 500G
[3:44pm] greg_logan: adhocbot: I would set the free space to the full size of
the drive, so 76235669504
[3:44pm] greg_logan: yeah, me too
[3:44pm] greg_logan: it's almost like this is some kind of VM
[3:45pm] greg_logan: the root drive is on /dev/mapper/centrinoopencast-root
[3:45pm] adhocbot: yup. no idea what that means
[3:45pm] greg_logan: yeah, that's totally a 500GB drive in theer
[3:45pm] greg_logan: weird
[3:46pm] greg_logan: adhocbot: set capture.cleaner.mindiskspace = 76235669504
[3:46pm] greg_logan: and then restart felix
[3:46pm] adhocbot: k.
[3:49pm] colinclark left the chat room. (Quit: colinclark)
[3:53pm] adhocbot: greg_logan I rebooted felix and the status of the recordings
in cache are unchanged
[3:54pm] adhocbot: since the min is set to the max disk space, shouldn't the
cache recordings delete right away after reboot?
[3:54pm] greg_logan: where are you getting the state of the recordings from?
[3:55pm] greg_logan: http://128.233.104.214:8080/state/recordings shows that
the CA isn't tracking any more recordings at all
[3:56pm] adhocbot: I go to the capture cache and ls -lhR. my expectation would
be that at least the media files are deleted. not clear whether the text files
and dir should remain. regardless the media files are still there
[3:58pm] adhocbot: http://pastebin.com/GQpMNeEb
[3:58pm] greg_logan: hrm, so it's removing the recordings from memory but not
removing them from disk
[3:58pm] greg_logan: just a sec, let me start eclipse up
[3:58pm] adhocbot: thx
[4:01pm] greg_logan: it looks like it should be doing the right thing...
[4:01pm] greg_logan: I don't see why it wouldn't be working tbh
[4:01pm] greg_logan: if it's removing them from memory it should be deleting
them at the same time
[4:02pm] adhocbot: bug? perhaps you should look at centrino. maybe I'm f'ing
up the test.
[4:03pm] greg_logan: oh, wait
[4:03pm] greg_logan: no, that doesn't make sense.
[4:03pm] greg_logan: hrm
[4:06pm] greg_logan: ok
[4:06pm] greg_logan: so there's a bug in FileUtil I think
[4:07pm] adhocbot: k. the unit test never covered deleting files on disk I
presume?
[4:08pm] greg_logan: aw fek
[4:08pm] greg_logan: no, it's a logic bug
[4:09pm] adhocbot: so we need a unit test for our unit test?
[4:10pm] greg_logan: no, the unit test works fine
[4:10pm] greg_logan: it's the integration that's wrong
[4:10pm] greg_logan: getKnownRecordings in CAImpl does not return completed
recordings
[4:10pm] greg_logan: soo... they never get cleaned up since the output of
getKnownRecordings is what the cleaner job looks at
[4:11pm] adhocbot: and this impacts the time driven garbage cleaner as well?
[4:11pm] greg_logan: yes
[4:11pm] greg_logan: it would impact all of it
[4:11pm] greg_logan: the cleaner doesn't clean up things that haven't been
ingested
[4:11pm] greg_logan: in effect, it's looking at things it can't ever clean up,
and only at things that it can't ever clean up
[4:13pm] adhocbot: sorry that last statement was a bit confusing. it should
clean up the cached recordings of ingested recordings. but it is not? sorry.
I'm a bit slow today
[4:13pm] greg_logan: the list of things it's being given to clean up doesn't
include the things it's allowed to clean up, only things which are still in
progress and can't be cleaned up.
[4:14pm] adhocbot: gotcha. thx for the help
[4:16pm] akm220: well I think I am out of here for the weekend, have a great
one everyone!
[4:16pm] greg_logan: tty
[4:16pm] adhocbot: bye adam
[4:17pm] akm220 left the chat room.
[4:21pm] greg_logan: adhocbot: I've got a patch for it, but I don't have time
to test it tonight
[4:21pm] greg_logan: have you opened a bug? I can attach it as a patch there
[4:21pm] adhocbot: np. will go into the 1.1.1 main branch
> CA Garbage collection does not delete cached ingested files
> -----------------------------------------------------------
>
> Key: MH-7602
> URL: http://opencast.jira.com/browse/MH-7602
> Project: Matterhorn Project
> Issue Type: Bug
> Components: Capture (Devices and Software)
> Affects Versions: 1.1
> Reporter: Adam Hochman
> Priority: Critical
> Fix For: 1.1.1
>
>
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
_______________________________________________
Matterhorn mailing list
[email protected]
http://lists.opencastproject.org/mailman/listinfo/matterhorn
To unsubscribe please email
[email protected]
_______________________________________________
