RE: Spurious write permission error

[C E Macfarlane]

Please see below ...

www.macfh.co.uk/CEMH.html

> -Original Message-
> From: get_iplayer [mailto:get_iplayer-boun...@lists.infradead.org]On
> Behalf Of iz
> Sent: 03 May 2016 19:59
> To: Nick Payne
> Cc: get_iplayer@lists.infradead.org
> Subject: Re: Spurious write permission error
>
> > Sent: Monday, May 02, 2016 at 1:41 PM
> > From: "Nick Payne" 
> > [snip]
> > Unable to write to a directory lacking write
> permission.INFO: Command
> > exit code 255 (raw code = 65280)
> > WARNING: Failed to tag MP4 file
>
> That error message comes from AtomicParsley.

Well spotted  -  searching for ...
Unable to write to a directory lacking write permission
... in the GiP PERL source draws a blank, but it's at 0x44D80 in
AtomicParsley.exe

AP has problems tagging large files, but I've just checked back to my
original post on that, and it generates a different error message, so I
don't know what to suggest further.

>   It is not
> related to GiP itself or to accessing the download_history
> file. That file is written before AtomicParsley runs. As you
> implied with the use of "spurious", it isn't a problem with
> access permissions - that is just assumed by AtomicParsley.
> The message really just means that AtomicParsley failed to
> rename the temporary (tagged) file back to the original file
> name for some reason. I used to occasionally see this happen
> on a USB disk attached to a router. I would also see ffmpeg
> glitches with the same setup, so I just chalked it up to disk
> contention and stopped using a network disk to host my output
> directory. Of course, that may not apply to your case.

Perhaps a timing and/or write cache issue  -  if the new contents of the
file have not been entirely written to disk by the time the rename
instruction comes, then the file will still be locked.  If a write cache is
in use, flushing it may solve that, which usually occurs automatically on
file closure, so is GiP/AP closing the file properly?  Or perhaps putting a
sleep command in between file closure and renaming may remove the problem.

But note that file contentions CAN occur with multiple instances of GiP
running as I have suggested  - I know, because I've had them in the dim and
distant past!


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: How good is HD supposed to be?

[Christopher Woods]

Yes, I could have been clearer. This depends on the deintelacing algorithm.

Aside from the very first frame of a 50p video, which can only ever be the 
first two fields (...well, that or black), every frame after that is 
effectively taking two adjacent fields and saying 'make a frame out of 
those', so each field is a part of one frame but it doesn't just go 1+2, 
3+4 etc.


IF you're making a full resolution frame 1, you'd use field 1 and field 2. 
For frame 2, field 2 and field 3. For frame 3, field 3 and field 4 etc.


Otherwise, it's black and field 1 for frame 1, field 1 and 2 for frame 2, etc.

I think of the process of capturing interlaced video as a constantly 
shuffling (up-down) grill which alternately covers even, then odd, lines of 
the CCD (just for conceptualising, not how it actually works). As you're 
freezeframing that point in time for half of your frame, the next field 
will be slightly advanced in time by microseconds so it's not the same as  
'taking a picture' every 25th of a second.


You're actually taking 50 shots per second and immediately discarding half 
the resolution, relying on persistence of vision and the inherent 
properties of the TV to mask this. The two half-resolution images 
interleave neatly and produce a full resolution image, and do so rapidly 
enough that everything works. You get pseudo-50fps as a happy by-product.


http://www.100fps.com has a good explanation and screenshots of various 
scenarios if you're interested in what raw interlaced video looks like and 
the problems you can have working with it.


(I hate interlaced video.)

200, no, 300 fps 4K video for all! It divides nicely with 25 and 29.97 fps 
standards, it's got the temporal and dimensional resolution, what's not to 
like... Except the transmission and storage costs... But H.265 will solve 
all of that ;)


Chris


On 2 May 2016 8:52:25 p.m. "Dave Liquorice"  wrote:


On Sat, 30 Apr 2016 22:49:01 +0100, Christopher Woods wrote:


The deinterlacing algorithm is doing no resizing - it's interpolating
between the frames and then 'printing' that to 50 progressive frames. The
resulting image will have slightly lower definition due to the bob
artifacts as it's reconstructing the frame from two sequentially
interlaced fields, but it hasn't changed resolution.


Sorry, I'm still missing what is actually going on but I think I'm getting
there.

Is the first reference to "frames" refering to a frame constructed from the
two fields designed to be shown at 25 fps? Lets call this F1.

At 50 fps we need twice as many frames so an F1 and an F2. F2 doesn't exist
and is created by interpolation between the current F1 and the next F1.

--
Cheers
Dave.



___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer




___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

RE: Spurious write permission error

[C E Macfarlane]

Please see below ...

www.macfh.co.uk/CEMH.html

> -Original Message-
> From: get_iplayer [mailto:get_iplayer-boun...@lists.infradead.org]On
> Behalf Of David Cantrell
> Sent: 03 May 2016 17:12
> To: get_iplayer@lists.infradead.org
> Subject: Re: Spurious write permission error
>
>
> On Tue, May 03, 2016 at 03:55:48PM +0100, C E Macfarlane wrote:
> > From: artisticforge . [mailto:artisticfo...@gmail.com]
> > > First a request. can you refrain from inserting your web page with
> > > every post? it is annoying
> > Tough shit.  It's my standard email sig, and I'm not
> changing it just for you.
>
> It is customary to put signatures at the end of messages, not the
> beginning. I'm sure that if you put it at the end no-one would object.

... and ...

> -Original Message-
> From: Roger Bell_West [mailto:ro...@firedrake.org]
> Sent: 03 May 2016 16:04
> To: C E Macfarlane
> Subject: Re: Spurious write permission error
>
> On Tue, May 03, 2016 at 03:55:48PM +0100, C E Macfarlane wrote:
> >Tough shit.  It's my standard email sig, and I'm not
> changing it just for you.
>
> No, a sig is something that goes at the _end_ of a message

The sig is put in automatically by Outlook.

>   and is
> separated from it by "\n-- \n".

You're thinking of newsgroups, this is email.

> No call for swearing just because someone calls you on your choice to
> do something non-standard.

I consider it rude for someone to pick on such trivia as posting styles,
sigs, etc, just because that person is losing an argument, hence my
down-to-earth response.


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Spurious write permission error

[David Cantrell]

On Tue, May 03, 2016 at 03:55:48PM +0100, C E Macfarlane wrote:
> From: artisticforge . [mailto:artisticfo...@gmail.com]
> > First a request. can you refrain from inserting your web page with
> > every post? it is annoying
> Tough shit.  It's my standard email sig, and I'm not changing it just for you.

It is customary to put signatures at the end of messages, not the
beginning. I'm sure that if you put it at the end no-one would object.

-- 
David Cantrell | http://www.cantrell.org.uk/david

Please stop rolling your Jargon Dice and explain the problem
you are having to me in plain English, using small words.
  -- John Hardin, in the Monastery

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Spurious write permission error

[David Cantrell]

On Tue, May 03, 2016 at 08:48:47AM -0500, artisticforge . wrote:

> second, I do not do "Windows". That said, any operating system should
> prevent the deletion of a file that is in use.

Indeed. However, Unix-a-likes do let you delete directory entries for
files that are in use - the file is only deleted when there are no
directory entries for it and no open file handles left. That's not
quite the same, of course, but in unless one is being unnaturally
precise in one's writing it is rare for people to discriminate between
them.

On Windows, on the other hand, my understanding is that you can't delete
a dirent for a file that is in use.

> MacOSX & Linux will not let me delete a directory if that directory is
> the "current working directory" for a process.

This is technically true - if you 'rmdir' a directory that is some
process's cwd then the dirent will go away but the inode will remain
until it is no longer in use. However, you won't be able to do anything
meaningful with it.

-- 
David Cantrell | Enforcer, South London Linguistic Massive

I think the most difficult moment that anyone could face is seeing
their domestic servants, whether maid or drivers, run away
  -- Abdul Rahman Al-Sheikh, writing on 25 Jan 2004 at
 http://www.arabnews.com/node/243486

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Spurious write permission error

[artisticforge .]

hello

First a request. can you refrain from inserting your web page with
every post? it is annoying

second, I do not do "Windows". That said, any operating system should
prevent the deletion of a file that is in use.
MacOSX & Linux will not let me delete a directory if that directory is
the "current working directory" for a process.
The test of WMP would only prove that the programmers were lazy and/or
just plain sloppy.
I use VLC or iTunes. 99.99% of the time it is iTunes.

I have 1 Toshiba laptop with Windows 10 installed. It sits in the
corner turned off until the rare event where i need to use
windows for something. Which are becoming more rare with each passing
day. I do not think it has been on this year.

The operating system is not going to allow N different processes
access the same resource at the exactly the same time. Not going to
happen. So the chance of N GiP all going for a write append to the
download_history file at the exact same instant in time is ZERO. The
operating system is not going to allow it to occur. The requests will
all go into the queue and processed in order.

What you have been describing all along is a "Race condition".

Concerning "Computer Hiccups", events of unknown causes, I have been
around enough years to say that "Stuff Happens"
which can never be explained. A Flipped bit, a random byte, etc.
Nothing is prefect and "Stuff Happens". Those random bit that go bump
in the night.

I have farm fields that need tending and firewood that needs to be
cut. the tractors do not drive themselves, at least not yet.
So enough of this diversion.


On Tue, May 3, 2016 at 7:26 AM, C E Macfarlane
 wrote:
> Please see below ...
>

>> -Original Message-
>> From: artisticforge . [mailto:artisticfo...@gmail.com]
>> Sent: 03 May 2016 12:27
>> To: c.e.macfarl...@macfh.co.uk
>> Cc: Nick Payne; get_iplayer
>> Subject: Re: Spurious write permission error
>>
>> Race conditions as you suggest
>
> I never mentioned race conditions.  If race conditions were encountered, at 
> very least Task Manager would have to be invoked to end each process involved 
> in the race, perhaps even the computer rebooted if the race hogged so much 
> CPU that TM could not be invoked.
>
>>   would be the fault of the operating
>> system and/or filesystem design.
>> Nearly every operating system has drivers which queue requests so that
>> there is no chaos in the allocation of resources.
>
> Yes.
>
>> A design flaw that glaring would relegate the operating system to
>> nothing but a toy and the operating system would never
>> be used in a commercial environment.
>
> Here's a test for you ...
>
> Open a file to play in Window Media Player, but then press the stop button so 
> that nothing is actually being played.  If you like,  another media 
> file and choose to add it to WMP's current playlist.  IME, even though WMP is 
> doing nothing with either file, commonly both are now locked, and cannot be 
> deleted, renamed, or written to, or, if they are appear to be successfully 
> deleted or renamed, examination of the directory shows that they have not 
> been, and won't be until WMP is either exited or the files are deleted from 
> its current playlist.  Sometimes, if a file gets updated when within WMP's 
> playlist  -  say if you use the back button to go to the previous playlist, 
> then update a file in the current playlist by, say, re-extracting it from a 
> longer file with slightly different start and stop time parameters, then use 
> the forward button to return to the current playlist, WMP gets hopelessly 
> confused, and tries to play the new file as if it were the old, and aborts 
> with an 
 error message, rather than simply realising that the file has been updated, 
and refetching its vital statistics from the new version of the file and 
carrying on from there.

>>   They are looked at at start up , at which point they are
>> "sucked into" memory
>> to be used. The download_history file is opened for append writes
>> after a file has been downloaded . Does not mean that the file has
>> been downloaded correctly just that it has been downloaded.
>
> Yes, so, as I suggested, if two instances happen simultaneously to finish 
> downloading their respective programmes, contention could occur when they try 
> simultaneously to update the download history accordingly.
>


-- 
terry l. ridder ><>

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Spurious write permission error

[David Cantrell]

On Mon, May 02, 2016 at 10:40:01PM +0100, C E Macfarlane wrote:

> 1)The OS  -  the OP was working with Windows, while all none of his 
> respondents are.  It might be that Windows is less forgiving of attempts by 
> one process to access a file that is currently being updated by another 
> process.

File locking semantics are completely different on Windows compared to
Unix, and get_iplayer is written in perl, which has very much a Unixy
background. I've not looked at the code, but unless there's extra
special Windows sauce in there it might not work.

-- 
David Cantrell | semi-evolved ape-thing

 I'm in retox

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

RE: Spurious write permission error

[C E Macfarlane]

Please see below ...

www.macfh.co.uk/CEMH.html

> -Original Message-
> From: artisticforge . [mailto:artisticfo...@gmail.com]
> Sent: 03 May 2016 12:27
> To: c.e.macfarl...@macfh.co.uk
> Cc: Nick Payne; get_iplayer
> Subject: Re: Spurious write permission error
> 
> Race conditions as you suggest

I never mentioned race conditions.  If race conditions were encountered, at 
very least Task Manager would have to be invoked to end each process involved 
in the race, perhaps even the computer rebooted if the race hogged so much CPU 
that TM could not be invoked.

>   would be the fault of the operating
> system and/or filesystem design.
> Nearly every operating system has drivers which queue requests so that
> there is no chaos in the allocation of resources.

Yes.

> A design flaw that glaring would relegate the operating system to
> nothing but a toy and the operating system would never
> be used in a commercial environment.

Here's a test for you ...

Open a file to play in Window Media Player, but then press the stop button so 
that nothing is actually being played.  If you like,  another media 
file and choose to add it to WMP's current playlist.  IME, even though WMP is 
doing nothing with either file, commonly both are now locked, and cannot be 
deleted, renamed, or written to, or, if they are appear to be successfully 
deleted or renamed, examination of the directory shows that they have not been, 
and won't be until WMP is either exited or the files are deleted from its 
current playlist.  Sometimes, if a file gets updated when within WMP's playlist 
 -  say if you use the back button to go to the previous playlist, then update 
a file in the current playlist by, say, re-extracting it from a longer file 
with slightly different start and stop time parameters, then use the forward 
button to return to the current playlist, WMP gets hopelessly confused, and 
tries to play the new file as if it were the old, and aborts with an er
 ror message, rather than simply realising that the file has been updated, and 
refetching its vital statistics from the new version of the file and carrying 
on from there.

... so your nice well-behaved picture of how an OS and the software running on 
it should behave just doesn't always happen in practice.

> Having read and studied the GiP Perl code over the years, no where in
> the code are any cache files nor the download_history file opened and
> kept open.

I never said that they were, merely that different processes might attempt to 
write to a given file at the same time.

>   They are looked at at start up , at which point they are
> "sucked into" memory
> to be used. The download_history file is opened for append writes
> after a file has been downloaded . Does not mean that the file has
> been downloaded correctly just that it has been downloaded.

Yes, so, as I suggested, if two instances happen simultaneously to finish 
downloading their respective programmes, contention could occur when they try 
simultaneously to update the download history accordingly.

> I would chalk the OP errors to computer "hiccups" , cause unknown. A
> glitch in time & space. A blink of the Twilight Zone or is it Night
> Gallery. The bits which go bump in the night.

That is great deal less rational than my explanation!  I fear we'll be 
discussing Ley Lines next :-(


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Spurious write permission error

[artisticforge .]

hello

Race conditions as you suggest would be the fault of the operating
system and/or filesystem design.
Nearly every operating system has drivers which queue requests so that
there is no chaos in the allocation of resources.
A design flaw that glaring would relegate the operating system to
nothing but a toy and the operating system would never
be used in a commercial environment.

Having read and studied the GiP Perl code over the years, no where in
the code are any cache files nor the download_history file opened and
kept open. They are looked at at start up , at which point they are
"sucked into" memory
to be used. The download_history file is opened for append writes
after a file has been downloaded . Does not mean that the file has
been downloaded correctly just that it has been downloaded.

I would chalk the OP errors to computer "hiccups" , cause unknown. A
glitch in time & space. A blink of the Twilight Zone or is it Night
Gallery. The bits which go bump in the night.




On Mon, May 2, 2016 at 4:40 PM, C E Macfarlane
 wrote:
> Please see below ...
>

> However, there are various things that might alter how much of a problem this 
> might be in practice, for example here are two:
>
> 1)  The OS  -  the OP was working with Windows, while all none of his 
> respondents are.  It might be that Windows is less forgiving of attempts by 
> one process to access a file that is currently being updated by another 
> process.
>
> 2)  The speed of the hardware.  It is likely that filesystem timeouts are 
> fairly accurately timed and constant across different hardware, while the 
> time taken to actually update a file will be determined, or rather inversely 
> determined, by the speed of the hardware.  Therefore, the faster the 
> hardware, the less the likelihood of a contention causing a timeout.  I have 
> fairly old hardware.
>
> Regards
>



-- 
terry l. ridder ><>

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer