Re: [FFmpeg-devel] snapshots differ from git clone

2016-08-02 Thread Jan Ehrhardt 
Nicolas George in gmane.comp.video.ffmpeg.devel (Wed, 3 Aug 2016
01:22:18 +0200):
>Actually, I was talking about snapshots generated by gitweb. I realize now
>that the snapshot we are talking are generated differently, and they contain
>a full pack of the .git directory. That makes them roughly equivalent to a
>full clone. But bigger.

It really makes them equivalent. Once unpacked, you can do everything on
it with git commands. In my workaround for resolving the stale files
issue I was using git stash, but git pull is possible as well. So for an
initial setup of the build environment it is faster to grab a snapshot,
unpack it and optionally do a git pull.

@Michael Niedermayer: The stale files did not re-appear, so the issue
was solved. Thanks.
-- 
Jan
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] snapshots differ from git clone

2016-08-02 Thread Michael Niedermayer 
On Tue, Aug 02, 2016 at 10:41:09PM +, Timothy Gu wrote:
> On Tue, Aug 2, 2016 at 3:10 PM Nicolas George  wrote:
> 
> > Le sextidi 16 thermidor, an CCXXIV, Jan Ehrhardt a écrit :
> > > Because it is faster to download a snapshot and extract it than
> > > doing a git clone
> >
> > I suggest you consider doing a shallow clone (--depth 1). It should take
> > roughly the same time as getting and unpacking a snapshot. IIRC, one of the
> > minor perks of doing that is that the version script will give a more
> > useful
> > version tag.
> >
> 
> At least here, Git is still slower. In addition, https://git.ffmpeg.org
> doesn't seem to support --depth.
> 
> $ time curl https://www.ffmpeg.org/releases/ffmpeg-snapshot.tar.bz2 | tar
> -jxf -
> real 0m10.331s
> user 0m2.272s
> sys 0m0.312s
> $ git clone --depth=1 https://git.ffmpeg.org/ffmpeg.git ffmpeg2
> Cloning into 'ffmpeg2'...
> fatal: The remote end hung up unexpectedly
> fatal: protocol error: bad pack header

this is due to git wanting to write temporary files into the repository
which is not writable.
Making it writable would be trivial and thats how linaro seems to
have "fixed" it, but iam not sure thats a good idea security wise
Updatng the git to something beyond whats available from the official
ubuntu we use might fix this but then future security updates would
become harder.
(recent git is available at
 https://launchpad.net/~git-core/+archive/ubuntu/ppa
 so we would still have distro provided packages but security support
 would be an open question ...)
so while all solutions are kind of not hard, iam a bit undecided
what should be done about that 

[...]

-- 
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Avoid a single point of failure, be that a person or equipment.


signature.asc
Description: Digital signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] snapshots differ from git clone

2016-08-02 Thread Nicolas George 
Le septidi 17 thermidor, an CCXXIV, Carl Eugen Hoyos a écrit :
> This would be a bug.

No, that would be an inherent limitation of the snapshots: they do not
contain anything that can give an accurate version.

Actually, I was talking about snapshots generated by gitweb. I realize now
that the snapshot we are talking are generated differently, and they contain
a full pack of the .git directory. That makes them roughly equivalent to a
full clone. But bigger.

Regards,

-- 
  Nicolas George


signature.asc
Description: Digital signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] snapshots differ from git clone

2016-08-02 Thread Carl Eugen Hoyos 
Hi!

2016-08-03 0:09 GMT+02:00 Nicolas George :
> IIRC, one of the minor perks of doing that is that the version
> script will give a more useful version tag.

This would be a bug.

Carl Eugen
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] snapshots differ from git clone

2016-08-02 Thread Nicolas George 
Le sextidi 16 thermidor, an CCXXIV, Timothy Gu a écrit :
> In addition, https://git.ffmpeg.org doesn't seem to support --depth.

git://source.ffmpeg.org/ffmpeg (i.e. the videolan server) does. It would
probably be better if our server did too, but I do not know how.

> $ time git clone --depth=1 https://github.com/FFmpeg/FFmpeg.git ffmpeg2
> real 0m17.540s

Slower indeed, but still much faster than a full clone, and with a few
benefits compared to a snapshot.

Regards,

-- 
  Nicolas George


signature.asc
Description: Digital signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] snapshots differ from git clone

2016-08-02 Thread Timothy Gu 
On Tue, Aug 2, 2016 at 3:10 PM Nicolas George  wrote:

> Le sextidi 16 thermidor, an CCXXIV, Jan Ehrhardt a écrit :
> > Because it is faster to download a snapshot and extract it than
> > doing a git clone
>
> I suggest you consider doing a shallow clone (--depth 1). It should take
> roughly the same time as getting and unpacking a snapshot. IIRC, one of the
> minor perks of doing that is that the version script will give a more
> useful
> version tag.
>

At least here, Git is still slower. In addition, https://git.ffmpeg.org
doesn't seem to support --depth.

$ time curl https://www.ffmpeg.org/releases/ffmpeg-snapshot.tar.bz2 | tar
-jxf -
real 0m10.331s
user 0m2.272s
sys 0m0.312s
$ git clone --depth=1 https://git.ffmpeg.org/ffmpeg.git ffmpeg2
Cloning into 'ffmpeg2'...
fatal: The remote end hung up unexpectedly
fatal: protocol error: bad pack header
$ time git clone --depth=1 https://github.com/FFmpeg/FFmpeg.git ffmpeg2
real 0m17.540s
user 0m1.308s
sys 0m0.308s

Timothy
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] snapshots differ from git clone

2016-08-02 Thread Nicolas George 
Le sextidi 16 thermidor, an CCXXIV, Jan Ehrhardt a écrit :
> Because it is faster to download a snapshot and extract it than
> doing a git clone

I suggest you consider doing a shallow clone (--depth 1). It should take
roughly the same time as getting and unpacking a snapshot. IIRC, one of the
minor perks of doing that is that the version script will give a more useful
version tag.

And of course, if you do frequent updates, using a full clone and pulling to
it incrementally will be even faster since it only downloads the changes.

Regards,

-- 
  Nicolas George


signature.asc
Description: Digital signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] snapshots differ from git clone

2016-08-02 Thread Michael Niedermayer 
On Tue, Aug 02, 2016 at 10:54:15PM +0200, Jan Ehrhardt wrote:
> I have a script that cross-compiles ffmpeg on Ubuntu for Windows.
> Because it is faster to download a snapshot and extract it than
> doing a git clone the script uses
> http://ffmpeg.org/releases/ffmpeg-snapshot-git.tar.bz2
> 
> Recently I ran into unresolved externals in x86/dirac_dwt_init.c/o.
> Details below. A clumsy workaround:
> 
> $ rm -f libavcodec/x86/dirac*
> $ git stash
> 
> The problem seems to be that the snapshots differ from a git clone
> and for instance contain >50 untracked files, amongst which:
> 
> libavcodec/x86/dirac_dwt.c
> libavcodec/x86/dirac_dwt.h
> libavcodec/x86/diracdsp_mmx.c
> libavcodec/x86/diracdsp_mmx.h
> libavcodec/x86/diracdsp_yasm.asm
> libavcodec/x86/dwt_yasm.asm
> libavcodec/x86/vc1dsp.asm
> 
> A git clone has only these files:
> 
> libavcodec/x86/diracdsp.asm
> libavcodec/x86/diracdsp_init.c
> libavcodec/x86/dirac_dwt.asm
> libavcodec/x86/dirac_dwt_init.c
> 
> Question: why are the snapshots different from a git clone?

some stale files in the directory used to build the snapshot
ive fixed it earlier today
this should not reoccur

[...]
-- 
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Into a blind darkness they enter who follow after the Ignorance,
they as if into a greater darkness enter who devote themselves
to the Knowledge alone. -- Isha Upanishad


signature.asc
Description: Digital signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] snapshots differ from git clone

2016-08-02 Thread Timothy Gu 
On Tue, Aug 2, 2016 at 1:54 PM Jan Ehrhardt  wrote:

> I have a script that cross-compiles ffmpeg on Ubuntu for Windows.
> Because it is faster to download a snapshot and extract it than
> doing a git clone the script uses
> http://ffmpeg.org/releases/ffmpeg-snapshot-git.tar.bz2


This sounds like a bug on the snapshot server. The files you mentioned were
removed from Git in e3461197b and 9fd6ea933fa.

Timothy
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

[FFmpeg-devel] snapshots differ from git clone

2016-08-02 Thread Jan Ehrhardt 
I have a script that cross-compiles ffmpeg on Ubuntu for Windows.
Because it is faster to download a snapshot and extract it than
doing a git clone the script uses
http://ffmpeg.org/releases/ffmpeg-snapshot-git.tar.bz2

Recently I ran into unresolved externals in x86/dirac_dwt_init.c/o.
Details below. A clumsy workaround:

$ rm -f libavcodec/x86/dirac*
$ git stash

The problem seems to be that the snapshots differ from a git clone
and for instance contain >50 untracked files, amongst which:

libavcodec/x86/dirac_dwt.c
libavcodec/x86/dirac_dwt.h
libavcodec/x86/diracdsp_mmx.c
libavcodec/x86/diracdsp_mmx.h
libavcodec/x86/diracdsp_yasm.asm
libavcodec/x86/dwt_yasm.asm
libavcodec/x86/vc1dsp.asm

A git clone has only these files:

libavcodec/x86/diracdsp.asm
libavcodec/x86/diracdsp_init.c
libavcodec/x86/dirac_dwt.asm
libavcodec/x86/dirac_dwt_init.c

Question: why are the snapshots different from a git clone?
-- 
Jan

Details of the builds below

(more undefined references)
/sandbox/win32/ffmpeg/libavcodec/x86/dirac_dwt_init.c:138: undefined reference 
to `ff_vertical_compose_dd137iL0_mmx'
libavcodec/libavcodec.a(dirac_dwt_init.o): In function 
`vertical_compose_dirac53iH0_mmx':
/home/jan/sandbox/win32/ffmpeg/libavcodec/x86/dirac_dwt_init.c:138: undefined 
reference to `ff_vertical_compose_dirac53iH0_mmx'
libavcodec/libavcodec.a(dirac_dwt_init.o): In function 
`vertical_compose_dd97iH0_mmx':
/home/jan/sandbox/win32/ffmpeg/libavcodec/x86/dirac_dwt_init.c:138: undefined 
reference to `ff_vertical_compose_dd97iH0_mmx'
libavcodec/libavcodec.a(dirac_dwt_init.o): In function 
`vertical_compose53iL0_mmx':
/home/jan/sandbox/win32/ffmpeg/libavcodec/x86/dirac_dwt_init.c:138: undefined 
reference to `ff_vertical_compose53iL0_mmx'
collect2: error: ld returned 1 exit status
collect2: error: ld returned 1 exit status
make: *** [ffmpeg_g.exe] Error 1
make: *** Waiting for unfinished jobs
make: *** [ffplay_g.exe] Error 1
jan@box:~/sandbox/win32/ffmpeg$ ls -la 
/home/jan/sandbox/win32/ffmpeg/libavcodec/x86/dirac*
-rw-rw-r-- 1 jan jan  8500 Jul 22 18:20 
/home/jan/sandbox/win32/ffmpeg/libavcodec/x86/diracdsp.asm
-rw-rw-r-- 1 jan jan   122 Aug  2 01:09 
/home/jan/sandbox/win32/ffmpeg/libavcodec/x86/diracdsp.d
-rw-rw-r-- 1 jan jan  8883 Jul 22 18:20 
/home/jan/sandbox/win32/ffmpeg/libavcodec/x86/diracdsp_init.c
-rw-rw-r-- 1 jan jan   225 Aug  2 01:09 
/home/jan/sandbox/win32/ffmpeg/libavcodec/x86/diracdsp_init.d
-rw-rw-r-- 1 jan jan 17146 Aug  2 01:09 
/home/jan/sandbox/win32/ffmpeg/libavcodec/x86/diracdsp_init.o
-rw-r--r-- 1 jan jan  6752 Jul 21  2014 
/home/jan/sandbox/win32/ffmpeg/libavcodec/x86/diracdsp_mmx.c
-rw-r--r-- 1 jan jan  2176 Nov  1  2012 
/home/jan/sandbox/win32/ffmpeg/libavcodec/x86/diracdsp_mmx.h
-rw-rw-r-- 1 jan jan  4262 Aug  2 01:09 
/home/jan/sandbox/win32/ffmpeg/libavcodec/x86/diracdsp.o
-rw-rw-r-- 1 jan jan  6687 Jul 26  2015 
/home/jan/sandbox/win32/ffmpeg/libavcodec/x86/diracdsp_yasm.asm
-rw-rw-r-- 1 jan jan  7514 Feb  6 19:20 
/home/jan/sandbox/win32/ffmpeg/libavcodec/x86/dirac_dwt.asm
-rw-r--r-- 1 jan jan  6946 Jul 21  2014 
/home/jan/sandbox/win32/ffmpeg/libavcodec/x86/dirac_dwt.c
-rw-rw-r-- 1 jan jan   243 Aug  2 01:09 
/home/jan/sandbox/win32/ffmpeg/libavcodec/x86/dirac_dwt.d
-rw-r--r-- 1 jan jan  1190 May 25  2013 
/home/jan/sandbox/win32/ffmpeg/libavcodec/x86/dirac_dwt.h
-rw-rw-r-- 1 jan jan  7800 Feb  6 19:20 
/home/jan/sandbox/win32/ffmpeg/libavcodec/x86/dirac_dwt_init.c
-rw-rw-r-- 1 jan jan   226 Aug  2 01:09 
/home/jan/sandbox/win32/ffmpeg/libavcodec/x86/dirac_dwt_init.d
-rw-rw-r-- 1 jan jan 18426 Aug  2 01:09 
/home/jan/sandbox/win32/ffmpeg/libavcodec/x86/dirac_dwt_init.o
-rw-rw-r-- 1 jan jan 15806 Aug  2 01:09 
/home/jan/sandbox/win32/ffmpeg/libavcodec/x86/dirac_dwt.o
jan@box:~/sandbox/win32/ffmpeg$ rm -f 
/home/jan/sandbox/win32/ffmpeg/libavcodec/x86/dirac*
jan@box:~/sandbox/win32/ffmpeg$ git stash
Saved working directory and index state WIP on master: d9b0b1a applied
HEAD is now at d9b0b1a applied
jan@box:~/sandbox/win32/ffmpeg$ ls -la 
/home/jan/sandbox/win32/ffmpeg/libavcodec/x86/dirac*
-rw-rw-r-- 1 jan jan 8500 Aug  2 01:16 
/home/jan/sandbox/win32/ffmpeg/libavcodec/x86/diracdsp.asm
-rw-rw-r-- 1 jan jan 8883 Aug  2 01:16 
/home/jan/sandbox/win32/ffmpeg/libavcodec/x86/diracdsp_init.c
-rw-rw-r-- 1 jan jan 7514 Aug  2 01:16 
/home/jan/sandbox/win32/ffmpeg/libavcodec/x86/dirac_dwt.asm
-rw-rw-r-- 1 jan jan 7800 Aug  2 01:16 
/home/jan/sandbox/win32/ffmpeg/libavcodec/x86/dirac_dwt_init.c
jan@box:~/sandbox/win32/ffmpeg$ git status
Untracked files:
  (use "git add ..." to include in what will be committed)

libavcodec/aac_tablegen.c
libavcodec/aac_tablegen.h
libavcodec/aac_tablegen_decl.h
libavcodec/aacsbr_fixed_tablegen.c
libavcodec/aacsbr_tablegen.c
libavcodec/arm/dcadsp_init_arm.c
libavcodec/arm/dcadsp_neon.S
libavcodec/arm/dcadsp_vfp.S
libavcodec/arm/dct-test.c
libavcodec/dct-test.c