Re: Bug#1064346: Source is missing errors on HTML source files

[Wookey]

On 2024-02-22 11:37 +0530, Shriram Ravindranathan wrote:
> Thank you Bastien,
> I tried doing this but it appears that the scripts to build these example
> files all depend on having the highlight binary itself installed on the
> machine. I am unsure whether it is okay to have the package depend on itself
> for building.

Packages that use themselves to build are problematic for bootstrapping and 
cross-building. 

When cross-building you can't (usually) run the binaries just built
because they are the wrong architecture.

When bootstrapping packages that depend on themselves cause dependency
loops which are a pain (we can't build foo because we don't have foo
yet). We have 'build profiles' (older name 'staged builds') to deal
with this. i.e you define a minimum 'stage1' build profile which does
not have the circular dependency (and the normal build which does) so
the building of the final version can be automated.

The choices here are to
1) use the binary just build during the build
2) have a self dependency and use a packaged binary

1) Is simple but prevents the package cross-building. Usually the best
thing to do is just skip that part if cross-building (you can live without the 
examples)
Just ensure that the build doesn't fail due to missing files.

2) Works for cross-building (by the magic of multiarch) but you should
add a stage notation so it can easily be built for new architectures,
which is a bit of a faff.

In case '2' you have to worry about version skew. How much does it
matter if examples/whatever are built with the previous/an older
version of the package? Case '1' avoids this. This is very specific to
the package in question: sometimes it needs to be exactly the same,
sometimes a version from a decade ago will be just fine. This is the
main reason people usually pick option '1'.

https://wiki.debian.org/BuildProfileSpec
https://wiki.debian.org/DebianBootstrap
https://wiki.debian.org/CrossBuildPackagingGuidelines

All the above is quite a lot to take in, and you don't need to worry
too much, but it is a good idea to at least have this stuff in the
back of your mind when packaging. Ideally this would all work
correctly on the whole archive, and packagers are the ones best-placed
to stick in the necessary runes.

Wookey
-- 
Principal hats:  Debian, Wookware, ARM
http://wookware.org/


signature.asc
Description: PGP signature

Re: Bug#1064346: Source is missing errors on HTML source files

[roucaries bastien]

Le jeu. 22 févr. 2024 à 06:07, Shriram Ravindranathan  a écrit :
>
> Thank you Bastien,
> I tried doing this but it appears that the scripts to build these
> example files all depend on having the highlight binary itself installed
> on the machine. I am unsure whether it is okay to have the package
> depend on itself for building.

Yes and no.

Best use the built package to generate the file

Other way is staged build

Bastien
>
> On 21/02/24 11:20 pm, roucaries bastien wrote:
> > You should rebuilt from source also... See for instance how I do with 
> > node-long
>
> --
> Shriram Ravindranathan
> ters
>

Re: Bug#1064346: Source is missing errors on HTML source files

[Shriram Ravindranathan]

Thank you Bastien,
I tried doing this but it appears that the scripts to build these 
example files all depend on having the highlight binary itself installed 
on the machine. I am unsure whether it is okay to have the package 
depend on itself for building.


On 21/02/24 11:20 pm, roucaries bastien wrote:

You should rebuilt from source also... See for instance how I do with node-long


--
Shriram Ravindranathan
ters



OpenPGP_0xFC7E951A7BEF0836.asc
Description: OpenPGP public key


OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: Bug#1064346: Source is missing errors on HTML source files

[roucaries bastien]

Le mer. 21 févr. 2024 à 15:38, Soren Stoutner  a écrit :
>
> Shriram,
>
> On Wednesday, February 21, 2024 8:30:54 AM MST Shriram Ravindranathan wrote:
> > Upon inspecting the embedded font, It seems to be a bespoke icon-font
> > generated using a tool called "Fontello" from one of the icons of the
> > octicons iconset from Atom  (MIT
> > Licensed SVGs)
> >
> > The font has only 1 glyph, Would it suffice to add this source image to
> > d/missing-souces and add that copyright info to d/copyright?
>
> I would assume so.  If anyone on mentors knows differently please speak up.
>
> > On 21/02/24 9:56 am, Soren Stoutner wrote:
> >
> > >
> > >
> > > Shriram,
> > >
> > >
> > >
> > >
> > > 1. For anything that has the unminified source in the upstream
> > > tarball, I would just create a lintian override with a comment listing
> > > the full path to the source for each file.  You can see an example of
> > > how this can be done here:
> > >
> > >
> > >
> > >
> > > https://salsa.debian.org/qt-kde-team/qt/qtwebengine/-/blob/master/debian/
> sou
> > > rce/lintian-overrides?ref_type=heads
> >
> > >
> > >
> > >
> > > Typically you only copy the source to the debian/missing-sources
> > > directory when it is not included in the upstream tarball and you have
> > > had to acquire it from another place.

You should rebuilt from source also... See for instance how I do with node-long
> > >
> > >
> > >
> > >
> > > 2. The github link below includes an embedded font in woff format.
> > > Typically, fonts like this would be considered compiled, so a separate
> > > font source would be needed.  However, I’m not sure what the Debian
> > > guidance for dealing with an HTML embedded font like this.  If someone
> > > else on mentors doesn’t know, I would recommend you ask on debian-legal.
> > >
> > >
> > >
> > >
> > > As these are mostly README files, and if it becomes difficult for you
> > > to acquire the source for some of them, you might consider excluding
> > > those you can’t get the source for, at least temporarily, using
> > > Files-Excluded in debian/copyright (and then running uscan, which will
> > > produce a modified tarball that does not include the problematic
> > > files).  For example, see:
> > >
> > >
> > >
> > >
> > > https://salsa.debian.org/cryptocoin-team/electrum/-/blob/master/debian/
> copyr
> > > ight?ref_type=heads
> >
> > >
> > >
> > >
> > > Whether this is a good option depends on how helpful those README
> > > files are for the users of your package.  If you go this route, you
> > > should add +dfsg to the version of your package to indicate that the
> > > upstream tarball has been repackaged to remove files that are not free
> > > (or for which the source is not available).
> > >
> > >
> > >
> > >
> > > Soren
> > >
> > >
> >
> > --
> > Shriram Ravindranathan
> > ters
> >
>
>
> --
> Soren Stoutner
> so...@debian.org

Re: Bug#1064346: Source is missing errors on HTML source files

[Soren Stoutner]

Shriram,

On Wednesday, February 21, 2024 8:30:54 AM MST Shriram Ravindranathan wrote:
> Upon inspecting the embedded font, It seems to be a bespoke icon-font 
> generated using a tool called "Fontello" from one of the icons of the 
> octicons iconset from Atom  (MIT 
> Licensed SVGs)
> 
> The font has only 1 glyph, Would it suffice to add this source image to 
> d/missing-souces and add that copyright info to d/copyright?

I would assume so.  If anyone on mentors knows differently please speak up.

> On 21/02/24 9:56 am, Soren Stoutner wrote:
> 
> >
> >
> > Shriram,
> >
> >
> >
> >
> > 1. For anything that has the unminified source in the upstream 
> > tarball, I would just create a lintian override with a comment listing 
> > the full path to the source for each file.  You can see an example of 
> > how this can be done here:
> >
> >
> >
> >
> > https://salsa.debian.org/qt-kde-team/qt/qtwebengine/-/blob/master/debian/
sou
> > rce/lintian-overrides?ref_type=heads
>
> >
> >
> >
> > Typically you only copy the source to the debian/missing-sources 
> > directory when it is not included in the upstream tarball and you have 
> > had to acquire it from another place.
> >
> >
> >
> >
> > 2. The github link below includes an embedded font in woff format. 
> > Typically, fonts like this would be considered compiled, so a separate 
> > font source would be needed.  However, I’m not sure what the Debian 
> > guidance for dealing with an HTML embedded font like this.  If someone 
> > else on mentors doesn’t know, I would recommend you ask on debian-legal.
> >
> >
> >
> >
> > As these are mostly README files, and if it becomes difficult for you 
> > to acquire the source for some of them, you might consider excluding 
> > those you can’t get the source for, at least temporarily, using 
> > Files-Excluded in debian/copyright (and then running uscan, which will 
> > produce a modified tarball that does not include the problematic 
> > files).  For example, see:
> >
> >
> >
> >
> > https://salsa.debian.org/cryptocoin-team/electrum/-/blob/master/debian/
copyr
> > ight?ref_type=heads
>
> >
> >
> >
> > Whether this is a good option depends on how helpful those README 
> > files are for the users of your package.  If you go this route, you 
> > should add +dfsg to the version of your package to indicate that the 
> > upstream tarball has been repackaged to remove files that are not free 
> > (or for which the source is not available).
> >
> >
> >
> >
> > Soren
> >
> >
> 
> -- 
> Shriram Ravindranathan
> ters
> 


-- 
Soren Stoutner
so...@debian.org

signature.asc
Description: This is a digitally signed message part.

Re: Bug#1064346: Source is missing errors on HTML source files

[Shriram Ravindranathan]

Upon inspecting the embedded font, It seems to be a bespoke icon-font 
generated using a tool called "Fontello" from one of the icons of the 
octicons iconset from Atom  (MIT 
Licensed SVGs)


The font has only 1 glyph, Would it suffice to add this source image to 
d/missing-souces and add that copyright info to d/copyright?


On 21/02/24 9:56 am, Soren Stoutner wrote:


Shriram,


1. For anything that has the unminified source in the upstream 
tarball, I would just create a lintian override with a comment listing 
the full path to the source for each file.  You can see an example of 
how this can be done here:



https://salsa.debian.org/qt-kde-team/qt/qtwebengine/-/blob/master/debian/source/lintian-overrides?ref_type=heads


Typically you only copy the source to the debian/missing-sources 
directory when it is not included in the upstream tarball and you have 
had to acquire it from another place.



2. The github link below includes an embedded font in woff format. 
Typically, fonts like this would be considered compiled, so a separate 
font source would be needed.  However, I’m not sure what the Debian 
guidance for dealing with an HTML embedded font like this.  If someone 
else on mentors doesn’t know, I would recommend you ask on debian-legal.



As these are mostly README files, and if it becomes difficult for you 
to acquire the source for some of them, you might consider excluding 
those you can’t get the source for, at least temporarily, using 
Files-Excluded in debian/copyright (and then running uscan, which will 
produce a modified tarball that does not include the problematic 
files).  For example, see:



https://salsa.debian.org/cryptocoin-team/electrum/-/blob/master/debian/copyright?ref_type=heads


Whether this is a good option depends on how helpful those README 
files are for the users of your package.  If you go this route, you 
should add +dfsg to the version of your package to indicate that the 
upstream tarball has been repackaged to remove files that are not free 
(or for which the source is not available).



Soren


--
Shriram Ravindranathan
ters



OpenPGP_0xFC7E951A7BEF0836.asc
Description: OpenPGP public key


OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: Bug#1064346: Source is missing errors on HTML source files

[Soren Stoutner]

Shriram,

1.  For anything that has the unminified source in the upstream tarball, I 
would just create a 
lintian override with a comment listing the full path to the source for each 
file.  You can see 
an example of how this can be done here:

https://salsa.debian.org/qt-kde-team/qt/qtwebengine/-/blob/master/debian/source/lintian-overrides?ref_type=heads[1]

Typically you only copy the source to the debian/missing-sources directory when 
it is not 
included in the upstream tarball and you have had to acquire it from another 
place.

2.  The github link below includes an embedded font in woff format.  Typically, 
fonts like 
this would be considered compiled, so a separate font source would be needed.  
However, 
I’m not sure what the Debian guidance for dealing with an HTML embedded font 
like this.  
If someone else on mentors doesn’t know, I would recommend you ask on 
debian-legal.

As these are mostly README files, and if it becomes difficult for you to 
acquire the source 
for some of them, you might consider excluding those you can’t get the source 
for, at least 
temporarily, using Files-Excluded in debian/copyright (and then running uscan, 
which will 
produce a modified tarball that does not include the problematic files).  For 
example, see:

https://salsa.debian.org/cryptocoin-team/electrum/-/blob/master/debian/copyright?
ref_type=heads[2]

Whether this is a good option depends on how helpful those README files are for 
the users 
of your package.  If you go this route, you should add +dfsg to the version of 
your package 
to indicate that the upstream tarball has been repackaged to remove files that 
are not free 
(or for which the source is not available).

Soren

On Tuesday, February 20, 2024 8:23:41 PM MST Shriram Ravindranathan wrote:
> Thanks, Soren.
> 
> It looks like most of these files have just one or two lines that are 
> extremely long.
> 
> These are mostly README files. Most of them seem to have this 
> github-markdown.css 
>  
> minified and pasted in them. While others have the sources that were 
> used to generate them listed in the same folder.
>
> Should I copy these sources into the d/missing-sources directory?

-- 
Soren Stoutner
so...@debian.org


[1] 
https://salsa.debian.org/qt-kde-team/qt/qtwebengine/-/blob/master/debian/source/
lintian-overrides?ref_type=heads
[2] 
https://salsa.debian.org/cryptocoin-team/electrum/-/blob/master/debian/copyright?
ref_type=heads


signature.asc
Description: This is a digitally signed message part.

Re: Bug#1064346: Source is missing errors on HTML source files

[Shriram Ravindranathan]

Thanks, Soren.

It looks like most of these files have just one or two lines that are 
extremely long.


These are mostly README files. Most of them seem to have this 
github-markdown.css 
 
minified and pasted in them. While others have the sources that were 
used to generate them listed in the same folder.


Should I copy these sources into the d/missing-sources directory?

On 21/02/24 2:28 am, Soren Stoutner wrote:

The question is if the long lines in these HTML files are actually indications
that the HTML files are not the original source.  This usually happens in one
of two cases.

1.  The files have been minified.
2.  The files were originally created in another format and converted to HTML.

Sometimes HTML files naturally have long lines.  If you look at the
descriptions of the lintian warnings, they acknowledge that this is an
imperfect check that will result in some false-positives.  If that is the
case, the HTML files are the original source, and they have not been minified,
then you can override these warnings with a description as to why.

On Tuesday, February 20, 2024 9:08:17 AM MST Shriram Ravindranathan wrote:

Hello mentors,

I am getting a few lintian "source-is-missing" errors for some HTML
files. These HTML files are infact present in the source code but they
have too many lines which triggers a
"very-long-line-length-in-source-file" lintian tag and that in turn
causes the "source-is-missing" error.

Most of the info I could find in the policy manual and in the forums
pertained to binary files that were included in the source, the strategy
these resources suggested were
1. Repack upstream tar with the source code of these files
2. Add the source code to the d/missing-sources directory

I don't think either of these are viable options in my case. I was
wondering whether it would be okay to suppress these errors. Is there
any other way to solve this?

--
Shriram Ravindranathan




--
Shriram Ravindranathan



OpenPGP_0xFC7E951A7BEF0836.asc
Description: OpenPGP public key


OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: Bug#1064346: Source is missing errors on HTML source files

[Soren Stoutner]

The question is if the long lines in these HTML files are actually indications 
that the HTML files are not the original source.  This usually happens in one 
of two cases.

1.  The files have been minified.
2.  The files were originally created in another format and converted to HTML.

Sometimes HTML files naturally have long lines.  If you look at the 
descriptions of the lintian warnings, they acknowledge that this is an 
imperfect check that will result in some false-positives.  If that is the 
case, the HTML files are the original source, and they have not been minified, 
then you can override these warnings with a description as to why.

On Tuesday, February 20, 2024 9:08:17 AM MST Shriram Ravindranathan wrote:
> Hello mentors,
> 
> I am getting a few lintian "source-is-missing" errors for some HTML 
> files. These HTML files are infact present in the source code but they 
> have too many lines which triggers a 
> "very-long-line-length-in-source-file" lintian tag and that in turn 
> causes the "source-is-missing" error.
> 
> Most of the info I could find in the policy manual and in the forums 
> pertained to binary files that were included in the source, the strategy 
> these resources suggested were
> 1. Repack upstream tar with the source code of these files
> 2. Add the source code to the d/missing-sources directory
> 
> I don't think either of these are viable options in my case. I was 
> wondering whether it would be okay to suppress these errors. Is there 
> any other way to solve this?
> 
> -- 
> Shriram Ravindranathan
> 


-- 
Soren Stoutner
so...@debian.org

signature.asc
Description: This is a digitally signed message part.

Bug#1064346: Source is missing errors on HTML source files

[Shriram Ravindranathan]

Hello mentors,

I am getting a few lintian "source-is-missing" errors for some HTML 
files. These HTML files are infact present in the source code but they 
have too many lines which triggers a 
"very-long-line-length-in-source-file" lintian tag and that in turn 
causes the "source-is-missing" error.


Most of the info I could find in the policy manual and in the forums 
pertained to binary files that were included in the source, the strategy 
these resources suggested were

1. Repack upstream tar with the source code of these files
2. Add the source code to the d/missing-sources directory

I don't think either of these are viable options in my case. I was 
wondering whether it would be okay to suppress these errors. Is there 
any other way to solve this?


--
Shriram Ravindranathan



OpenPGP_0xFC7E951A7BEF0836.asc
Description: OpenPGP public key


OpenPGP_signature.asc
Description: OpenPGP digital signature