Bug#1034196: unblock: openrefine/3.6.2-2

2023-04-20 Thread Markus Koschany
Hi Paul,

Am Donnerstag, dem 20.04.2023 um 18:07 +0200 schrieb Paul Gevers:
> [...]
> > Since I already followed the Debian Policy and included the missing sources
> > in
> > debian/missing-sources, I felt that shipping the 3rdparty directory in
> > debian/missing-sources/3rdparty would be a good intermediate solution.
> 
> But you added a more files than you have in testing (including jquery.js).

Yes, because 3.6.1 was the first version after 3.5.2 that removed those
3rdparty Javascript files. I noticed it after I had uploaded the package. I
corrected this problem in 3.6.2-1.

> > If you
> > insist I can repack the tarball, add the 3rdparty directory and remove it
> > from
> > debian/missing-sources but in the end it would not make any difference.
> 
> Huh? I wasn't asking you to do that. I was asking you to use packaged 
> binaries as a dependency.
> 
> > Openrefine is a desktop application which only runs on your own computer.
> 
> You know, now I know. Does the security team also know? Should they really?

I am a member of the security team. It is on my radar.

> 
> > If
> > you insist I can depend on libjs-jquery and replace the local copy with a
> > symlink but I feel this would be an example of over-engineering without any
> > real value to our users in this specific case.
> 
> That argument holds for a lot of things we do. What I try to say is that 
> there's a price we pay in our community too by not doing it. In this 
> case: tracking embedded versions. Because of the popularity of things 
> like jquery they are embedded a lot and we're trying to track them *and* 
> remove them. Just to clear, I wasn't *only* talking about jquery.js, but 
> also about the others that are covered by binaries in our archive. Even 
> if upstream added stuff back, I would still recommend you to link (and 
> depend on) the files shipped in e.g. libjs-jquery. I know what I talking 
> about, my upstream cacti ships a lot of embedded libraries too; I do my 
> best to remove things that we already ship in Debian. My upstream 
> complained once in a while that my versions are wrong; I still believe 
> it's the right thing to do in a distribution like Debian. I think they 
> are starting to see the value of our side too.
> 
> Lintian is telling you that too:
> https://udd.debian.org/lintian/?packages=openrefine

As I had explained in my previous reply, there is no security impact. Cacti is
a network application and can be accessed remotely by different parties.
Instead openrefine is a privacy focused standalone application. Keeping your
data on your own computer is a feature here. The security team would triage any
Javascript CVE for openrefine as either unimportant or minor because openrefine
is intended to be used for local and single person usage.

Code deduplication is also not a problem, JS files are often small. On the
other hand there is a huge downside for all those Javascript system libraries.
Even minor changes may break existing applications, APIs are not stable,
upstream is unable to help you if you diverge from their version. And then
there is another point which is often neglected in Debian discussions:
developer time. We waste too much time for over-optimizations and often neglect
user experience and maintainability aspects. 

It seems we treat every computer language and every package as it was C only
and try to use always the same hammer because everything looks like a nail. I
believe it is important to differentiate from time to time. 

Regards,

Markus



signature.asc
Description: This is a digitally signed message part


Bug#1034196: unblock: openrefine/3.6.2-2

2023-04-20 Thread Paul Gevers

Hi Markus,

On 20-04-2023 15:21, Markus Koschany wrote:

In version 3.5.x upstream included all Javascript files in the original source
tarball but also shipped some minified files without the unminified sources.


[...]

This was a missing piece. At least it explains how you got where you are 
now.



Since I already followed the Debian Policy and included the missing sources in
debian/missing-sources, I felt that shipping the 3rdparty directory in
debian/missing-sources/3rdparty would be a good intermediate solution.


But you added a more files than you have in testing (including jquery.js).


If you
insist I can repack the tarball, add the 3rdparty directory and remove it from
debian/missing-sources but in the end it would not make any difference.


Huh? I wasn't asking you to do that. I was asking you to use packaged 
binaries as a dependency.



Openrefine is a desktop application which only runs on your own computer.


You know, now I know. Does the security team also know? Should they really?


If
you insist I can depend on libjs-jquery and replace the local copy with a
symlink but I feel this would be an example of over-engineering without any
real value to our users in this specific case.


That argument holds for a lot of things we do. What I try to say is that 
there's a price we pay in our community too by not doing it. In this 
case: tracking embedded versions. Because of the popularity of things 
like jquery they are embedded a lot and we're trying to track them *and* 
remove them. Just to clear, I wasn't *only* talking about jquery.js, but 
also about the others that are covered by binaries in our archive. Even 
if upstream added stuff back, I would still recommend you to link (and 
depend on) the files shipped in e.g. libjs-jquery. I know what I talking 
about, my upstream cacti ships a lot of embedded libraries too; I do my 
best to remove things that we already ship in Debian. My upstream 
complained once in a while that my versions are wrong; I still believe 
it's the right thing to do in a distribution like Debian. I think they 
are starting to see the value of our side too.


Lintian is telling you that too:
https://udd.debian.org/lintian/?packages=openrefine

Paul


OpenPGP_signature
Description: OpenPGP digital signature


Bug#1034196: unblock: openrefine/3.6.2-2

2023-04-20 Thread Markus Koschany
Hello,

Am Donnerstag, dem 20.04.2023 um 11:57 +0200 schrieb Paul Gevers:
> Control: tags -1 moreinfo
> 
> Hi,
> 
> On Mon, 10 Apr 2023 23:55:44 +0200 Markus Koschany  wrote:
> > This unblock is related to #1034127 and the unblock of rhino.
> 
> rhino is now unblocked.

Thank you.

> 
> > The main reason for
> > upgrading from 3.6.1 to 3.6.2 was to include missing Javascript files
> > which are needed to run the web / desktop application of openrefine.
> 
> I *think* you are abusing missing-sources. Quoting policy [1]:
> """
> Sometimes upstream does not include the source code for some files in 
> the upstream tarball. In order to satisfy the DFSG for packages in main 
> or contrib, you should either:
> 
>  repack the upstream tarball to include those sources; or
> 
>  include a copy of the sources in the debian/missing-sources directory.
> """
> But you are *installing* those missing sources.

In version 3.5.x upstream included all Javascript files in the original source
tarball but also shipped some minified files without the unminified sources. I
kindly asked them to change that. They then decided to remove third party
Javascript files completely and download them separately with npm while
building their own version of openrefine. I didn't have the chance to discuss
this change with them yet and how they want to distribute those third party
Javascript files in the future.

Since I already followed the Debian Policy and included the missing sources in
debian/missing-sources, I felt that shipping the 3rdparty directory in
debian/missing-sources/3rdparty would be a good intermediate solution. If you
insist I can repack the tarball, add the 3rdparty directory and remove it from
debian/missing-sources but in the end it would not make any difference. The
debdiff and the functionality would be the same. I feel such a change could be
postponed for the next release cycle when I know upstream's thoughts. 


>  On top of that, you are 
> shipping yet another copy of e.g. jquery.js [2]. Please, if remotely 
> possible, use bin:libjs-jquery (and similar for the other dependencies) 
> instead.

Openrefine is a desktop application which only runs on your own computer.
Openrefine is not affected by any possible security vulnerabilities in jquery
or any other Javascript library hence why it is more beneficial to use a local
copy that is closely integrated and tested with Openrefine. The risk of
breaking the application whenever the system library changes is much higher. If
you insist I can depend on libjs-jquery and replace the local copy with a
symlink but I feel this would be an example of over-engineering without any
real value to our users in this specific case. 

Regards,

Markus


signature.asc
Description: This is a digitally signed message part


Bug#1034196: unblock: openrefine/3.6.2-2

2023-04-20 Thread Paul Gevers

Control: tags -1 moreinfo

Hi,

On Mon, 10 Apr 2023 23:55:44 +0200 Markus Koschany  wrote:

This unblock is related to #1034127 and the unblock of rhino.


rhino is now unblocked.


The main reason for
upgrading from 3.6.1 to 3.6.2 was to include missing Javascript files
which are needed to run the web / desktop application of openrefine.


I *think* you are abusing missing-sources. Quoting policy [1]:
"""
Sometimes upstream does not include the source code for some files in 
the upstream tarball. In order to satisfy the DFSG for packages in main 
or contrib, you should either:


repack the upstream tarball to include those sources; or

include a copy of the sources in the debian/missing-sources directory.
"""
But you are *installing* those missing sources. On top of that, you are 
shipping yet another copy of e.g. jquery.js [2]. Please, if remotely 
possible, use bin:libjs-jquery (and similar for the other dependencies) 
instead.


Paul

[1] 
https://www.debian.org/doc/debian-policy/ch-source.html#missing-sources-debian-missing-sources
[2] 
https://salsa.debian.org/security-tracker-team/security-tracker/-/blob/master/data/embedded-code-copies


OpenPGP_signature
Description: OpenPGP digital signature