Re: Web documentation available offline by default?

[Vincenzo Nicosia]

On Wed, Mar 04, 2020 at 01:49:01PM +0100, Peter N. M. Hansteen wrote:
> On Tue, Mar 03, 2020 at 10:15:31AM -, Stuart Henderson wrote:
> > On 2020-03-02, Peter N. M. Hansteen  wrote:
> > > I was thinking of the probably quite unlikely event that somebody who 
> > > wants this 
> > > comes up with an actually reproducible way that could be turned into an 
> > > otherwise 
> > > unremarkable make target. 
> > 
> > >From experience with other generated files: it won't get used by
> > everyone who updates the faq, meaning that it's another thing that
> > somebody has to watch out for and fix it.
> 
> I'd state the problem more generally: Changing the way people work (their 
> 'workflow')
> is hard.
> 

I genuinely think this is unneded in this particular case. The
documentation team has done a particularly good jon in ensuring that
the html FAQ remains quite consistent throughout, e.g., in the
specification and usage of anchors.

This makes batch conversion quite easy and maintainance-free, as shown
by the relatively primitive script I sent. I am working on it to
include ports and pf, and then post a link to it here.

HTH

Re: Web documentation available offline by default?

[Peter N. M. Hansteen]

On Tue, Mar 03, 2020 at 10:15:31AM -, Stuart Henderson wrote:
> On 2020-03-02, Peter N. M. Hansteen  wrote:
> > I was thinking of the probably quite unlikely event that somebody who wants 
> > this 
> > comes up with an actually reproducible way that could be turned into an 
> > otherwise 
> > unremarkable make target. 
> 
> >From experience with other generated files: it won't get used by
> everyone who updates the faq, meaning that it's another thing that
> somebody has to watch out for and fix it.

I'd state the problem more generally: Changing the way people work (their 
'workflow')
is hard.

I've spent years maintaining documentation with one source format
and several target formats, each one an 'unremarkable make target',
usually html, pdf and text.

I personally saw the benefit of working with a single source format
and several target formats, but each time I tried teaching that particular
toolset to others, the learning curve proved too steep for the 
perceived benefit.

Also, the source files were not in a familiar format such as html or mdoc,
so that specific tool would be pretty much dead in the water here.

(that tool was DocBook SGML, and one of the several docs I maintained
over the years was the early versions of the PF tutorial that eventually
morphed into The Book of PF -- which for totally separate reasons was
written and rewritten using OpenOffice and later LibreOffice).

All the best,
Peter

-- 
Peter N. M. Hansteen, member of the first RFC 1149 implementation team
http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/
"Remember to set the evil bit on all malicious network traffic"
delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.

Re: Web documentation available offline by default?

[Frank Beuth]

On Tue, Mar 03, 2020 at 10:15:31AM -, Stuart Henderson wrote:

On 2020-03-02, Peter N. M. Hansteen  wrote:

I was thinking of the probably quite unlikely event that somebody who wants this
comes up with an actually reproducible way that could be turned into an 
otherwise
unremarkable make target.


From experience with other generated files: it won't get used by
everyone who updates the faq, meaning that it's another thing that
somebody has to watch out for and fix it.

IMHO the only way to fix this is to convert the faq to some other
format that is used to generate a variety of output files (such that
the html files aren't stored in the repository, only the "input"
files, so there's no chance of getting it wrong). And *that* has
enough implications that I don't think it will work well either.


You're right, but I wish you weren't!

Re: Web documentation available offline by default?

[Stuart Henderson]

On 2020-03-02, Peter N. M. Hansteen  wrote:
> I was thinking of the probably quite unlikely event that somebody who wants 
> this 
> comes up with an actually reproducible way that could be turned into an 
> otherwise 
> unremarkable make target. 

>From experience with other generated files: it won't get used by
everyone who updates the faq, meaning that it's another thing that
somebody has to watch out for and fix it.

IMHO the only way to fix this is to convert the faq to some other
format that is used to generate a variety of output files (such that
the html files aren't stored in the repository, only the "input"
files, so there's no chance of getting it wrong). And *that* has
enough implications that I don't think it will work well either.

simple script to merge faq files in a single html (was Re: Web documentation available offline by default?)

[Vincenzo Nicosia]

Please find attached a preliminary rough shell script that does the
job for the faq[0-9]+.html files, keeping track of anchors
appropriately. It is missing pf, ports, and other files, but it's a
starting point.

Disclaimer: this is unofficial stuff and I am not asking for this
script to be supported by OpenBSD or included in the release
workflow. I will probably put the script in my git repo, just in case
somebody wants to use it.

Comments are welcome.

HTH


faq_local.sh
Description: Bourne shell script

Re: Web documentation available offline by default?

[Ottavio Caruso]

On Mon, 2 Mar 2020 at 14:18, Peter N. M. Hansteen  wrote:
>
> On Mon, Mar 02, 2020 at 07:03:25AM -0700, Theo de Raadt wrote:
> > > If you find a way to do that in a way that does not break and require 
> > > manual
> > > labor after each change to the source files, I'm sure your contributing
> > > the code back to the project would be appreciated.
> >
> > I'll say it again, no thanks.
> >
> > Any proposal here requires us to do something.  We don't want to do this.
>
> I was thinking of the probably quite unlikely event that somebody who wants 
> this
> comes up with an actually reproducible way that could be turned into an 
> otherwise
> unremarkable make target.
>
> The mention of a "BSD specialist" certification had me thinking that possibly
> somebody aiming for that status would have been able to think along those 
> lines
> with proper encouragement, if nothing else to automate away an otherwise 
> tedious
> task.

The "BSD specialist" is just an entry-level certification and doesn't
assume that the candidate has the tools and the skills to actually
contribute code to upstream (incidentally, I have submitted bug
reports and small patches to NetBSD and that was it).

For the sake of clarity: I won't propose or submit any changes on this
issue, as this is clearly not welcome. Amen to that and let's move on.


-- 
Ottavio Caruso

Re: Web documentation available offline by default?

[Peter N. M. Hansteen]

On Mon, Mar 02, 2020 at 07:03:25AM -0700, Theo de Raadt wrote:
> > If you find a way to do that in a way that does not break and require 
> > manual 
> > labor after each change to the source files, I'm sure your contributing
> > the code back to the project would be appreciated.
> 
> I'll say it again, no thanks.
> 
> Any proposal here requires us to do something.  We don't want to do this.

I was thinking of the probably quite unlikely event that somebody who wants 
this 
comes up with an actually reproducible way that could be turned into an 
otherwise 
unremarkable make target. 

The mention of a "BSD specialist" certification had me thinking that possibly
somebody aiming for that status would have been able to think along those lines
with proper encouragement, if nothing else to automate away an otherwise tedious
task.

-- 
Peter N. M. Hansteen, member of the first RFC 1149 implementation team
http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/
"Remember to set the evil bit on all malicious network traffic"
delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.

Re: Web documentation available offline by default?

[Theo de Raadt]

Peter N. M. Hansteen  wrote:

> On Sat, Feb 29, 2020 at 03:36:02PM +, Ottavio Caruso wrote:
>  
> > It's also a pity the the faq are not available in a single html or pdf
> > format. This would be handy for those who, like me, are studying for
> > the BSD Specialist certification. Having a single document makes it
> > easier to search for a specific command.
> 
> It is not my intention to be or even sound rude, but this is a matter of
> combining a collection of uniformly formatted html files into one. 
> 
> My first impulse would be to see if there is something available either in 
> the base system or as a package that would either 'just work' for the purpose 
> or
> could be extremely helpful in achieving the feat with a not-too-burdensome
> amount of programming (scripting?).
> 
> If you find a way to do that in a way that does not break and require manual 
> labor after each change to the source files, I'm sure your contributing
> the code back to the project would be appreciated.

I'll say it again, no thanks.

Any proposal here requires us to do something.  We don't want to do this.
We are a multi-faceted operating system project which does not need to
keep adding extra mandates requested by 1 person.

We used to have such a file, btw.  I remember it stopped being updated for
various reasons, probably use of offline tools, and extra steps wasting
the time of the group maintaining the FAQ.  It took years before people
noticed the file had stopped receiving updates.  Which meant noone was
relying on it.  And our reaction was obvious: It was deleted.

Re: Web documentation available offline by default?

[Peter N. M. Hansteen]

On Sat, Feb 29, 2020 at 03:36:02PM +, Ottavio Caruso wrote:
 
> It's also a pity the the faq are not available in a single html or pdf
> format. This would be handy for those who, like me, are studying for
> the BSD Specialist certification. Having a single document makes it
> easier to search for a specific command.

It is not my intention to be or even sound rude, but this is a matter of
combining a collection of uniformly formatted html files into one. 

My first impulse would be to see if there is something available either in 
the base system or as a package that would either 'just work' for the purpose or
could be extremely helpful in achieving the feat with a not-too-burdensome
amount of programming (scripting?).

If you find a way to do that in a way that does not break and require manual 
labor after each change to the source files, I'm sure your contributing
the code back to the project would be appreciated.

All the best,
Peter

-- 
Peter N. M. Hansteen, member of the first RFC 1149 implementation team
http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/
"Remember to set the evil bit on all malicious network traffic"
delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.

Re: Web documentation available offline by default?

[Theo de Raadt]

Vincenzo Nicosia  wrote:

> On Mon, Mar 02, 2020 at 01:30:02AM +0100, Ingo Schwarze wrote:
> 
> [cut]
> 
> > 
> > Besides, the FAQ only applies to -stable and not to -current, so
> > installing it on a -current system would be badly misleading.
> > And we certainly don't want the release(8) process to work differently
> > for -current and -stable: -current is where most of the testing for
> > -stable gets done, so it should better be as similar as possible or
> > we would be in for surprises at release time, or even worse, for
> > surprises after release.
> >
> 
> Indeed. It could probably make sense though to have it as a tar.gz in
> the -stable folder, along with src.tar.gz, sys.tar.gz, ports, and
> xenocara, so that it's readily available through mirrors.

I don't agree, it is a waste of time.

Re: Web documentation available offline by default?

[Vincenzo Nicosia]

On Mon, Mar 02, 2020 at 07:41:56AM -0500, Ian Darwin wrote:

[cut]

> 
> How about if the people who want this would, instead of pitying the fact
> that it's not available in the format you want, create a port (with a
> build depends on wkthmltopdf) to generate the files. And keep the port
> updated regularly or it would be deleted.

That's indeed a good suggestion. I will give a try and post updates
here before submiting to ports@

HTH

Re: Web documentation available offline by default?

[Ian Darwin]

On Mon, Mar 02, 2020 at 12:28:25PM +0100, Wolfgang Pfeiffer wrote:
> > It's also a pity the the faq are not available in a single html or pdf
> > format. This would be handy for those who, like me, are studying for
> > the BSD Specialist certification. Having a single document makes it
> > easier to search for a specific command.
> 
> Seems to work on Linux at least: to "wget" the pages one needs, and
> then "wkhtmltopdf" them to a pdf file.
> 
> Takes time to get it done nicely with the correct flags for
> wkhtmltopdf - and the wget procedure might not get all pages needed,
> so intervening manually might be an option to get those, too ...
> 
> On OBSD ports there's  textproc/wkhtmltopdf. Didn't test the latter
> tho'.
> 

How about if the people who want this would, instead of pitying the fact
that it's not available in the format you want, create a port (with a
build depends on wkthmltopdf) to generate the files. And keep the port
updated regularly or it would be deleted.

Re: Web documentation available offline by default?

[Wolfgang Pfeiffer]

On Sat, Feb 29, 2020 at 03:36:02PM +, Ottavio Caruso wrote:


It's also a pity the the faq are not available in a single html or pdf
format. This would be handy for those who, like me, are studying for
the BSD Specialist certification. Having a single document makes it
easier to search for a specific command.


Seems to work on Linux at least: to "wget" the pages one needs, and
then "wkhtmltopdf" them to a pdf file.

Takes time to get it done nicely with the correct flags for
wkhtmltopdf - and the wget procedure might not get all pages needed,
so intervening manually might be an option to get those, too ...

On OBSD ports there's  textproc/wkhtmltopdf. Didn't test the latter
tho'.

Good luck!
Wolfgang

Re: Web documentation available offline by default?

[Vincenzo Nicosia]

On Mon, Mar 02, 2020 at 01:30:02AM +0100, Ingo Schwarze wrote:

[cut]

> 
> Besides, the FAQ only applies to -stable and not to -current, so
> installing it on a -current system would be badly misleading.
> And we certainly don't want the release(8) process to work differently
> for -current and -stable: -current is where most of the testing for
> -stable gets done, so it should better be as similar as possible or
> we would be in for surprises at release time, or even worse, for
> surprises after release.
>

Indeed. It could probably make sense though to have it as a tar.gz in
the -stable folder, along with src.tar.gz, sys.tar.gz, ports, and
xenocara, so that it's readily available through mirrors.

My2cents

Re: Web documentation available offline by default?

[Ingo Schwarze]

Hi Ottavio,

Ottavio Caruso wrote on Fri, Feb 28, 2020 at 11:27:30AM +:

> Warning: beginner here.

That's OK, as long as you don't feel offended when not every idea
is instantly acted on.  ;-)  (Actually, that applies to experienced
users as well - and even to developers.)

> What about having good old plain text obsd-faq.txt
>  mirrored in
> base system?

That would be inconvenient.  It helps the release(8) process that
the base system - https://cvsweb.openbsd.org/src/ - is a stand-alone
repository.  Reacharound to the www repository doesn't strike me
as desirable.  Remember that the release(8) process is also followed
for snapshot builds, which are done often on multiple architectures.
Keeping that process as simple as possible and requiring as few
dependencies as possible really helps development and avoids
annoying distractions for developers.

Besides, the FAQ only applies to -stable and not to -current, so
installing it on a -current system would be badly misleading.
And we certainly don't want the release(8) process to work differently
for -current and -stable: -current is where most of the testing for
-stable gets done, so it should better be as similar as possible or
we would be in for surprises at release time, or even worse, for
surprises after release.

> Or maybe it's there but I could't find it:
> 
> oc@OpenBSD:~$ find /usr/share/ -iname *faq.txt*
> oc@OpenBSD:~$

No, it isn't there.  But you can easily find it elsewhere
and download it to any place where you need it.

Yours,
  Ingo

Re: Web documentation available offline by default?

[Ottavio Caruso]

On Fri, 28 Feb 2020 at 18:14, Luke A. Call  wrote:
>
> Another option I found helpful once is to use wget to download the
> FAQs' content to a local copy (unless that puts too much load on the server),
> then have a simple local shell alias to view it with links or w3m.
> (At the time, it was a quick way for me, to preserve the content
> in case I wanted it while offline, or if things like X weren't working.)
> There are probably pros & cons of doing that, vs. CVS -- maybe making a
> CVS copy is actually cleaner & simpler for this, and for updating it.
>
> I can fish out my old wget line for that, if it is of interest and not
> considered harmful.

It's also a pity the the faq are not available in a single html or pdf
format. This would be handy for those who, like me, are studying for
the BSD Specialist certification. Having a single document makes it
easier to search for a specific command.

There's a one-page text file on the ftp server but this is quite old
(it doesn't even mention doas).


-- 
Ottavio Caruso

Re: Web documentation available offline by default?

[Luke A. Call]

Another option I found helpful once is to use wget to download the
FAQs' content to a local copy (unless that puts too much load on the server),
then have a simple local shell alias to view it with links or w3m.
(At the time, it was a quick way for me, to preserve the content
in case I wanted it while offline, or if things like X weren't working.)
There are probably pros & cons of doing that, vs. CVS -- maybe making a 
CVS copy is actually cleaner & simpler for this, and for updating it.

I can fish out my old wget line for that, if it is of interest and not
considered harmful.

--  
 
Luke Call   
 
My general thoughts:  http://lukecall.net  (updated 2020-02-18)

Re: Web documentation available offline by default?

[prx]

I made this : 
https://ybad.name/Logiciel-libre/OpenBSD/FAQ-offline.html

Le 28 février 2020 12:27:30 GMT+01:00, Ottavio Caruso 
 a écrit :
>On Fri, 28 Feb 2020 at 06:24, Ingo Schwarze  wrote:
>>
>> Hi Frank,
>>
>> Frank Beuth wrote on Fri, Feb 28, 2020 at 04:22:27AM +:
>>
>> > Is the web documentation (FAQ etc) included in the base system by
>> > default anywhere,
>>
>> No it isn't.
>>
>[...]
>>
>> We don't want to lose the valued contributions of those developers
>> who actually spend all the work maintaining the FAQ or make their
>> work any harder than it is now.
>
>Warning: beginner here.
>
>What about having good old plain text obsd-faq.txt
> mirrored in
>base system?
>
>Or maybe it's there but I could't find it:
>
>oc@OpenBSD:~$ find /usr/share/ -iname *faq.txt*
>oc@OpenBSD:~$

Re: Web documentation available offline by default?

[Ottavio Caruso]

On Fri, 28 Feb 2020 at 06:24, Ingo Schwarze  wrote:
>
> Hi Frank,
>
> Frank Beuth wrote on Fri, Feb 28, 2020 at 04:22:27AM +:
>
> > Is the web documentation (FAQ etc) included in the base system by
> > default anywhere,
>
> No it isn't.
>
[...]
>
> We don't want to lose the valued contributions of those developers
> who actually spend all the work maintaining the FAQ or make their
> work any harder than it is now.

Warning: beginner here.

What about having good old plain text obsd-faq.txt
 mirrored in
base system?

Or maybe it's there but I could't find it:

oc@OpenBSD:~$ find /usr/share/ -iname *faq.txt*
oc@OpenBSD:~$


-- 
Ottavio Caruso

Re: Web documentation available offline by default?

[lists]

Fri, 28 Feb 2020 07:54:49 + Frank Beuth 
> On Fri, Feb 28, 2020 at 07:24:50AM +0100, Ingo Schwarze wrote:
> >Hi Frank,
> >
> >Frank Beuth wrote on Fri, Feb 28, 2020 at 04:22:27AM +:
> >  
> >> Is the web documentation (FAQ etc) included in the base system by
> >> default anywhere,  
> >
> >No it isn't.
> >
> >I offered some years ago to translate the FAQ from HTML to mdoc(7)
> >and to include it in /usr/share/man/faq/ such that it would become
> >available for both -current and -stable both online and offline
> >without additional maintenance effort just like any other documentation
> >and such that it would automatically be included in apropos(1)
> >searches, but the proposal was rejected because the developers who
> >actually maintain the content of the FAQ consider it easier to
> >maintain in HTML than in mdoc(7) format.
> >
> >We don't want to lose the valued contributions of those developers
> >who actually spend all the work maintaining the FAQ or make their
> >work any harder than it is now.  
> 
> Thanks. Too bad the mdoc idea failed!
> 

Hi Frank,

Maybe it just got deferred for the moment of more elegant (simplified) mandoc
markup frontend editing.  Editing HTML by hand is inconvenient + highly error
prone at the same time as shown in many iterations of missing closing tag and
other numerous errors hidden by the viewing pagers (error covering browsers).

Editing HTML by hand should be reserved for fine tuning and maintenance of an
export filter of a document specific authoring and editing convenient markup.

Using mandoc for the purpose of system and accompanying documentation is "the
best" current practice solution, it was designed for/in this specific domain.

Lightweight markup formats are an enabler for document authoring and editing,
as demonstrated by the largest online collaborative systems (encyclopaedias).

At the same time, lightweight markup can optionally contain semantic (mandoc)
markup and export cleanly to HTML, as mandoc can export to lightweight markup
and HTML and other preformated (typeset) document output formats.  An missing
at the time system has never been an impediment to outstanding, and excelling
work in the maintenance and preparation of the documentation as if it's here.

Kind regards,
Anton Lazarov
MScEng EECSIT

Re: Web documentation available offline by default?

[Frank Beuth]

On Fri, Feb 28, 2020 at 07:24:50AM +0100, Ingo Schwarze wrote:

Hi Frank,

Frank Beuth wrote on Fri, Feb 28, 2020 at 04:22:27AM +:


Is the web documentation (FAQ etc) included in the base system by
default anywhere,


No it isn't.

I offered some years ago to translate the FAQ from HTML to mdoc(7)
and to include it in /usr/share/man/faq/ such that it would become
available for both -current and -stable both online and offline
without additional maintenance effort just like any other documentation
and such that it would automatically be included in apropos(1)
searches, but the proposal was rejected because the developers who
actually maintain the content of the FAQ consider it easier to
maintain in HTML than in mdoc(7) format.

We don't want to lose the valued contributions of those developers
who actually spend all the work maintaining the FAQ or make their
work any harder than it is now.


Thanks. Too bad the mdoc idea failed!

Re: Web documentation available offline by default?

[Ingo Schwarze]

Hi Frank,

Frank Beuth wrote on Fri, Feb 28, 2020 at 04:22:27AM +:

> Is the web documentation (FAQ etc) included in the base system by 
> default anywhere,

No it isn't.

I offered some years ago to translate the FAQ from HTML to mdoc(7)
and to include it in /usr/share/man/faq/ such that it would become
available for both -current and -stable both online and offline
without additional maintenance effort just like any other documentation
and such that it would automatically be included in apropos(1)
searches, but the proposal was rejected because the developers who
actually maintain the content of the FAQ consider it easier to
maintain in HTML than in mdoc(7) format.

We don't want to lose the valued contributions of those developers
who actually spend all the work maintaining the FAQ or make their
work any harder than it is now.

> or do we have to pull it from CVS manually?

That would be one simple option, yes.

Yours,
  Ingo

Web documentation available offline by default?

[Frank Beuth]

Is the web documentation (FAQ etc) included in the base system by 
default anywhere, or do we have to pull it from CVS manually?