Re: Site generation ideas - JSON and Jekyll

[sebb]

On 19 April 2018 at 21:40, Jan Iversen  wrote:
>
>
> Sent from my iPad
>
>> On 19 Apr 2018, at 22:22, sebb  wrote:
>>
>>> On 19 April 2018 at 21:08, Jan Iversen  wrote:
>>>
>>>
>>> Sent from my iPad
>>>
> On 19 Apr 2018, at 22:02, sebb  wrote:
>
> On 19 April 2018 at 19:41, Jan Iversen  wrote:
>
>
> Sent from my iPad
>
>>> On 19 Apr 2018, at 17:10, sebb  wrote:
>>>
>>> On 19 April 2018 at 13:27, sebb  wrote:
>>> I have done some further investigation of using JSON and Jekyll.
>>>
>>> I think JSON is not the best format for maintenance.
>>> This is because it is messy including chunks of text (e.g. for
>>> additional info on the project).
>>> Also it does not allow any comments.
>>> The format is rather strict, with lots of quotes needed, and brackets
>>> and braces.
>>>
>>> I think we should use YAML for the raw data, and (if necessary)
>>> extract a subset into a JSON file for external consumption.
> Personally I think it is a mistake. json is rather simple, and are 
> supported everywhere. Yaml on the other hand is less supported (e.g. 
> looked for an online validator without success). Safari and xcode formats 
> json nicely whereas yaml is viewed as raw text.

 The problem is that JSON gets very messy for anything but a short text 
 string.
 This is not particularly obvious with the current contents, because
 the longest text string is the description.
 But even now, some of the lines are very long.
>>> I do not understand what “messy” means, json allows multiple lines text 
>>> (see in my attic.js) using a ` instead of a “, that way yaml and json are 
>>> equal “messy”.
>>
>> The attic.js file is not JSON format; it is Javascript source. JS is
>> much more flexible.
>>
>>> I have another project where json is used to store translations (loaded 
>>> with php), and that is sometimes 3/4 of a page, easy readable due to the 
>>> multiline feature.
>>
>> Again, I assume the JSON is embedded using PHP syntax.
>
> well in both cases you assume wrong, if you save the json part of the js file 
> it validates as valid json.

I think we are talking about different things.

I agree the JSON in attic.js is spread over multiple lines.
But *values* are confined to a single line; even very long strings
have to be on a single line.
That is fine for short values, but gets very messy with longer text blocks.

For example the description field here:

http://svn.apache.org/viewvc/attic/site/docs/scripts/attic.js?revision=1829196=markup=1829368#l71

This is not easy to read unless one has an editor that softwraps the display.

And JSON can get really messy if the text contains quotes.

> In the other project we use a pure json file. json was extended some years 
> ago with the multiline facility.

I cannot find any reference to multi-line JSON strings.
AFAICT the only multi-line aspect of the syntax is that whitespace
(including new lines) is allowed between any two tokens.

However a token cannot contain a real new-line.

> rgds
> jan i
>>
>>> but please see this as just a factual correction, if you prefer yaml I am 
>>> the last to go against it.
>>>
>>
>> Have a look at some of the .md files and see.
>> I think they are much simpler.
>>
>>> rgds
>>> jan i

> Why do we need comments in the file, it has not been needed until now, so 
> let us not introduce “hidden” data.

 Allowing comments is just an additional benefit; it's not the reason
 for choosing YAML.

 Sometimes it's useful to be able to add a TODO to a part of a data file.
 Or explain why a particular value is present.
 But again; not the reason for changing.

> but of course it is a matter of taste (just like js was).
>
> extracting a json file from the yaml file seems to be overdoing it.

 Again, only if necessary.

>>>
>>> As to Jekyll:
>>>
>>> Jekyll can equally use a YAML data file, so it is not a problem
>>> changing to YAML.
>>>
>>> At present the attic-test PoC includes a single JSON data file which
>>> is processed in a plugin script that generates the individual page
>>> data.
>>>
>>> This works well (and it looks like BuildBot supports the use of Jekyll
>>> plugin scripts - other sites such as GitHub may not)
>>>
>>> But I think it would be better to have a separate YAML file per project.
>>>
>>> Jekyll can process these as part of a collection.
>>> This avoids the need to use a plugin to generate the pages.
>>> I think it also makes it a bit more obvious what is going on (each
>>> output file has an input file)
>>>
>>> And the YAML body can contain arbitrary markup to be added to the
>>> generated page.
>>> This would make it easier to preserve the 

Re: Site generation ideas - JSON and Jekyll

[Jan Iversen]

Sent from my iPad

> On 19 Apr 2018, at 22:22, sebb  wrote:
> 
>> On 19 April 2018 at 21:08, Jan Iversen  wrote:
>> 
>> 
>> Sent from my iPad
>> 
 On 19 Apr 2018, at 22:02, sebb  wrote:
 
 On 19 April 2018 at 19:41, Jan Iversen  wrote:
 
 
 Sent from my iPad
 
>> On 19 Apr 2018, at 17:10, sebb  wrote:
>> 
>> On 19 April 2018 at 13:27, sebb  wrote:
>> I have done some further investigation of using JSON and Jekyll.
>> 
>> I think JSON is not the best format for maintenance.
>> This is because it is messy including chunks of text (e.g. for
>> additional info on the project).
>> Also it does not allow any comments.
>> The format is rather strict, with lots of quotes needed, and brackets
>> and braces.
>> 
>> I think we should use YAML for the raw data, and (if necessary)
>> extract a subset into a JSON file for external consumption.
 Personally I think it is a mistake. json is rather simple, and are 
 supported everywhere. Yaml on the other hand is less supported (e.g. 
 looked for an online validator without success). Safari and xcode formats 
 json nicely whereas yaml is viewed as raw text.
>>> 
>>> The problem is that JSON gets very messy for anything but a short text 
>>> string.
>>> This is not particularly obvious with the current contents, because
>>> the longest text string is the description.
>>> But even now, some of the lines are very long.
>> I do not understand what “messy” means, json allows multiple lines text (see 
>> in my attic.js) using a ` instead of a “, that way yaml and json are equal 
>> “messy”.
> 
> The attic.js file is not JSON format; it is Javascript source. JS is
> much more flexible.
> 
>> I have another project where json is used to store translations (loaded with 
>> php), and that is sometimes 3/4 of a page, easy readable due to the 
>> multiline feature.
> 
> Again, I assume the JSON is embedded using PHP syntax.

well in both cases you assume wrong, if you save the json part of the js file 
it validates as valid json. In the other project we use a pure json file. json 
was extended some years ago with the multiline facility.

rgds
jan i
> 
>> but please see this as just a factual correction, if you prefer yaml I am 
>> the last to go against it.
>> 
> 
> Have a look at some of the .md files and see.
> I think they are much simpler.
> 
>> rgds
>> jan i
>>> 
 Why do we need comments in the file, it has not been needed until now, so 
 let us not introduce “hidden” data.
>>> 
>>> Allowing comments is just an additional benefit; it's not the reason
>>> for choosing YAML.
>>> 
>>> Sometimes it's useful to be able to add a TODO to a part of a data file.
>>> Or explain why a particular value is present.
>>> But again; not the reason for changing.
>>> 
 but of course it is a matter of taste (just like js was).
 
 extracting a json file from the yaml file seems to be overdoing it.
>>> 
>>> Again, only if necessary.
>>> 
>> 
>> As to Jekyll:
>> 
>> Jekyll can equally use a YAML data file, so it is not a problem
>> changing to YAML.
>> 
>> At present the attic-test PoC includes a single JSON data file which
>> is processed in a plugin script that generates the individual page
>> data.
>> 
>> This works well (and it looks like BuildBot supports the use of Jekyll
>> plugin scripts - other sites such as GitHub may not)
>> 
>> But I think it would be better to have a separate YAML file per project.
>> 
>> Jekyll can process these as part of a collection.
>> This avoids the need to use a plugin to generate the pages.
>> I think it also makes it a bit more obvious what is going on (each
>> output file has an input file)
>> 
>> And the YAML body can contain arbitrary markup to be added to the
>> generated page.
>> This would make it easier to preserve the extra information present in
>> some of the existing xml files
>> 
>> A question:
>> On a tablet, would it be harder to maintain one file per project
>> rather than a single large file?
 significantly ! that was one of the major reasons for my js solution.
 
>>> 
>>> I don't understand.
>>> Why is a single file easier?
>>> Surely each new retirement just needs to have a new file? Does it
>>> matter that there is no other data in it?
>>> 
>>> I tried adding more data from the XML files to see if I could
>>> reproduce all the information on the current pages.
>>> It's really difficult to ensure that all the quoting is done
>>> correctly; a single misplaced quote affects the entire file.
>>> Eclipse started having problems, but even in a plain text editor it
>>> was awkward to handle.
>>> 
 but let us focus on the solution wanted by the community and what a single 
 person needs.
 
 

Re: Site generation ideas - JSON and Jekyll

[sebb]

On 19 April 2018 at 21:08, Jan Iversen  wrote:
>
>
> Sent from my iPad
>
>> On 19 Apr 2018, at 22:02, sebb  wrote:
>>
>>> On 19 April 2018 at 19:41, Jan Iversen  wrote:
>>>
>>>
>>> Sent from my iPad
>>>
> On 19 Apr 2018, at 17:10, sebb  wrote:
>
> On 19 April 2018 at 13:27, sebb  wrote:
> I have done some further investigation of using JSON and Jekyll.
>
> I think JSON is not the best format for maintenance.
> This is because it is messy including chunks of text (e.g. for
> additional info on the project).
> Also it does not allow any comments.
> The format is rather strict, with lots of quotes needed, and brackets
> and braces.
>
> I think we should use YAML for the raw data, and (if necessary)
> extract a subset into a JSON file for external consumption.
>>> Personally I think it is a mistake. json is rather simple, and are 
>>> supported everywhere. Yaml on the other hand is less supported (e.g. looked 
>>> for an online validator without success). Safari and xcode formats json 
>>> nicely whereas yaml is viewed as raw text.
>>
>> The problem is that JSON gets very messy for anything but a short text 
>> string.
>> This is not particularly obvious with the current contents, because
>> the longest text string is the description.
>> But even now, some of the lines are very long.
> I do not understand what “messy” means, json allows multiple lines text (see 
> in my attic.js) using a ` instead of a “, that way yaml and json are equal 
> “messy”.

The attic.js file is not JSON format; it is Javascript source. JS is
much more flexible.

> I have another project where json is used to store translations (loaded with 
> php), and that is sometimes 3/4 of a page, easy readable due to the multiline 
> feature.

Again, I assume the JSON is embedded using PHP syntax.

> but please see this as just a factual correction, if you prefer yaml I am the 
> last to go against it.
>

Have a look at some of the .md files and see.
I think they are much simpler.

> rgds
> jan i
>>
>>> Why do we need comments in the file, it has not been needed until now, so 
>>> let us not introduce “hidden” data.
>>
>> Allowing comments is just an additional benefit; it's not the reason
>> for choosing YAML.
>>
>> Sometimes it's useful to be able to add a TODO to a part of a data file.
>> Or explain why a particular value is present.
>> But again; not the reason for changing.
>>
>>> but of course it is a matter of taste (just like js was).
>>>
>>> extracting a json file from the yaml file seems to be overdoing it.
>>
>> Again, only if necessary.
>>
>
> As to Jekyll:
>
> Jekyll can equally use a YAML data file, so it is not a problem
> changing to YAML.
>
> At present the attic-test PoC includes a single JSON data file which
> is processed in a plugin script that generates the individual page
> data.
>
> This works well (and it looks like BuildBot supports the use of Jekyll
> plugin scripts - other sites such as GitHub may not)
>
> But I think it would be better to have a separate YAML file per project.
>
> Jekyll can process these as part of a collection.
> This avoids the need to use a plugin to generate the pages.
> I think it also makes it a bit more obvious what is going on (each
> output file has an input file)
>
> And the YAML body can contain arbitrary markup to be added to the
> generated page.
> This would make it easier to preserve the extra information present in
> some of the existing xml files
>
> A question:
> On a tablet, would it be harder to maintain one file per project
> rather than a single large file?
>>> significantly ! that was one of the major reasons for my js solution.
>>>
>>
>> I don't understand.
>> Why is a single file easier?
>> Surely each new retirement just needs to have a new file? Does it
>> matter that there is no other data in it?
>>
>> I tried adding more data from the XML files to see if I could
>> reproduce all the information on the current pages.
>> It's really difficult to ensure that all the quoting is done
>> correctly; a single misplaced quote affects the entire file.
>> Eclipse started having problems, but even in a plain text editor it
>> was awkward to handle.
>>
>>> but let us focus on the solution wanted by the community and what a single 
>>> person needs.
>>>
>>> rgds
>>> jan i
>
> i.e. instead of updating projects.json one would need to create/update
> a projects/project.md file.

 I have updated the attic-test tree with a YAML-based version:

 https://svn.apache.org/repos/asf/attic/site-test/yaml

 The yaml.sh script in the parent directory will create the output in 
 docs3/.
 This should currently be identical to the output in docs2/ - apart
 from the date in feed.xml

 If this 

Re: Site generation ideas - JSON and Jekyll

[sebb]

On 19 April 2018 at 19:43, Jan Iversen  wrote:
>
>
> Sent from my iPad
>
>> On 19 Apr 2018, at 17:10, sebb  wrote:
>>
>>> On 19 April 2018 at 13:27, sebb  wrote:
>>> I have done some further investigation of using JSON and Jekyll.
>>>
>>> I think JSON is not the best format for maintenance.
>>> This is because it is messy including chunks of text (e.g. for
>>> additional info on the project).
>>> Also it does not allow any comments.
>>> The format is rather strict, with lots of quotes needed, and brackets
>>> and braces.
>>>
>>> I think we should use YAML for the raw data, and (if necessary)
>>> extract a subset into a JSON file for external consumption.
>>>
>>> As to Jekyll:
>>>
>>> Jekyll can equally use a YAML data file, so it is not a problem
>>> changing to YAML.
>>>
>>> At present the attic-test PoC includes a single JSON data file which
>>> is processed in a plugin script that generates the individual page
>>> data.
>>>
>>> This works well (and it looks like BuildBot supports the use of Jekyll
>>> plugin scripts - other sites such as GitHub may not)
>>>
>>> But I think it would be better to have a separate YAML file per project.
>>>
>>> Jekyll can process these as part of a collection.
>>> This avoids the need to use a plugin to generate the pages.
>>> I think it also makes it a bit more obvious what is going on (each
>>> output file has an input file)
>>>
>>> And the YAML body can contain arbitrary markup to be added to the
>>> generated page.
>>> This would make it easier to preserve the extra information present in
>>> some of the existing xml files
>>>
>>> A question:
>>> On a tablet, would it be harder to maintain one file per project
>>> rather than a single large file?
>>>
>>> i.e. instead of updating projects.json one would need to create/update
>>> a projects/project.md file.
>>
>> I have updated the attic-test tree with a YAML-based version:
>>
>> https://svn.apache.org/repos/asf/attic/site-test/yaml
> is this a correct link, seems To miss yaml files and still contains a 
> projects.json ?

Yes, it is correct.

The files have YAML headers which set up the variables for the template.
For example:
https://svn.apache.org/repos/asf/attic/site-test/yaml/_projects/abdera.md

The rest of each file can be used for additional free-form content if required.

I kept the data file projects.json purely to ensure that the projects
list was created in the same order (which is not completely
alphabetical)
This makes it easier to compare the output.

> rgds
> jan i
>>
>> The yaml.sh script in the parent directory will create the output in docs3/.
>> This should currently be identical to the output in docs2/ - apart
>> from the date in feed.xml
>>
>> If this looks like it will be suitable for use when working from
>> tablets, I can tidy it up and start migrating some of the additional
>> text from the xdocs/projects files
>>
>> If not, please advise what needs to be done to make it possible to
>> update the site from a tablet.

Re: Site generation ideas - JSON and Jekyll

[Jan Iversen]

Sent from my iPad

> On 19 Apr 2018, at 22:02, sebb  wrote:
> 
>> On 19 April 2018 at 19:41, Jan Iversen  wrote:
>> 
>> 
>> Sent from my iPad
>> 
 On 19 Apr 2018, at 17:10, sebb  wrote:
 
 On 19 April 2018 at 13:27, sebb  wrote:
 I have done some further investigation of using JSON and Jekyll.
 
 I think JSON is not the best format for maintenance.
 This is because it is messy including chunks of text (e.g. for
 additional info on the project).
 Also it does not allow any comments.
 The format is rather strict, with lots of quotes needed, and brackets
 and braces.
 
 I think we should use YAML for the raw data, and (if necessary)
 extract a subset into a JSON file for external consumption.
>> Personally I think it is a mistake. json is rather simple, and are supported 
>> everywhere. Yaml on the other hand is less supported (e.g. looked for an 
>> online validator without success). Safari and xcode formats json nicely 
>> whereas yaml is viewed as raw text.
> 
> The problem is that JSON gets very messy for anything but a short text string.
> This is not particularly obvious with the current contents, because
> the longest text string is the description.
> But even now, some of the lines are very long.
I do not understand what “messy” means, json allows multiple lines text (see in 
my attic.js) using a ` instead of a “, that way yaml and json are equal “messy”.

I have another project where json is used to store translations (loaded with 
php), and that is sometimes 3/4 of a page, easy readable due to the multiline 
feature.

but please see this as just a factual correction, if you prefer yaml I am the 
last to go against it.

rgds
jan i
> 
>> Why do we need comments in the file, it has not been needed until now, so 
>> let us not introduce “hidden” data.
> 
> Allowing comments is just an additional benefit; it's not the reason
> for choosing YAML.
> 
> Sometimes it's useful to be able to add a TODO to a part of a data file.
> Or explain why a particular value is present.
> But again; not the reason for changing.
> 
>> but of course it is a matter of taste (just like js was).
>> 
>> extracting a json file from the yaml file seems to be overdoing it.
> 
> Again, only if necessary.
> 
 
 As to Jekyll:
 
 Jekyll can equally use a YAML data file, so it is not a problem
 changing to YAML.
 
 At present the attic-test PoC includes a single JSON data file which
 is processed in a plugin script that generates the individual page
 data.
 
 This works well (and it looks like BuildBot supports the use of Jekyll
 plugin scripts - other sites such as GitHub may not)
 
 But I think it would be better to have a separate YAML file per project.
 
 Jekyll can process these as part of a collection.
 This avoids the need to use a plugin to generate the pages.
 I think it also makes it a bit more obvious what is going on (each
 output file has an input file)
 
 And the YAML body can contain arbitrary markup to be added to the
 generated page.
 This would make it easier to preserve the extra information present in
 some of the existing xml files
 
 A question:
 On a tablet, would it be harder to maintain one file per project
 rather than a single large file?
>> significantly ! that was one of the major reasons for my js solution.
>> 
> 
> I don't understand.
> Why is a single file easier?
> Surely each new retirement just needs to have a new file? Does it
> matter that there is no other data in it?
> 
> I tried adding more data from the XML files to see if I could
> reproduce all the information on the current pages.
> It's really difficult to ensure that all the quoting is done
> correctly; a single misplaced quote affects the entire file.
> Eclipse started having problems, but even in a plain text editor it
> was awkward to handle.
> 
>> but let us focus on the solution wanted by the community and what a single 
>> person needs.
>> 
>> rgds
>> jan i
 
 i.e. instead of updating projects.json one would need to create/update
 a projects/project.md file.
>>> 
>>> I have updated the attic-test tree with a YAML-based version:
>>> 
>>> https://svn.apache.org/repos/asf/attic/site-test/yaml
>>> 
>>> The yaml.sh script in the parent directory will create the output in docs3/.
>>> This should currently be identical to the output in docs2/ - apart
>>> from the date in feed.xml
>>> 
>>> If this looks like it will be suitable for use when working from
>>> tablets, I can tidy it up and start migrating some of the additional
>>> text from the xdocs/projects files
>>> 
>>> If not, please advise what needs to be done to make it possible to
>>> update the site from a tablet.

Re: Site generation ideas - JSON and Jekyll

[sebb]

On 19 April 2018 at 19:41, Jan Iversen  wrote:
>
>
> Sent from my iPad
>
>> On 19 Apr 2018, at 17:10, sebb  wrote:
>>
>>> On 19 April 2018 at 13:27, sebb  wrote:
>>> I have done some further investigation of using JSON and Jekyll.
>>>
>>> I think JSON is not the best format for maintenance.
>>> This is because it is messy including chunks of text (e.g. for
>>> additional info on the project).
>>> Also it does not allow any comments.
>>> The format is rather strict, with lots of quotes needed, and brackets
>>> and braces.
>>>
>>> I think we should use YAML for the raw data, and (if necessary)
>>> extract a subset into a JSON file for external consumption.
> Personally I think it is a mistake. json is rather simple, and are supported 
> everywhere. Yaml on the other hand is less supported (e.g. looked for an 
> online validator without success). Safari and xcode formats json nicely 
> whereas yaml is viewed as raw text.

The problem is that JSON gets very messy for anything but a short text string.
This is not particularly obvious with the current contents, because
the longest text string is the description.
But even now, some of the lines are very long.

> Why do we need comments in the file, it has not been needed until now, so let 
> us not introduce “hidden” data.

Allowing comments is just an additional benefit; it's not the reason
for choosing YAML.

Sometimes it's useful to be able to add a TODO to a part of a data file.
Or explain why a particular value is present.
But again; not the reason for changing.

> but of course it is a matter of taste (just like js was).
>
> extracting a json file from the yaml file seems to be overdoing it.

Again, only if necessary.

>>>
>>> As to Jekyll:
>>>
>>> Jekyll can equally use a YAML data file, so it is not a problem
>>> changing to YAML.
>>>
>>> At present the attic-test PoC includes a single JSON data file which
>>> is processed in a plugin script that generates the individual page
>>> data.
>>>
>>> This works well (and it looks like BuildBot supports the use of Jekyll
>>> plugin scripts - other sites such as GitHub may not)
>>>
>>> But I think it would be better to have a separate YAML file per project.
>>>
>>> Jekyll can process these as part of a collection.
>>> This avoids the need to use a plugin to generate the pages.
>>> I think it also makes it a bit more obvious what is going on (each
>>> output file has an input file)
>>>
>>> And the YAML body can contain arbitrary markup to be added to the
>>> generated page.
>>> This would make it easier to preserve the extra information present in
>>> some of the existing xml files
>>>
>>> A question:
>>> On a tablet, would it be harder to maintain one file per project
>>> rather than a single large file?
> significantly ! that was one of the major reasons for my js solution.
>

I don't understand.
Why is a single file easier?
Surely each new retirement just needs to have a new file? Does it
matter that there is no other data in it?

I tried adding more data from the XML files to see if I could
reproduce all the information on the current pages.
It's really difficult to ensure that all the quoting is done
correctly; a single misplaced quote affects the entire file.
Eclipse started having problems, but even in a plain text editor it
was awkward to handle.

> but let us focus on the solution wanted by the community and what a single 
> person needs.
>
> rgds
> jan i
>>>
>>> i.e. instead of updating projects.json one would need to create/update
>>> a projects/project.md file.
>>
>> I have updated the attic-test tree with a YAML-based version:
>>
>> https://svn.apache.org/repos/asf/attic/site-test/yaml
>>
>> The yaml.sh script in the parent directory will create the output in docs3/.
>> This should currently be identical to the output in docs2/ - apart
>> from the date in feed.xml
>>
>> If this looks like it will be suitable for use when working from
>> tablets, I can tidy it up and start migrating some of the additional
>> text from the xdocs/projects files
>>
>> If not, please advise what needs to be done to make it possible to
>> update the site from a tablet.

Re: Site generation ideas - JSON and Jekyll

[Jan Iversen]

Sent from my iPad

> On 19 Apr 2018, at 17:10, sebb  wrote:
> 
>> On 19 April 2018 at 13:27, sebb  wrote:
>> I have done some further investigation of using JSON and Jekyll.
>> 
>> I think JSON is not the best format for maintenance.
>> This is because it is messy including chunks of text (e.g. for
>> additional info on the project).
>> Also it does not allow any comments.
>> The format is rather strict, with lots of quotes needed, and brackets
>> and braces.
>> 
>> I think we should use YAML for the raw data, and (if necessary)
>> extract a subset into a JSON file for external consumption.
>> 
>> As to Jekyll:
>> 
>> Jekyll can equally use a YAML data file, so it is not a problem
>> changing to YAML.
>> 
>> At present the attic-test PoC includes a single JSON data file which
>> is processed in a plugin script that generates the individual page
>> data.
>> 
>> This works well (and it looks like BuildBot supports the use of Jekyll
>> plugin scripts - other sites such as GitHub may not)
>> 
>> But I think it would be better to have a separate YAML file per project.
>> 
>> Jekyll can process these as part of a collection.
>> This avoids the need to use a plugin to generate the pages.
>> I think it also makes it a bit more obvious what is going on (each
>> output file has an input file)
>> 
>> And the YAML body can contain arbitrary markup to be added to the
>> generated page.
>> This would make it easier to preserve the extra information present in
>> some of the existing xml files
>> 
>> A question:
>> On a tablet, would it be harder to maintain one file per project
>> rather than a single large file?
>> 
>> i.e. instead of updating projects.json one would need to create/update
>> a projects/project.md file.
> 
> I have updated the attic-test tree with a YAML-based version:
> 
> https://svn.apache.org/repos/asf/attic/site-test/yaml
is this a correct link, seems To miss yaml files and still contains a 
projects.json ?
rgds
jan i
> 
> The yaml.sh script in the parent directory will create the output in docs3/.
> This should currently be identical to the output in docs2/ - apart
> from the date in feed.xml
> 
> If this looks like it will be suitable for use when working from
> tablets, I can tidy it up and start migrating some of the additional
> text from the xdocs/projects files
> 
> If not, please advise what needs to be done to make it possible to
> update the site from a tablet.

Re: Site generation ideas - JSON and Jekyll

[Jan Iversen]

Sent from my iPad

> On 19 Apr 2018, at 17:10, sebb  wrote:
> 
>> On 19 April 2018 at 13:27, sebb  wrote:
>> I have done some further investigation of using JSON and Jekyll.
>> 
>> I think JSON is not the best format for maintenance.
>> This is because it is messy including chunks of text (e.g. for
>> additional info on the project).
>> Also it does not allow any comments.
>> The format is rather strict, with lots of quotes needed, and brackets
>> and braces.
>> 
>> I think we should use YAML for the raw data, and (if necessary)
>> extract a subset into a JSON file for external consumption.
Personally I think it is a mistake. json is rather simple, and are supported 
everywhere. Yaml on the other hand is less supported (e.g. looked for an online 
validator without success). Safari and xcode formats json nicely whereas yaml 
is viewed as raw text.

Why do we need comments in the file, it has not been needed until now, so let 
us not introduce “hidden” data.

but of course it is a matter of taste (just like js was).

extracting a json file from the yaml file seems to be overdoing it.
>> 
>> As to Jekyll:
>> 
>> Jekyll can equally use a YAML data file, so it is not a problem
>> changing to YAML.
>> 
>> At present the attic-test PoC includes a single JSON data file which
>> is processed in a plugin script that generates the individual page
>> data.
>> 
>> This works well (and it looks like BuildBot supports the use of Jekyll
>> plugin scripts - other sites such as GitHub may not)
>> 
>> But I think it would be better to have a separate YAML file per project.
>> 
>> Jekyll can process these as part of a collection.
>> This avoids the need to use a plugin to generate the pages.
>> I think it also makes it a bit more obvious what is going on (each
>> output file has an input file)
>> 
>> And the YAML body can contain arbitrary markup to be added to the
>> generated page.
>> This would make it easier to preserve the extra information present in
>> some of the existing xml files
>> 
>> A question:
>> On a tablet, would it be harder to maintain one file per project
>> rather than a single large file?
significantly ! that was one of the major reasons for my js solution.

but let us focus on the solution wanted by the community and what a single 
person needs.

rgds
jan i
>> 
>> i.e. instead of updating projects.json one would need to create/update
>> a projects/project.md file.
> 
> I have updated the attic-test tree with a YAML-based version:
> 
> https://svn.apache.org/repos/asf/attic/site-test/yaml
> 
> The yaml.sh script in the parent directory will create the output in docs3/.
> This should currently be identical to the output in docs2/ - apart
> from the date in feed.xml
> 
> If this looks like it will be suitable for use when working from
> tablets, I can tidy it up and start migrating some of the additional
> text from the xdocs/projects files
> 
> If not, please advise what needs to be done to make it possible to
> update the site from a tablet.

Re: Site generation ideas - JSON and Jekyll

[sebb]

On 19 April 2018 at 13:27, sebb  wrote:
> I have done some further investigation of using JSON and Jekyll.
>
> I think JSON is not the best format for maintenance.
> This is because it is messy including chunks of text (e.g. for
> additional info on the project).
> Also it does not allow any comments.
> The format is rather strict, with lots of quotes needed, and brackets
> and braces.
>
> I think we should use YAML for the raw data, and (if necessary)
> extract a subset into a JSON file for external consumption.
>
> As to Jekyll:
>
> Jekyll can equally use a YAML data file, so it is not a problem
> changing to YAML.
>
> At present the attic-test PoC includes a single JSON data file which
> is processed in a plugin script that generates the individual page
> data.
>
> This works well (and it looks like BuildBot supports the use of Jekyll
> plugin scripts - other sites such as GitHub may not)
>
> But I think it would be better to have a separate YAML file per project.
>
> Jekyll can process these as part of a collection.
> This avoids the need to use a plugin to generate the pages.
> I think it also makes it a bit more obvious what is going on (each
> output file has an input file)
>
> And the YAML body can contain arbitrary markup to be added to the
> generated page.
> This would make it easier to preserve the extra information present in
> some of the existing xml files
>
> A question:
> On a tablet, would it be harder to maintain one file per project
> rather than a single large file?
>
> i.e. instead of updating projects.json one would need to create/update
> a projects/project.md file.

I have updated the attic-test tree with a YAML-based version:

https://svn.apache.org/repos/asf/attic/site-test/yaml

The yaml.sh script in the parent directory will create the output in docs3/.
This should currently be identical to the output in docs2/ - apart
from the date in feed.xml

If this looks like it will be suitable for use when working from
tablets, I can tidy it up and start migrating some of the additional
text from the xdocs/projects files

If not, please advise what needs to be done to make it possible to
update the site from a tablet.