Re: [xwiki-devs] XAR source projects should allow source files

[Paul Libbrecht]

Thank you for your comments Thomas and Vincent,
Thanks for merging my pull request.

Dear all,

Since 11.4-SNAPSHOT (it will 11.4-RC1 in I suppose), it is possible to 
use the transformation actions INSERT_TEXT and 
INSERT_ATTACHMENT_CONTENT.
This is documented in 
https://dev.xwiki.org/xwiki/bin/view/Community/XARPlugin#HInsertTextContentofaFile


paul

On 18 Apr 2019, at 14:39, Thomas Mortagne wrote:


Hi Paul, I commented the jira issue and the PR.

I don't see any harm in this approach at meast so it's a +1 from me.

On Tue, Apr 16, 2019 at 3:06 PM  wrote:



On 26 Aug 2016, at 13:50, Vincent Massol wrote:

[...]
* What I understand is that you’re not proposing to change the 
XAR
format but to introduce some tooling to generate a XAR from a set 
of

source files, at build time. Correct?

[...]

I am rewaking this rather older subject (see
http://xwiki.475771.n2.nabble.com/XAR-source-projects-should-allow-source-files-td7600774i20.html
and older things mentioned there) with yet another attempt which 
might
satisfy everyone: an attempt to use XAR-Transformations to include 
files

into XARs. It defines two extra transformations:

- INSERT_TEXT (using an XPath expression): inserts the content of the
text file
- INSERT_ATTACHMENT_CONTENT (using a file name): inserts the content 
of

a file in base64

The attempt is detailed in 
https://jira.xwiki.org/browse/XCOMMONS-1614

and has a pull-request and an example realisation.

I claim that this attempt allows project developers to nicely edit 
files
and let them be combined into a xar when it is the time. See the 
issue

for an example realisation.

I would love your comments.

Paul




--
Thomas Mortagne

Re: [xwiki-devs] XAR source projects should allow source files

[Thomas Mortagne]

Hi Paul, I commented the jira issue and the PR.

I don't see any harm in this approach at meast so it's a +1 from me.

On Tue, Apr 16, 2019 at 3:06 PM  wrote:
>
> >> On 26 Aug 2016, at 13:50, Vincent Massol wrote:
> [...]
> >> * What I understand is that you’re not proposing to change the XAR
> >> format but to introduce some tooling to generate a XAR from a set of
> >> source files, at build time. Correct?
> [...]
>
> I am rewaking this rather older subject (see
> http://xwiki.475771.n2.nabble.com/XAR-source-projects-should-allow-source-files-td7600774i20.html
> and older things mentioned there) with yet another attempt which might
> satisfy everyone: an attempt to use XAR-Transformations to include files
> into XARs. It defines two extra transformations:
>
> - INSERT_TEXT (using an XPath expression): inserts the content of the
> text file
> - INSERT_ATTACHMENT_CONTENT (using a file name): inserts the content of
> a file in base64
>
> The attempt is detailed in https://jira.xwiki.org/browse/XCOMMONS-1614
> and has a pull-request and an example realisation.
>
> I claim that this attempt allows project developers to nicely edit files
> and let them be combined into a xar when it is the time. See the issue
> for an example realisation.
>
> I would love your comments.
>
> Paul



-- 
Thomas Mortagne

Re: [xwiki-devs] XAR source projects should allow source files

[paul]

On 26 Aug 2016, at 13:50, Vincent Massol wrote:

[...]
* What I understand is that you’re not proposing to change the XAR 
format but to introduce some tooling to generate a XAR from a set of 
source files, at build time. Correct?

[...]

I am rewaking this rather older subject (see 
http://xwiki.475771.n2.nabble.com/XAR-source-projects-should-allow-source-files-td7600774i20.html 
and older things mentioned there) with yet another attempt which might 
satisfy everyone: an attempt to use XAR-Transformations to include files 
into XARs. It defines two extra transformations:


- INSERT_TEXT (using an XPath expression): inserts the content of the 
text file
- INSERT_ATTACHMENT_CONTENT (using a file name): inserts the content of 
a file in base64


The attempt is detailed in https://jira.xwiki.org/browse/XCOMMONS-1614 
and has a pull-request and an example realisation.


I claim that this attempt allows project developers to nicely edit files 
and let them be combined into a xar when it is the time. See the issue 
for an example realisation.


I would love your comments.

Paul

Re: [xwiki-devs] XAR source projects should allow source files

[Paul Libbrecht]

Vincent Massol wrote:
>> * Modify XWiki’s export to use CDATA instead of xml-encoding the contents. 
>> This allows to more easily and manually copy content into separate files.
>
> BTW on a related note, I was wondering if XInclude allows including non-XML 
> encoded content. I’ve found 
> https://msdn.microsoft.com/en-us/library/aa302291.aspx#xinc_topic4 which 
> seems to indicate it’s supported (with parse=text and by using the encoding 
> attribute too).
Sure it does and this is unit-tested! See the xincludeTTest.xml in here
 
http://direct.hoplahup.net/tmp/xwiki-commons-tool-xar-plugin/src/test/resources/org/xwiki/tool/xar/
it is tested to match xincludeTProof.xml.
However, it does not do base64 conversion... Hence I've introduced two
elements textInclude (does the same, just no namespace fuss) and
binaryInclude) includes the attachment converted to base64.

Paul
___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Vincent Massol]

> On 26 Aug 2016, at 18:51, Vincent Massol  wrote:
> 
>> 
>> On 26 Aug 2016, at 13:24, Paul Libbrecht  wrote:
>> 
>> Guillaume,
>> 
>> The risk of dead code is always present at a developer.
>> But the addition of inclusions seems to make this risk higher for you.
>> 
>> Obviously we develop on very different wikis.
>> Mine tends to be having several development traces, often parallel,
>> sometimes not, and I will only reinstall if I really need it.
>> When developing xars, you seem to be trusting that yours contain just a few.
>> 
>> Because mine is crowdy, I will not trust a XAR export and will only pick
>> the necessary files and/or elements form a xar export, if at all, or
>> undergo a deep cleaning (which is typical at start).
>> 
>> Because yours (I assume) is somewhat clean, you trust a xar export and
>> commit it the eyes shut.
>> Thus introducing an include that would be forgotten makes it probable
>> for you and not for me, which is what you state.
>> 
>> What I can offer is to add to my ready available code that the validate
>> Mojo to check that no file that is not included in in the source tree.
>> Useful?
> 
> Useful but not enough IMO. This looks like just a workaround and it still 
> means some manual action to do after you’ve export your XAR into the source 
> tree. You still need to manually convert the document content which is 
> XML-encoded as a non-xml encoded content in the separate file.
> 
> BTW on this topic, our XAR export should really use CDATA construct and stop 
> encoding the content. This is just a major PITA.
> 
> Another idea is to offer 2 new Mojos in the XAR plugin:
> 
> * mvn xar:deploy. This would push the sources into a running XWiki. The URL 
> to be used would be defined as maven properties (and so would be the import 
> options: overwrite, backup pack, etc).
> 
> * mvn xar:fetch. This would do the opposite and it would accept 2 modes. Mode 
> 1: it would fetch new versions of existing files in the source directory 
> structure by fetching them from a running XWiki. Mode 2: there could be some 
> maven property to define the name of a space it should fetch from and it 
> would fetch all content inside that space.
> 
> Now let’s come back to Paul’s proposal. The fetch operation could check the 
> current source file in the filesystem prior to the fetching an find all 
> xinclude and when it fetches a new version of that file (i.e. of the page), 
> it would replace the content with the xinclude it had found and copy the 
> content(s) in their separate file(s).
> 
> This would allow not loosing the xincludes. And it would automate the whole 
> process even more.
> 
> WDYT?
> 
> FTR I find Paul’s idea quite clever with XInclude. I don’t think it’s our 
> final solution but it’s a clever one that addresses one need: that of being 
> able to use desktop tools to edit various file types (css, js, plain text).
> 
> So to conclude and while waiting for more elaborate and final solutions, I’d 
> agree with Paul’s idea but I think we need:
> 
> * Modify XWiki’s export to use CDATA instead of xml-encoding the contents. 
> This allows to more easily and manually copy content into separate files.

BTW on a related note, I was wondering if XInclude allows including non-XML 
encoded content. I’ve found 
https://msdn.microsoft.com/en-us/library/aa302291.aspx#xinc_topic4 which seems 
to indicate it’s supported (with parse=text and by using the encoding attribute 
too).

Thanks
-Vincent

> * Implement the 2 mojos I mentioned above which should automate nicely our 
> process + avoid loosing xincludes.
> * Paul’s idea of generating a warning if there’s a file in the tree that 
> doesn’t have a corresponding xinclude.
> 
> WDYT?
> 
> Thanks
> -Vincent
> 
>> Offering a diff is tempting but it's a lot more complex I feel (e.g.
>> it's better if it's three way which developers rarely all have on a
>> single disk, one or two of them being on a versioning server).
>> 
>> Paul
>> 
>> 
>> Guillaume Delhumeau wrote:
>>> I would be very concerned if we have wiki pages with dead code in some
>>> repository.
>>> 
>>> I am very skeptical, because if we cannot rely on the standard exporter to
>>> generate the source correctly, who will use these source files? When you
>>> develop an application, do you write it in the XML page in your repository
>>> or directly in your wiki?
>>> 
>>> I am a bit conservative here and I don't want to block you, but I really
>>> wonder if the advantages you mention worth the pain of being sure we don't
>>> have dead code, and manually cutting/pasting some bit of the code from a
>>> file to an other everytime you commit a page (and I do it often).
>> 
>> ___
>> devs mailing list
>> devs@xwiki.org
>> http://lists.xwiki.org/mailman/listinfo/devs

___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Vincent Massol]

> On 26 Aug 2016, at 18:33, Vincent Massol  wrote:
> 
>> 
>> On 26 Aug 2016, at 13:50, Vincent Massol  wrote:
>> 
>> Hi Paul,
>> 
>> I’ve finally taken the time to read this discussion (a bit quickly for the 
>> last mail though, I admit ;)).
>> 
>> Let me give my POV:
>> 
>> * What I understand is that you’re not proposing to change the XAR format 
>> but to introduce some tooling to generate a XAR from a set of source files, 
>> at build time. Correct?
>> 
>> * IMO we should not touch the existing XAR plugin. This is current the 
>> standard way to generate XAR files and we need to  keep, if only for 
>> backward compatibility with existing sources.
>> 
>> * IMO you should develop your tool as a new maven plugin in xwiki-contrib 
>> and use it on some contrib projects to validate and show to others how it 
>> works. Of course you can depend on the existing XAR plugin if you need tor 
>> reuse some code (it’s even possible to introduce a new api module if there’s 
>> an important need to share common code). 
>> 
>> * FTR this is exactly what Jean Simard as done with the XFF tools 
>> (https://github.com/xwiki-contrib/api-xff). He’s done is separately to 
>> validate it and show it. We need to know where we stand on XFF since it’s 
>> currently the most advanced POC that we have on introducing a new format for 
>> storing wiki pages in the SCM. It also supports both directions: from source 
>> to XAR and from wiki pages to source. I’m planning to check what’s missing 
>> in that progress to be able to use it (not much IMO).
>> 
>> * If you were to develop your maven plugin, you would just compete with the 
>> XFF code. This is fine, although I’d have preferred that we spend our time 
>> finishing the XFF work. It’s your time so your choice though. In the future, 
>> we’ll be able to choose which solution we want to use between the 3 that 
>> will exist (current XAR plugin, XFF solution and your solution).
>> 
>> * FTR Fabio had also worked on this topic in the past, see 
>> http://design.xwiki.org/xwiki/bin/view/Design/DirectoryStructureforXWikiApplications.
>>  Caleb has also worked on something related: 
>> https://github.com/xwiki-contrib/xwiki-tools-node
> 
> More notes:
> 
> - Caleb’s solution is working and is being used for some projects. For 
> example:
> https://github.com/xwiki-contrib/realtime-wysiwyg/tree/master/src/xwiki
> 
> Cons of Caleb solutions are listed at 
> http://design.xwiki.org/xwiki/bin/view/Design/DirectoryStructureforXWikiApplications.
>  Main issue points IMO:
> * There is no automated way to create this structure from code in the Wiki 
> (TODO)
> • Still lacking a Maven job for building extensions (TODO). Requires nodejs 
> to build.
> 
> - There’s also work done by Fabio here that I had missed: 
> https://github.com/fmancinelli/xwikifs. Fabio has chosen YAML, see 
> https://github.com/fmancinelli/xwikifs/tree/master/xwikifs-maven-plugin/src/it/xar/src/main/resources/Main.WebHomefor
>  an example. However I don’t know what is its status. There seems to be a 
> Maven XAR mojo written to generate a XAR file.

FTR here’s what Paul was saying in 2013 concerning xwikifs:
http://markmail.org/message/az2mh65c6xuc47bp

:)

Thanks
-Vincent

> 
> So we have plenty of stuff that have been tried but none has been finished 
> and none are addressing the full needs. To summarize, so far I’ve found:
> 
> * Caleb’s nodejs tool: https://github.com/xwiki-contrib/xwiki-tools-node
> * Fabio’s xwikifs: https://github.com/fmancinelli/xwikifs
> * Jean’s XFF: https://github.com/xwiki-contrib/api-xff
> * Paul’s XInclude extension to the XAR plugin: 
> http://jira.xwiki.org/browse/XWIKI-13643
> 
> They all seem to be missing the ability to export wiki pages from a running 
> wiki into the source filesystem.
> 
> Thanks
> -Vincent
> 
>> So for me the important part is for you to develop your solution in a 
>> separate module in xwiki-contrib as a demonstator and possibly start using 
>> it on some contrib project to validate it. The alternative is to work on XFF 
>> and finish it.
>> 
>> Personally I’m not a big fan at all of modifying the existing XAR plugin for 
>> various reasons:
>> * We don’t know yet what solution we’ll choose in the future. At this point 
>> in time I even have a preference for XFF. So it wouldn’t be fair and nice to 
>> modify the existing XAR plugin with your solution and then have to remove 
>> the code if we don’ want it in the future.
>> * It will be much simpler for you to develop your own project in 
>> xwiki-contrib since you’ll have commit access (which you won’t have in 
>> xwiki-commons).
>> 
>> WDYT?
>> 
>> Thanks
>> -Vincent
>> 
>> 
>>> On 22 Aug 2016, at 15:59, Paul Libbrecht  wrote:
>>> 
>>> 
>>> Hello fellow developers,
>>> 
>>> many of you are developing XAR-packaged resources. The exchangeability
>>> of these archives is one of the precious aspects of XWiki, I believe.
>>> 
>>> However, if one see all the many GitHub-contributed xar packages, the
>>> source is 

Re: [xwiki-devs] XAR source projects should allow source files

[Vincent Massol]

> On 26 Aug 2016, at 18:51, Vincent Massol  wrote:
> 
>> 
>> On 26 Aug 2016, at 13:24, Paul Libbrecht  wrote:
>> 
>> Guillaume,
>> 
>> The risk of dead code is always present at a developer.
>> But the addition of inclusions seems to make this risk higher for you.
>> 
>> Obviously we develop on very different wikis.
>> Mine tends to be having several development traces, often parallel,
>> sometimes not, and I will only reinstall if I really need it.
>> When developing xars, you seem to be trusting that yours contain just a few.
>> 
>> Because mine is crowdy, I will not trust a XAR export and will only pick
>> the necessary files and/or elements form a xar export, if at all, or
>> undergo a deep cleaning (which is typical at start).
>> 
>> Because yours (I assume) is somewhat clean, you trust a xar export and
>> commit it the eyes shut.
>> Thus introducing an include that would be forgotten makes it probable
>> for you and not for me, which is what you state.
>> 
>> What I can offer is to add to my ready available code that the validate
>> Mojo to check that no file that is not included in in the source tree.
>> Useful?
> 
> Useful but not enough IMO. This looks like just a workaround and it still 
> means some manual action to do after you’ve export your XAR into the source 
> tree. You still need to manually convert the document content which is 
> XML-encoded as a non-xml encoded content in the separate file.
> 
> BTW on this topic, our XAR export should really use CDATA construct and stop 
> encoding the content. This is just a major PITA.
> 
> Another idea is to offer 2 new Mojos in the XAR plugin:
> 
> * mvn xar:deploy. This would push the sources into a running XWiki. The URL 
> to be used would be defined as maven properties (and so would be the import 
> options: overwrite, backup pack, etc).
> 
> * mvn xar:fetch. This would do the opposite and it would accept 2 modes. Mode 
> 1: it would fetch new versions of existing files in the source directory 
> structure by fetching them from a running XWiki. Mode 2: there could be some 
> maven property to define the name of a space it should fetch from and it 
> would fetch all content inside that space.
> 
> Now let’s come back to Paul’s proposal. The fetch operation could check the 
> current source file in the filesystem prior to the fetching an find all 
> xinclude and when it fetches a new version of that file (i.e. of the page), 
> it would replace the content with the xinclude it had found and copy the 
> content(s) in their separate file(s).
> 
> This would allow not loosing the xincludes. And it would automate the whole 
> process even more.
> 
> WDYT?
> 
> FTR I find Paul’s idea quite clever with XInclude. I don’t think it’s our 
> final solution but it’s a clever one that addresses one need: that of being 
> able to use desktop tools to edit various file types (css, js, plain text).
> 
> So to conclude and while waiting for more elaborate and final solutions, I’d 
> agree with Paul’s idea but I think we need:
> 
> * Modify XWiki’s export to use CDATA instead of xml-encoding the contents. 
> This allows to more easily and manually copy content into separate files.
> * Implement the 2 mojos I mentioned above which should automate nicely our 
> process + avoid loosing xincludes.
> * Paul’s idea of generating a warning if there’s a file in the tree that 
> doesn’t have a corresponding xinclude.
> 
> WDYT?

Another option is to modify xar:format to always extract content + textarea 
properties + attachments into separate files. And agree that’s we’re ok to 
switch to this representation.

Thanks
-Vincent

> Thanks
> -Vincent
> 
>> Offering a diff is tempting but it's a lot more complex I feel (e.g.
>> it's better if it's three way which developers rarely all have on a
>> single disk, one or two of them being on a versioning server).
>> 
>> Paul
>> 
>> 
>> Guillaume Delhumeau wrote:
>>> I would be very concerned if we have wiki pages with dead code in some
>>> repository.
>>> 
>>> I am very skeptical, because if we cannot rely on the standard exporter to
>>> generate the source correctly, who will use these source files? When you
>>> develop an application, do you write it in the XML page in your repository
>>> or directly in your wiki?
>>> 
>>> I am a bit conservative here and I don't want to block you, but I really
>>> wonder if the advantages you mention worth the pain of being sure we don't
>>> have dead code, and manually cutting/pasting some bit of the code from a
>>> file to an other everytime you commit a page (and I do it often).

___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Vincent Massol]

> On 26 Aug 2016, at 13:24, Paul Libbrecht  wrote:
> 
> Guillaume,
> 
> The risk of dead code is always present at a developer.
> But the addition of inclusions seems to make this risk higher for you.
> 
> Obviously we develop on very different wikis.
> Mine tends to be having several development traces, often parallel,
> sometimes not, and I will only reinstall if I really need it.
> When developing xars, you seem to be trusting that yours contain just a few.
> 
> Because mine is crowdy, I will not trust a XAR export and will only pick
> the necessary files and/or elements form a xar export, if at all, or
> undergo a deep cleaning (which is typical at start).
> 
> Because yours (I assume) is somewhat clean, you trust a xar export and
> commit it the eyes shut.
> Thus introducing an include that would be forgotten makes it probable
> for you and not for me, which is what you state.
> 
> What I can offer is to add to my ready available code that the validate
> Mojo to check that no file that is not included in in the source tree.
> Useful?

Useful but not enough IMO. This looks like just a workaround and it still means 
some manual action to do after you’ve export your XAR into the source tree. You 
still need to manually convert the document content which is XML-encoded as a 
non-xml encoded content in the separate file.

BTW on this topic, our XAR export should really use CDATA construct and stop 
encoding the content. This is just a major PITA.

Another idea is to offer 2 new Mojos in the XAR plugin:

* mvn xar:deploy. This would push the sources into a running XWiki. The URL to 
be used would be defined as maven properties (and so would be the import 
options: overwrite, backup pack, etc).

* mvn xar:fetch. This would do the opposite and it would accept 2 modes. Mode 
1: it would fetch new versions of existing files in the source directory 
structure by fetching them from a running XWiki. Mode 2: there could be some 
maven property to define the name of a space it should fetch from and it would 
fetch all content inside that space.

Now let’s come back to Paul’s proposal. The fetch operation could check the 
current source file in the filesystem prior to the fetching an find all 
xinclude and when it fetches a new version of that file (i.e. of the page), it 
would replace the content with the xinclude it had found and copy the 
content(s) in their separate file(s).

This would allow not loosing the xincludes. And it would automate the whole 
process even more.

WDYT?

FTR I find Paul’s idea quite clever with XInclude. I don’t think it’s our final 
solution but it’s a clever one that addresses one need: that of being able to 
use desktop tools to edit various file types (css, js, plain text).

So to conclude and while waiting for more elaborate and final solutions, I’d 
agree with Paul’s idea but I think we need:

* Modify XWiki’s export to use CDATA instead of xml-encoding the contents. This 
allows to more easily and manually copy content into separate files.
* Implement the 2 mojos I mentioned above which should automate nicely our 
process + avoid loosing xincludes.
* Paul’s idea of generating a warning if there’s a file in the tree that 
doesn’t have a corresponding xinclude.

WDYT?

Thanks
-Vincent

> Offering a diff is tempting but it's a lot more complex I feel (e.g.
> it's better if it's three way which developers rarely all have on a
> single disk, one or two of them being on a versioning server).
> 
> Paul
> 
> 
> Guillaume Delhumeau wrote:
>> I would be very concerned if we have wiki pages with dead code in some
>> repository.
>> 
>> I am very skeptical, because if we cannot rely on the standard exporter to
>> generate the source correctly, who will use these source files? When you
>> develop an application, do you write it in the XML page in your repository
>> or directly in your wiki?
>> 
>> I am a bit conservative here and I don't want to block you, but I really
>> wonder if the advantages you mention worth the pain of being sure we don't
>> have dead code, and manually cutting/pasting some bit of the code from a
>> file to an other everytime you commit a page (and I do it often).
> 
> ___
> devs mailing list
> devs@xwiki.org
> http://lists.xwiki.org/mailman/listinfo/devs

___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Vincent Massol]

> On 26 Aug 2016, at 13:50, Vincent Massol  wrote:
> 
> Hi Paul,
> 
> I’ve finally taken the time to read this discussion (a bit quickly for the 
> last mail though, I admit ;)).
> 
> Let me give my POV:
> 
> * What I understand is that you’re not proposing to change the XAR format but 
> to introduce some tooling to generate a XAR from a set of source files, at 
> build time. Correct?
> 
> * IMO we should not touch the existing XAR plugin. This is current the 
> standard way to generate XAR files and we need to  keep, if only for backward 
> compatibility with existing sources.
> 
> * IMO you should develop your tool as a new maven plugin in xwiki-contrib and 
> use it on some contrib projects to validate and show to others how it works. 
> Of course you can depend on the existing XAR plugin if you need tor reuse 
> some code (it’s even possible to introduce a new api module if there’s an 
> important need to share common code). 
> 
> * FTR this is exactly what Jean Simard as done with the XFF tools 
> (https://github.com/xwiki-contrib/api-xff). He’s done is separately to 
> validate it and show it. We need to know where we stand on XFF since it’s 
> currently the most advanced POC that we have on introducing a new format for 
> storing wiki pages in the SCM. It also supports both directions: from source 
> to XAR and from wiki pages to source. I’m planning to check what’s missing in 
> that progress to be able to use it (not much IMO).
> 
> * If you were to develop your maven plugin, you would just compete with the 
> XFF code. This is fine, although I’d have preferred that we spend our time 
> finishing the XFF work. It’s your time so your choice though. In the future, 
> we’ll be able to choose which solution we want to use between the 3 that will 
> exist (current XAR plugin, XFF solution and your solution).
> 
> * FTR Fabio had also worked on this topic in the past, see 
> http://design.xwiki.org/xwiki/bin/view/Design/DirectoryStructureforXWikiApplications.
>  Caleb has also worked on something related: 
> https://github.com/xwiki-contrib/xwiki-tools-node

More notes:

- Caleb’s solution is working and is being used for some projects. For example:
https://github.com/xwiki-contrib/realtime-wysiwyg/tree/master/src/xwiki

Cons of Caleb solutions are listed at 
http://design.xwiki.org/xwiki/bin/view/Design/DirectoryStructureforXWikiApplications.
 Main issue points IMO:
* There is no automated way to create this structure from code in the Wiki 
(TODO)
• Still lacking a Maven job for building extensions (TODO). Requires nodejs to 
build.

- There’s also work done by Fabio here that I had missed: 
https://github.com/fmancinelli/xwikifs. Fabio has chosen YAML, see 
https://github.com/fmancinelli/xwikifs/tree/master/xwikifs-maven-plugin/src/it/xar/src/main/resources/Main.WebHome
 for an example. However I don’t know what is its status. There seems to be a 
Maven XAR mojo written to generate a XAR file.

So we have plenty of stuff that have been tried but none has been finished and 
none are addressing the full needs. To summarize, so far I’ve found:

* Caleb’s nodejs tool: https://github.com/xwiki-contrib/xwiki-tools-node
* Fabio’s xwikifs: https://github.com/fmancinelli/xwikifs
* Jean’s XFF: https://github.com/xwiki-contrib/api-xff
* Paul’s XInclude extension to the XAR plugin: 
http://jira.xwiki.org/browse/XWIKI-13643

They all seem to be missing the ability to export wiki pages from a running 
wiki into the source filesystem.

Thanks
-Vincent

> So for me the important part is for you to develop your solution in a 
> separate module in xwiki-contrib as a demonstator and possibly start using it 
> on some contrib project to validate it. The alternative is to work on XFF and 
> finish it.
> 
> Personally I’m not a big fan at all of modifying the existing XAR plugin for 
> various reasons:
> * We don’t know yet what solution we’ll choose in the future. At this point 
> in time I even have a preference for XFF. So it wouldn’t be fair and nice to 
> modify the existing XAR plugin with your solution and then have to remove the 
> code if we don’ want it in the future.
> * It will be much simpler for you to develop your own project in 
> xwiki-contrib since you’ll have commit access (which you won’t have in 
> xwiki-commons).
> 
> WDYT?
> 
> Thanks
> -Vincent
> 
> 
>> On 22 Aug 2016, at 15:59, Paul Libbrecht  wrote:
>> 
>> 
>> Hello fellow developers,
>> 
>> many of you are developing XAR-packaged resources. The exchangeability
>> of these archives is one of the precious aspects of XWiki, I believe.
>> 
>> However, if one see all the many GitHub-contributed xar packages, the
>> source is not there: it's in a Wiki with which one would "open it". A
>> pity since much of the content is code.
>> 
>> I'd like to propose to allow XARs to be built form source code which is
>> true source that one edits.
>> Code files with their proper syntax coloring, editors, auto-completion,
>> search. Media-files that are edited wit

Re: [xwiki-devs] XAR source projects should allow source files

[Vincent Massol]

> On 26 Aug 2016, at 13:50, Vincent Massol  wrote:
> 
> Hi Paul,
> 
> I’ve finally taken the time to read this discussion (a bit quickly for the 
> last mail though, I admit ;)).
> 
> Let me give my POV:
> 
> * What I understand is that you’re not proposing to change the XAR format but 
> to introduce some tooling to generate a XAR from a set of source files, at 
> build time. Correct?
> 
> * IMO we should not touch the existing XAR plugin. This is current the 
> standard way to generate XAR files and we need to  keep, if only for backward 
> compatibility with existing sources.
> 
> * IMO you should develop your tool as a new maven plugin in xwiki-contrib and 
> use it on some contrib projects to validate and show to others how it works. 
> Of course you can depend on the existing XAR plugin if you need tor reuse 
> some code (it’s even possible to introduce a new api module if there’s an 
> important need to share common code). 
> 
> * FTR this is exactly what Jean Simard as done with the XFF tools 
> (https://github.com/xwiki-contrib/api-xff). He’s done is separately to 
> validate it and show it. We need to know where we stand on XFF since it’s 
> currently the most advanced POC that we have on introducing a new format for 
> storing wiki pages in the SCM. It also supports both directions: from source 
> to XAR and from wiki pages to source. I’m planning to check what’s missing in 
> that progress to be able to use it (not much IMO).

So I’ve posted a quick analysis of the XFF module here:
http://markmail.org/message/2pqcyfflxhcp3udh

In short:
* We should be able to import an XFF structure into a running XWiki
* It doesn’t seem that the round tripping is implemented and I don’t think we 
can export from a running XWiki into an XFF structure.
* The XFF format needs to be reviewed

Thanks
-Vincent

> * If you were to develop your maven plugin, you would just compete with the 
> XFF code. This is fine, although I’d have preferred that we spend our time 
> finishing the XFF work. It’s your time so your choice though. In the future, 
> we’ll be able to choose which solution we want to use between the 3 that will 
> exist (current XAR plugin, XFF solution and your solution).
> 
> * FTR Fabio had also worked on this topic in the past, see 
> http://design.xwiki.org/xwiki/bin/view/Design/DirectoryStructureforXWikiApplications.
>  Caleb has also worked on something related: 
> https://github.com/xwiki-contrib/xwiki-tools-node
> 
> So for me the important part is for you to develop your solution in a 
> separate module in xwiki-contrib as a demonstator and possibly start using it 
> on some contrib project to validate it. The alternative is to work on XFF and 
> finish it.
> 
> Personally I’m not a big fan at all of modifying the existing XAR plugin for 
> various reasons:
> * We don’t know yet what solution we’ll choose in the future. At this point 
> in time I even have a preference for XFF. So it wouldn’t be fair and nice to 
> modify the existing XAR plugin with your solution and then have to remove the 
> code if we don’ want it in the future.
> * It will be much simpler for you to develop your own project in 
> xwiki-contrib since you’ll have commit access (which you won’t have in 
> xwiki-commons).
> 
> WDYT?
> 
> Thanks
> -Vincent
> 
> 
>> On 22 Aug 2016, at 15:59, Paul Libbrecht  wrote:
>> 
>> 
>> Hello fellow developers,
>> 
>> many of you are developing XAR-packaged resources. The exchangeability
>> of these archives is one of the precious aspects of XWiki, I believe.
>> 
>> However, if one see all the many GitHub-contributed xar packages, the
>> source is not there: it's in a Wiki with which one would "open it". A
>> pity since much of the content is code.
>> 
>> I'd like to propose to allow XARs to be built form source code which is
>> true source that one edits.
>> Code files with their proper syntax coloring, editors, auto-completion,
>> search. Media-files that are edited with our tool of choice by a simple
>> double-click and not a complex pull, edit, and replace.
>> 
>> Thus, my proposal is to allow inclusions within the XML files.
>> Inclusions should be of type:
>> - text to include page content or object properties (e.g. css of cssx
>> objects)
>> - attachments to include attached objects
>> 
>> My current setup is documented here:http://jira.xwiki.org/browse/XWIKI-13643
>> It does binaryInclude to generate base64 and textInclude to generate
>> XML-text.
>> Thus, I can advertise a github project which is a macro and whose
>> complex function is in a velocity file that is in a .vm file. However, I
>> do not need to explode everything, e.g. the text-content describing the
>> macro or (in this case too small) translation set.
>> 
>> What do you think?
>> Should I rather generate complete attachment elements (name, length,
>> modif date, ...) instead of doing a binaryInclude that just generates
>> base64 ?
>> Is fetching files for inclusion beyond the boundaries of the project of
>> any use? (

Re: [xwiki-devs] XAR source projects should allow source files

[Vincent Massol]

> On 26 Aug 2016, at 17:14, Paul Libbrecht  wrote:
> 
> 
>>> Now we still wouldn’t be able to use it in the xwiki dev team because we 
>>> need a round-trip solution (we need to be able to convert a wiki page into 
>>> that structure). This is what Guillaume was mentioning.
 
 So we need to solve that first or it won’t have much of a future (and it 
 could be argued that there’s little rationale for putting this code in 
 commons if it’s not used).
>> Is the warning proposal made to Guillaume sufficient?
> And... which team do you mean?
> The product team? All employees? All developers? The client team?

None of those! :) (btw all of these have no meaning on this devs list…)

“
Now we still wouldn’t be able to use it in the xwiki dev team because we need a 
round-trip solution (we need to be able to convert a wiki page into that 
structure). This is what Guillaume was mentioning.

So we need to solve that first or it won’t have much of a future (and it could 
be argued that there’s little rationale for putting this code in commons if 
it’s not used).
“

team = "xwiki dev team “ == committers of the xwiki github organization == the 
committers of where the XAR plugin is located (in xwiki-commons in the xwiki 
github organization).

Thanks
-Vincent


> 
> Paul
> ___
> devs mailing list
> devs@xwiki.org
> http://lists.xwiki.org/mailman/listinfo/devs

___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Paul Libbrecht]

>> Now we still wouldn’t be able to use it in the xwiki dev team because we 
>> need a round-trip solution (we need to be able to convert a wiki page into 
>> that structure). This is what Guillaume was mentioning.
>> >
>> > So we need to solve that first or it won’t have much of a future (and it 
>> > could be argued that there’s little rationale for putting this code in 
>> > commons if it’s not used).
> Is the warning proposal made to Guillaume sufficient?
And... which team do you mean?
The product team? All employees? All developers? The client team?

Paul
___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Paul Libbrecht]

Vincent Massol wrote:
> Now we still wouldn’t be able to use it in the xwiki dev team because we need 
> a round-trip solution (we need to be able to convert a wiki page into that 
> structure). This is what Guillaume was mentioning.
>
> So we need to solve that first or it won’t have much of a future (and it 
> could be argued that there’s little rationale for putting this code in 
> commons if it’s not used).
Is the warning proposal made to Guillaume sufficient?

Paul
___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Vincent Massol]

> On 26 Aug 2016, at 16:04, Paul Libbrecht  wrote:
> 
> Vincent Massol wrote:
>> Can I test it somewhere so that I can fully understand your technical 
>> solution?
> http://extensions.xwiki.org/xwiki/bin/view/Extension/DownloadAttachments/
> is an example source with it.
> This can also be browsed at:
> 
> http://direct.hoplahup.net/tmp/DownloadAttachments/src/main/resources/Macros/
> where DownloadAttachments.xml includes DownloadAttachments_content.groovy.


Just checked 
http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter#HFormat and I 
was surprised to see that Jean has not defined a different file for the content 
and there’s still XML.

I remember seeing other proposals int he past where there was not a single line 
of XML (I think I even proposed some format on the list a long time ago based 
on json for the metadata since that’s much simpler to write).

It does separate CSS and JS and attachments but it should also do the same for 
content. We’d need to work on that.


In my vision of a nice filesystem format, you’d have a multiple level directory 
structure (to represent nested pages) and a lot of conventions (such as having 
a file name content.txt in each directory to represent that content and an 
optional meta.json file to represent the metadata: authors, language, etc) and 
some reserved directory names for xobjects, etc.

> I guess that the way to test it is to build and install the project I
> have (which would be 8.3-SNAPHOT):
> 
> http://direct.hoplahup.net/tmp/xwiki-commons-tool-xar-plugin-8.3-SNAPSHOT-polx.zip
> (which you'd obtain by applying the patch I've put in the jira).
>> What I fail to understand is why you don’t want to commit your code in
>> contrib. XWiki is a platform with extensions. You don’t need to be in
>> the core to be able to contribute an extension to XWiki (XAR plugin is
>> core).
> I'd like to avoid forking. This creates niches.
> My laptop is full of such attempts. Creating a github fork, a maven
> polxar or something like this is fairly equivalent.

Ok so if I understand correctly what you’re proposing is to keep the XML files 
and just use manual XInclude for separating out the content and any other part 
that a dev wants to separate (textarea properties, etc).

So it would be a convention to use xinclude or not and not something enforced. 
Meaning that it’s up to the developers to use an xinclude or not.

Indeed, for this part I wouldn't see a real issue in adding support for this in 
the existing XAR mojo since it’s very lightweight and is just an optional thing.

Now we still wouldn’t be able to use it in the xwiki dev team because we need a 
round-trip solution (we need to be able to convert a wiki page into that 
structure). This is what Guillaume was mentioning.

So we need to solve that first or it won’t have much of a future (and it could 
be argued that there’s little rationale for putting this code in commons if 
it’s not used).


If we introduce a nice filesystem format, it will be tempting to make 
modifications to it directly and commit. This introduces the risk of breaking 
things from what we do now. Right now we also ensure it works first because we 
make the modifications in the wiki - ie in our execution environment - and only 
after we export. One way to circumvent this will be to develop XAR unit testing 
or functional testing a lot more. Right now our process at least enforces a 
manual testing...


Thanks
-Vincent

> Paul

___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Paul Libbrecht]

Vincent Massol wrote:
> Can I test it somewhere so that I can fully understand your technical 
> solution?
http://extensions.xwiki.org/xwiki/bin/view/Extension/DownloadAttachments/
is an example source with it.
This can also be browsed at:
   
http://direct.hoplahup.net/tmp/DownloadAttachments/src/main/resources/Macros/
where DownloadAttachments.xml includes DownloadAttachments_content.groovy.

I guess that the way to test it is to build and install the project I
have (which would be 8.3-SNAPHOT):

http://direct.hoplahup.net/tmp/xwiki-commons-tool-xar-plugin-8.3-SNAPSHOT-polx.zip
(which you'd obtain by applying the patch I've put in the jira).
> What I fail to understand is why you don’t want to commit your code in
> contrib. XWiki is a platform with extensions. You don’t need to be in
> the core to be able to contribute an extension to XWiki (XAR plugin is
> core).
I'd like to avoid forking. This creates niches.
My laptop is full of such attempts. Creating a github fork, a maven
polxar or something like this is fairly equivalent.

Paul
___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Vincent Massol]

> On 26 Aug 2016, at 14:03, Paul Libbrecht  wrote:
> 
> Hello VIncent,
>> So for me the important part is for you to develop your solution in a 
>> separate module in xwiki-contrib as a demonstator and possibly start using 
>> it on some contrib project to validate it. The alternative is to work on XFF 
>> and finish it.
> Well. That brings the question back: how many are using it?
> Obviously, I see almost no-one.

ofc since it isn’t finished and has never been promoted and the xwiki product 
team has not even decided whether they’d like to use it or not for the apps in 
the platform repo...

> As long as:
> - it is not complete
> - it is not used by developers big time
> ... it will not be considered useful.
> So... requesting to "go away" is very close to what you're asking, I feel.

Too bad you’re seeing it this way but I was actually trying to help you, so 
that you could progress (not be blocked) and show something (to convince 
people).

Now if your point is that either your tool should be used by the xwiki dev team 
or you don’t want to spend time working on it, then it requires a lot more 
discussions and it's required that it solves the issue of moving from wiki 
pages to sources which you haven’t addressed (but which XFF has).

To repeat myself, ATM, with my hat of xwiki dev team member, I prefer the XFF 
solution because 1) it’s a complete solution and 2) it’s following the 
direction we want to take (Filters). We still need to assess how much work 
remain but I’m confident it’s not too much.

> In the contrary, I believe that changing the XAR plugin in a fully
> conservative way is the way to go if one wants incremental progress.

As soon as the code is in, it’s the work of the xwiki dev team to support it 
even though we may not even be using it. What’s the point?

Honestly I don’t understand why you cannot put your code in xwiki-contrib under 
a different maven plugin than the XAR one. Name it XAR2 if you want. As for 
using it in a project it’s a simple as declaring 3 lines so that cannot be the 
issue.

> Can it be that you did not notice, it is fully backwards compatible?

What I haven’t fully understood is how would the sources of wiki pages look 
like in the file system with your proposal. Do you have an example?

Can I test it somewhere so that I can fully understand your technical solution?

>> Personally I’m not a big fan at all of modifying the existing XAR plugin for 
>> various reasons:
>> * We don’t know yet what solution we’ll choose in the future. At this point 
>> in time I even have a preference for XFF. So it wouldn’t be fair and nice to 
>> modify the existing XAR plugin with your solution and then have to remove 
>> the code if we don’ want it in the future.
> But obviously the time has not been invested about it.
> All issues are closed for XFF.

So let’s do it! :)

It hasn’t been invested more because the xwiki dev team doesn’t have such a 
pressing need compared to other stuff, that’s all. Now I’ve personally been 
wanting to review it, make it work and make a proposal on the list about using 
it for the platform github repo.

Now there are still problems probably. For example even though it supports 
exporting, it’s not convenient unless it’s bundled by default (since otherwise 
you’d need to install the extension everytime when you’re in dev mode, which is 
a pain). So we’d need to agree we want to bundle it by default for example.

>> * It will be much simpler for you to develop your own project in 
>> xwiki-contrib since you’ll have commit access (which you won’t have in 
>> xwiki-commons).
> That's a moot criterion.
> My code is really not a big change and also not very high risk.

What I fail to understand is why you don’t want to commit your code in contrib. 
XWiki is a platform with extensions. You don’t need to be in the core to be 
able to contribute an extension to XWiki (XAR plugin is core).

Thanks
-Vincent

> Paul
___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Paul Libbrecht]

Hello VIncent,
> So for me the important part is for you to develop your solution in a 
> separate module in xwiki-contrib as a demonstator and possibly start using it 
> on some contrib project to validate it. The alternative is to work on XFF and 
> finish it.
Well. That brings the question back: how many are using it?
Obviously, I see almost no-one.
As long as:
- it is not complete
- it is not used by developers big time
... it will not be considered useful.
So... requesting to "go away" is very close to what you're asking, I feel.

In the contrary, I believe that changing the XAR plugin in a fully
conservative way is the way to go if one wants incremental progress.

Can it be that you did not notice, it is fully backwards compatible?
> Personally I’m not a big fan at all of modifying the existing XAR plugin for 
> various reasons:
> * We don’t know yet what solution we’ll choose in the future. At this point 
> in time I even have a preference for XFF. So it wouldn’t be fair and nice to 
> modify the existing XAR plugin with your solution and then have to remove the 
> code if we don’ want it in the future.
But obviously the time has not been invested about it.
All issues are closed for XFF.
> * It will be much simpler for you to develop your own project in 
> xwiki-contrib since you’ll have commit access (which you won’t have in 
> xwiki-commons).
That's a moot criterion.
My code is really not a big change and also not very high risk.

Paul
___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Vincent Massol]

Hi Paul,

I’ve finally taken the time to read this discussion (a bit quickly for the last 
mail though, I admit ;)).

Let me give my POV:

* What I understand is that you’re not proposing to change the XAR format but 
to introduce some tooling to generate a XAR from a set of source files, at 
build time. Correct?

* IMO we should not touch the existing XAR plugin. This is current the standard 
way to generate XAR files and we need to  keep, if only for backward 
compatibility with existing sources.

* IMO you should develop your tool as a new maven plugin in xwiki-contrib and 
use it on some contrib projects to validate and show to others how it works. Of 
course you can depend on the existing XAR plugin if you need tor reuse some 
code (it’s even possible to introduce a new api module if there’s an important 
need to share common code). 

* FTR this is exactly what Jean Simard as done with the XFF tools 
(https://github.com/xwiki-contrib/api-xff). He’s done is separately to validate 
it and show it. We need to know where we stand on XFF since it’s currently the 
most advanced POC that we have on introducing a new format for storing wiki 
pages in the SCM. It also supports both directions: from source to XAR and from 
wiki pages to source. I’m planning to check what’s missing in that progress to 
be able to use it (not much IMO).

* If you were to develop your maven plugin, you would just compete with the XFF 
code. This is fine, although I’d have preferred that we spend our time 
finishing the XFF work. It’s your time so your choice though. In the future, 
we’ll be able to choose which solution we want to use between the 3 that will 
exist (current XAR plugin, XFF solution and your solution).

* FTR Fabio had also worked on this topic in the past, see 
http://design.xwiki.org/xwiki/bin/view/Design/DirectoryStructureforXWikiApplications.
 Caleb has also worked on something related: 
https://github.com/xwiki-contrib/xwiki-tools-node

So for me the important part is for you to develop your solution in a separate 
module in xwiki-contrib as a demonstator and possibly start using it on some 
contrib project to validate it. The alternative is to work on XFF and finish it.

Personally I’m not a big fan at all of modifying the existing XAR plugin for 
various reasons:
* We don’t know yet what solution we’ll choose in the future. At this point in 
time I even have a preference for XFF. So it wouldn’t be fair and nice to 
modify the existing XAR plugin with your solution and then have to remove the 
code if we don’ want it in the future.
* It will be much simpler for you to develop your own project in xwiki-contrib 
since you’ll have commit access (which you won’t have in xwiki-commons).

WDYT?

Thanks
-Vincent


> On 22 Aug 2016, at 15:59, Paul Libbrecht  wrote:
> 
> 
> Hello fellow developers,
> 
> many of you are developing XAR-packaged resources. The exchangeability
> of these archives is one of the precious aspects of XWiki, I believe.
> 
> However, if one see all the many GitHub-contributed xar packages, the
> source is not there: it's in a Wiki with which one would "open it". A
> pity since much of the content is code.
> 
> I'd like to propose to allow XARs to be built form source code which is
> true source that one edits.
> Code files with their proper syntax coloring, editors, auto-completion,
> search. Media-files that are edited with our tool of choice by a simple
> double-click and not a complex pull, edit, and replace.
> 
> Thus, my proposal is to allow inclusions within the XML files.
> Inclusions should be of type:
> - text to include page content or object properties (e.g. css of cssx
> objects)
> - attachments to include attached objects
> 
> My current setup is documented here:http://jira.xwiki.org/browse/XWIKI-13643
> It does binaryInclude to generate base64 and textInclude to generate
> XML-text.
> Thus, I can advertise a github project which is a macro and whose
> complex function is in a velocity file that is in a .vm file. However, I
> do not need to explode everything, e.g. the text-content describing the
> macro or (in this case too small) translation set.
> 
> What do you think?
> Should I rather generate complete attachment elements (name, length,
> modif date, ...) instead of doing a binaryInclude that just generates
> base64 ?
> Is fetching files for inclusion beyond the boundaries of the project of
> any use? (it is currently prohibited for safety)
> 
> thanks in advance.
> 
> Paul
___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Paul Libbrecht]

Guillaume,

The risk of dead code is always present at a developer.
But the addition of inclusions seems to make this risk higher for you.

Obviously we develop on very different wikis.
Mine tends to be having several development traces, often parallel,
sometimes not, and I will only reinstall if I really need it.
When developing xars, you seem to be trusting that yours contain just a few.

Because mine is crowdy, I will not trust a XAR export and will only pick
the necessary files and/or elements form a xar export, if at all, or
undergo a deep cleaning (which is typical at start).

Because yours (I assume) is somewhat clean, you trust a xar export and
commit it the eyes shut.
Thus introducing an include that would be forgotten makes it probable
for you and not for me, which is what you state.

What I can offer is to add to my ready available code that the validate
Mojo to check that no file that is not included in in the source tree.
Useful?
Offering a diff is tempting but it's a lot more complex I feel (e.g.
it's better if it's three way which developers rarely all have on a
single disk, one or two of them being on a versioning server).

Paul


Guillaume Delhumeau wrote:
> I would be very concerned if we have wiki pages with dead code in some
> repository.
>
> I am very skeptical, because if we cannot rely on the standard exporter to
> generate the source correctly, who will use these source files? When you
> develop an application, do you write it in the XML page in your repository
> or directly in your wiki?
>
> I am a bit conservative here and I don't want to block you, but I really
> wonder if the advantages you mention worth the pain of being sure we don't
> have dead code, and manually cutting/pasting some bit of the code from a
> file to an other everytime you commit a page (and I do it often).

___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Guillaume Delhumeau]

2016-08-24 20:31 GMT+02:00 Paul Libbrecht :

> Guillaume,
>
> sure, this is *doable* but it can also be done at discretion.
> E.g. you'd only copy from the exploded xar the other files you copied
> but not the source.
> Also, if you replace the XML files with those of the XARs, you'll just
> ignore the sources, since you've removed the includes.
>

I would be very concerned if we have wiki pages with dead code in some
repository.

I am very skeptical, because if we cannot rely on the standard exporter to
generate the source correctly, who will use these source files? When you
develop an application, do you write it in the XML page in your repository
or directly in your wiki?

I am a bit conservative here and I don't want to block you, but I really
wonder if the advantages you mention worth the pain of being sure we don't
have dead code, and manually cutting/pasting some bit of the code from a
file to an other everytime you commit a page (and I do it often).

Thanks,
Guillaume



>
> So, I really think that this is a progress.
> I agree an XFF to XAR converter would be more interesting but my patch
> is ready to be applied. A XAR generator is not yet.
>
> paul
>
> > Guillaume Delhumeau 
> > 23 August 2016 at 13:19
> > Of course, we could still change our workflow and copy/paste source
> > contents manually. Not sure it's gonna be more effective. So I prefer the
> > XFF approach (except that I would love having a Nested Document
> > terminology
> > instead of the current Space/Document one).
> >
> > 2016-08-23 13:11 GMT+02:00 Guillaume Delhumeau <
> >
> >
> >
> > Guillaume Delhumeau 
> > 23 August 2016 at 13:11
> > 2016-08-23 12:36 GMT+02:00 Paul Libbrecht :
> >
> >> Thomas Mortagne wrote:
> >>> It takes the following IMO:
> >>> * it works
> >>> * it does not break any retro compatibility
> >> both are claimed.
> >> There are unit tests.
> >> The only danger of breaking anything is if dom4j does not faithfully
> >> restore the XML source. I believe this can be ignored just as it has
> >> been ignored by the implementation of transformations.
> >>> (probably not enabled by default for example)
> >> No, it is enabled by default.
> >> I could replace the dom4j output by a file copy if no relevant elements
> >> are found.
> >>
> >> Note however, that it operates on elements that were never allowed
> before.
> >>> Now Guillaume has a point. The issue you will quickly have as soon as
> >>> you start using this is that you HAVE to modify your XAR trough
> >>> filesystem because you can't edit it in XWiki and export it anymore.
> >>> That is if you don't provide any tool to export this kind of XAR
> >>> extension.
> >> I believe this is best practice to insure that the wiki does not insert
> >> you surprise elements such as the author name or the edit comment.
> >
> >
> > In practice, this is the only manual step. I do a diff of the
> modifications
> > I have made and I revert all changes that are meaningless. But it's
> quick &
> > easy using my IDE (IntelliJ). I know some developers here do the same.
> The
> > only issue is when the XAR format have changed: some fields are not
> ordered
> > in the same way, and it makes harder to compute the real diff.
> >
> >
> >
> >> Is
> >> this practice not corresponding at all what others are doing?
> >>
> >> Using an XML-diff might be answering Guillaume's point... That's not
> >> completely trivial.
> >>
> >> Paul
> >> ___
> >> devs mailing list
> >> devs@xwiki.org
> >> http://lists.xwiki.org/mailman/listinfo/devs
> >>
> >
> >
> >
> > Paul Libbrecht 
> > 23 August 2016 at 12:36
> > Thomas Mortagne wrote:
> >> It takes the following IMO:
> >> * it works
> >> * it does not break any retro compatibility
> > both are claimed.
> > There are unit tests.
> > The only danger of breaking anything is if dom4j does not faithfully
> > restore the XML source. I believe this can be ignored just as it has
> > been ignored by the implementation of transformations.
> >> (probably not enabled by default for example)
> > No, it is enabled by default.
> > I could replace the dom4j output by a file copy if no relevant elements
> > are found.
> >
> > Note however, that it operates on elements that were never allowed
> before.
> >> Now Guillaume has a point. The issue you will quickly have as soon as
> >> you start using this is that you HAVE to modify your XAR trough
> >> filesystem because you can't edit it in XWiki and export it anymore.
> >> That is if you don't provide any tool to export this kind of XAR
> >> extension.
> > I believe this is best practice to insure that the wiki does not insert
> > you surprise elements such as the author name or the edit comment. Is
> > this practice not corresponding at all what others are doing?
> >
> > Using an XML-diff might be answering Guillaume's point... That's not
> > completely trivial.
> >
> > Paul
> > 

Re: [xwiki-devs] XAR source projects should allow source files

[Paul Libbrecht]

Guillaume,

sure, this is *doable* but it can also be done at discretion.
E.g. you'd only copy from the exploded xar the other files you copied
but not the source.
Also, if you replace the XML files with those of the XARs, you'll just
ignore the sources, since you've removed the includes.

So, I really think that this is a progress.
I agree an XFF to XAR converter would be more interesting but my patch
is ready to be applied. A XAR generator is not yet.

paul

> Guillaume Delhumeau 
> 23 August 2016 at 13:19
> Of course, we could still change our workflow and copy/paste source
> contents manually. Not sure it's gonna be more effective. So I prefer the
> XFF approach (except that I would love having a Nested Document
> terminology
> instead of the current Space/Document one).
>
> 2016-08-23 13:11 GMT+02:00 Guillaume Delhumeau <
>
>
>
> Guillaume Delhumeau 
> 23 August 2016 at 13:11
> 2016-08-23 12:36 GMT+02:00 Paul Libbrecht :
>
>> Thomas Mortagne wrote:
>>> It takes the following IMO:
>>> * it works
>>> * it does not break any retro compatibility
>> both are claimed.
>> There are unit tests.
>> The only danger of breaking anything is if dom4j does not faithfully
>> restore the XML source. I believe this can be ignored just as it has
>> been ignored by the implementation of transformations.
>>> (probably not enabled by default for example)
>> No, it is enabled by default.
>> I could replace the dom4j output by a file copy if no relevant elements
>> are found.
>>
>> Note however, that it operates on elements that were never allowed before.
>>> Now Guillaume has a point. The issue you will quickly have as soon as
>>> you start using this is that you HAVE to modify your XAR trough
>>> filesystem because you can't edit it in XWiki and export it anymore.
>>> That is if you don't provide any tool to export this kind of XAR
>>> extension.
>> I believe this is best practice to insure that the wiki does not insert
>> you surprise elements such as the author name or the edit comment.
>
>
> In practice, this is the only manual step. I do a diff of the modifications
> I have made and I revert all changes that are meaningless. But it's quick &
> easy using my IDE (IntelliJ). I know some developers here do the same. The
> only issue is when the XAR format have changed: some fields are not ordered
> in the same way, and it makes harder to compute the real diff.
>
>
>
>> Is
>> this practice not corresponding at all what others are doing?
>>
>> Using an XML-diff might be answering Guillaume's point... That's not
>> completely trivial.
>>
>> Paul
>> ___
>> devs mailing list
>> devs@xwiki.org
>> http://lists.xwiki.org/mailman/listinfo/devs
>>
>
>
>
> Paul Libbrecht 
> 23 August 2016 at 12:36
> Thomas Mortagne wrote:
>> It takes the following IMO:
>> * it works
>> * it does not break any retro compatibility 
> both are claimed.
> There are unit tests.
> The only danger of breaking anything is if dom4j does not faithfully
> restore the XML source. I believe this can be ignored just as it has
> been ignored by the implementation of transformations.
>> (probably not enabled by default for example)
> No, it is enabled by default.
> I could replace the dom4j output by a file copy if no relevant elements
> are found.
>
> Note however, that it operates on elements that were never allowed before.
>> Now Guillaume has a point. The issue you will quickly have as soon as
>> you start using this is that you HAVE to modify your XAR trough
>> filesystem because you can't edit it in XWiki and export it anymore.
>> That is if you don't provide any tool to export this kind of XAR
>> extension.
> I believe this is best practice to insure that the wiki does not insert
> you surprise elements such as the author name or the edit comment. Is
> this practice not corresponding at all what others are doing?
>
> Using an XML-diff might be answering Guillaume's point... That's not
> completely trivial.
>
> Paul
> ___
> devs mailing list
> devs@xwiki.org
> http://lists.xwiki.org/mailman/listinfo/devs
> Thomas Mortagne 
> 23 August 2016 at 12:28
> On Tue, Aug 23, 2016 at 12:26 PM, Thomas Mortagne
>  wrote:
>> On Tue, Aug 23, 2016 at 12:17 PM, Paul Libbrecht  wrote:
> What I obtain after mvn package is a file my-application.xff.
>> I can't seem to import that. Should I be able to?
 Not trough XAR tools no, it's completely different format.
>>> So what needs to be done here is a converter which is a lot bigger than
>>> a packager.
>> Which is what I indicated: you just use XFF filter (which already
>> exist) in input and XAR filter (which also already exist) as output.
>>
>>> We'll see what Jean says.
>>>
>>>
>>>
>>> In the meantime, do I understand at least that an enrichment to the
>>> xar-mojo is a more conservative approa

Re: [xwiki-devs] XAR source projects should allow source files

[Guillaume Delhumeau]

Of course, we could still change our workflow and copy/paste source
contents manually. Not sure it's gonna be more effective. So I prefer the
XFF approach (except that I would love having a Nested Document terminology
instead of the current Space/Document one).

2016-08-23 13:11 GMT+02:00 Guillaume Delhumeau <
guillaume.delhum...@xwiki.com>:

> 2016-08-23 12:36 GMT+02:00 Paul Libbrecht :
>
>>
>>
>> Thomas Mortagne wrote:
>> > It takes the following IMO:
>> > * it works
>> > * it does not break any retro compatibility
>> both are claimed.
>> There are unit tests.
>> The only danger of breaking anything is if dom4j does not faithfully
>> restore the XML source. I believe this can be ignored just as it has
>> been ignored by the implementation of transformations.
>> > (probably not enabled by default for example)
>> No, it is enabled by default.
>> I could replace the dom4j output by a file copy if no relevant elements
>> are found.
>>
>> Note however, that it operates on elements that were never allowed before.
>> > Now Guillaume has a point. The issue you will quickly have as soon as
>> > you start using this is that you HAVE to modify your XAR trough
>> > filesystem because you can't edit it in XWiki and export it anymore.
>> > That is if you don't provide any tool to export this kind of XAR
>> > extension.
>> I believe this is best practice to insure that the wiki does not insert
>> you surprise elements such as the author name or the edit comment.
>
>
> In practice, this is the only manual step. I do a diff of the
> modifications I have made and I revert all changes that are meaningless.
> But it's quick & easy using my IDE (IntelliJ). I know some developers here
> do the same. The only issue is when the XAR format have changed: some
> fields are not ordered in the same way, and it makes harder to compute the
> real diff.
>
>
>
>> Is
>> this practice not corresponding at all what others are doing?
>>
>> Using an XML-diff might be answering Guillaume's point... That's not
>> completely trivial.
>>
>> Paul
>> ___
>> devs mailing list
>> devs@xwiki.org
>> http://lists.xwiki.org/mailman/listinfo/devs
>>
>
>
>
> --
> Guillaume Delhumeau (guillaume.delhum...@xwiki.com)
> Research & Development Engineer at XWiki SAS
> Committer on the XWiki.org project
>



-- 
Guillaume Delhumeau (guillaume.delhum...@xwiki.com)
Research & Development Engineer at XWiki SAS
Committer on the XWiki.org project
___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Guillaume Delhumeau]

2016-08-23 12:36 GMT+02:00 Paul Libbrecht :

>
>
> Thomas Mortagne wrote:
> > It takes the following IMO:
> > * it works
> > * it does not break any retro compatibility
> both are claimed.
> There are unit tests.
> The only danger of breaking anything is if dom4j does not faithfully
> restore the XML source. I believe this can be ignored just as it has
> been ignored by the implementation of transformations.
> > (probably not enabled by default for example)
> No, it is enabled by default.
> I could replace the dom4j output by a file copy if no relevant elements
> are found.
>
> Note however, that it operates on elements that were never allowed before.
> > Now Guillaume has a point. The issue you will quickly have as soon as
> > you start using this is that you HAVE to modify your XAR trough
> > filesystem because you can't edit it in XWiki and export it anymore.
> > That is if you don't provide any tool to export this kind of XAR
> > extension.
> I believe this is best practice to insure that the wiki does not insert
> you surprise elements such as the author name or the edit comment.


In practice, this is the only manual step. I do a diff of the modifications
I have made and I revert all changes that are meaningless. But it's quick &
easy using my IDE (IntelliJ). I know some developers here do the same. The
only issue is when the XAR format have changed: some fields are not ordered
in the same way, and it makes harder to compute the real diff.



> Is
> this practice not corresponding at all what others are doing?
>
> Using an XML-diff might be answering Guillaume's point... That's not
> completely trivial.
>
> Paul
> ___
> devs mailing list
> devs@xwiki.org
> http://lists.xwiki.org/mailman/listinfo/devs
>



-- 
Guillaume Delhumeau (guillaume.delhum...@xwiki.com)
Research & Development Engineer at XWiki SAS
Committer on the XWiki.org project
___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Paul Libbrecht]

Thomas Mortagne wrote:
> It takes the following IMO:
> * it works
> * it does not break any retro compatibility 
both are claimed.
There are unit tests.
The only danger of breaking anything is if dom4j does not faithfully
restore the XML source. I believe this can be ignored just as it has
been ignored by the implementation of transformations.
> (probably not enabled by default for example)
No, it is enabled by default.
I could replace the dom4j output by a file copy if no relevant elements
are found.

Note however, that it operates on elements that were never allowed before.
> Now Guillaume has a point. The issue you will quickly have as soon as
> you start using this is that you HAVE to modify your XAR trough
> filesystem because you can't edit it in XWiki and export it anymore.
> That is if you don't provide any tool to export this kind of XAR
> extension.
I believe this is best practice to insure that the wiki does not insert
you surprise elements such as the author name or the edit comment. Is
this practice not corresponding at all what others are doing?

Using an XML-diff might be answering Guillaume's point... That's not
completely trivial.

Paul
___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Thomas Mortagne]

On Tue, Aug 23, 2016 at 12:26 PM, Thomas Mortagne
 wrote:
> On Tue, Aug 23, 2016 at 12:17 PM, Paul Libbrecht  wrote:
>>
 What I obtain after mvn package is a file my-application.xff.
 > I can't seem to import that. Should I be able to?
>>> Not trough XAR tools no, it's completely different format.
>> So what needs to be done here is a converter which is a lot bigger than
>> a packager.
>
> Which is what I indicated: you just use XFF filter (which already
> exist) in input and XAR filter (which also already exist) as output.
>
>> We'll see what Jean says.
>>
>>
>>
>> In the meantime, do I understand at least that an enrichment to the
>> xar-mojo is a more conservative approach that might have a better chance
>> of uptake?

>> What does it take for me to get the patch accepted?

It takes the following IMO:
* it works
* it does not break any retro compatibility (probably not enabled by
default for example)

Now Guillaume has a point. The issue you will quickly have as soon as
you start using this is that you HAVE to modify your XAR trough
filesystem because you can't edit it in XWiki and export it anymore.
That is if you don't provide any tool to export this kind of XAR
extension.

>>
>> thanks
>>
>> Paul
>> ___
>> devs mailing list
>> devs@xwiki.org
>> http://lists.xwiki.org/mailman/listinfo/devs
>
>
>
> --
> Thomas Mortagne



-- 
Thomas Mortagne
___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Thomas Mortagne]

On Tue, Aug 23, 2016 at 12:17 PM, Paul Libbrecht  wrote:
>
>>> What I obtain after mvn package is a file my-application.xff.
>>> > I can't seem to import that. Should I be able to?
>> Not trough XAR tools no, it's completely different format.
> So what needs to be done here is a converter which is a lot bigger than
> a packager.

Which is what I indicated: you just use XFF filter (which already
exist) in input and XAR filter (which also already exist) as output.

> We'll see what Jean says.
>
>
>
> In the meantime, do I understand at least that an enrichment to the
> xar-mojo is a more conservative approach that might have a better chance
> of uptake?
> What does it take for me to get the patch accepted?
>
> thanks
>
> Paul
> ___
> devs mailing list
> devs@xwiki.org
> http://lists.xwiki.org/mailman/listinfo/devs



-- 
Thomas Mortagne
___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Paul Libbrecht]

>> What I obtain after mvn package is a file my-application.xff.
>> > I can't seem to import that. Should I be able to?
> Not trough XAR tools no, it's completely different format.
So what needs to be done here is a converter which is a lot bigger than
a packager.
We'll see what Jean says.



In the meantime, do I understand at least that an enrichment to the
xar-mojo is a more conservative approach that might have a better chance
of uptake?
What does it take for me to get the patch accepted?

thanks

Paul
___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Thomas Mortagne]

On Tue, Aug 23, 2016 at 11:46 AM, Paul Libbrecht  wrote:
> I've just tried it.
>http://direct.hoplahup.net/tmp/xffAttempt-first.zip
> After having checked-out https://github.com/xwiki-contrib/api-xff and
> run mvn -Denforcer.skip=true install.
> What I obtain after mvn package is a file my-application.xff.
> I can't seem to import that. Should I be able to?

Not trough XAR tools no, it's completely different format.

>
> thanks
> Paul
>
>> Vincent Massol 
>> 23 August 2016 at 11:35
>>> On 23 Aug 2016, at 11:15, Vincent Massol  wrote:
>>>
>>>
 On 23 Aug 2016, at 11:13, Paul Libbrecht  wrote:

 Let's ask things differently: what's missing XFF for others to use?
>>> I think it’s just missing a maven plugin to use it. All the other parts are 
>>> ready.
>>
>> Actually just noticed that Jean has started one:
>> https://github.com/xwiki-contrib/api-xff/tree/master/xff-maven
>>
>> I don’t know its status though.
>>
>> So what needs to be done is try the full chain and see what works/what 
>> doesn’t.
>>
>> Unless jean could tell us what’s the status? (I’ll ping him on skype).
>>
>> Thanks
>> -Vincent
>>
>>> Thanks
>>> -Vincent
>>>
 paul
> Vincent Massol 
> 23 August 2016 at 11:01
> The format seems to be defined here:
> http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter#HFormat
>>
>> ___
>> devs mailing list
>> devs@xwiki.org
>> http://lists.xwiki.org/mailman/listinfo/devs
>> Vincent Massol 
>> 23 August 2016 at 11:15
>>> On 23 Aug 2016, at 11:13, Paul Libbrecht  wrote:
>>>
>>> Let's ask things differently: what's missing XFF for others to use?
>>
>> I think it’s just missing a maven plugin to use it. All the other parts are 
>> ready.
>>
>> Thanks
>> -Vincent
>>
>>> paul
 Vincent Massol 
 23 August 2016 at 11:01
 The format seems to be defined here:
 http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter#HFormat
>>
>> ___
>> devs mailing list
>> devs@xwiki.org
>> http://lists.xwiki.org/mailman/listinfo/devs
>> Paul Libbrecht 
>> 23 August 2016 at 11:13
>> Let's ask things differently: what's missing XFF for others to use?
>>
>> paul
>>
>> ___
>> devs mailing list
>> devs@xwiki.org
>> http://lists.xwiki.org/mailman/listinfo/devs
>> Vincent Massol 
>> 23 August 2016 at 11:01
>> The format seems to be defined here:
>> http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter#HFormat
>>
>> Thanks
>> -Vincent
>>
>> ___
>> devs mailing list
>> devs@xwiki.org
>> http://lists.xwiki.org/mailman/listinfo/devs
>> Thomas Mortagne 
>> 23 August 2016 at 11:00
>> I can't give you much details on the XFF format, I just know the
>> general goal. I only helped Jean on the "how to use Filter API" side.
>>
>>
>>
>
> ___
> devs mailing list
> devs@xwiki.org
> http://lists.xwiki.org/mailman/listinfo/devs



-- 
Thomas Mortagne
___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Thomas Mortagne]

On Tue, Aug 23, 2016 at 11:35 AM, Vincent Massol  wrote:
>
>> On 23 Aug 2016, at 11:15, Vincent Massol  wrote:
>>
>>
>>> On 23 Aug 2016, at 11:13, Paul Libbrecht  wrote:
>>>
>>> Let's ask things differently: what's missing XFF for others to use?
>>
>> I think it’s just missing a maven plugin to use it. All the other parts are 
>> ready.
>
> Actually just noticed that Jean has started one:
> https://github.com/xwiki-contrib/api-xff/tree/master/xff-maven
>
> I don’t know its status though.
>
> So what needs to be done is try the full chain and see what works/what 
> doesn’t.
>
> Unless jean could tell us what’s the status? (I’ll ping him on skype).

This plugin has nothing to do with XAR AFAIK. It just zip a XFF package.

>
> Thanks
> -Vincent
>
>> Thanks
>> -Vincent
>>
>>> paul
 Vincent Massol 
 23 August 2016 at 11:01
 The format seems to be defined here:
 http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter#HFormat
>>
>
> ___
> devs mailing list
> devs@xwiki.org
> http://lists.xwiki.org/mailman/listinfo/devs



-- 
Thomas Mortagne
___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Paul Libbrecht]

I've just tried it.
   http://direct.hoplahup.net/tmp/xffAttempt-first.zip
After having checked-out https://github.com/xwiki-contrib/api-xff and
run mvn -Denforcer.skip=true install.
What I obtain after mvn package is a file my-application.xff.
I can't seem to import that. Should I be able to?

thanks
Paul

> Vincent Massol 
> 23 August 2016 at 11:35
>> On 23 Aug 2016, at 11:15, Vincent Massol  wrote:
>>
>>
>>> On 23 Aug 2016, at 11:13, Paul Libbrecht  wrote:
>>>
>>> Let's ask things differently: what's missing XFF for others to use?
>> I think it’s just missing a maven plugin to use it. All the other parts are 
>> ready.
>
> Actually just noticed that Jean has started one:
> https://github.com/xwiki-contrib/api-xff/tree/master/xff-maven
>
> I don’t know its status though.
>
> So what needs to be done is try the full chain and see what works/what 
> doesn’t.
>
> Unless jean could tell us what’s the status? (I’ll ping him on skype).
>
> Thanks
> -Vincent
>
>> Thanks
>> -Vincent
>>
>>> paul
 Vincent Massol 
 23 August 2016 at 11:01
 The format seems to be defined here:
 http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter#HFormat
>
> ___
> devs mailing list
> devs@xwiki.org
> http://lists.xwiki.org/mailman/listinfo/devs
> Vincent Massol 
> 23 August 2016 at 11:15
>> On 23 Aug 2016, at 11:13, Paul Libbrecht  wrote:
>>
>> Let's ask things differently: what's missing XFF for others to use?
>
> I think it’s just missing a maven plugin to use it. All the other parts are 
> ready.
>
> Thanks
> -Vincent
>
>> paul
>>> Vincent Massol 
>>> 23 August 2016 at 11:01
>>> The format seems to be defined here:
>>> http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter#HFormat
>
> ___
> devs mailing list
> devs@xwiki.org
> http://lists.xwiki.org/mailman/listinfo/devs
> Paul Libbrecht 
> 23 August 2016 at 11:13
> Let's ask things differently: what's missing XFF for others to use?
>
> paul
>
> ___
> devs mailing list
> devs@xwiki.org
> http://lists.xwiki.org/mailman/listinfo/devs
> Vincent Massol 
> 23 August 2016 at 11:01
> The format seems to be defined here:
> http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter#HFormat
>
> Thanks
> -Vincent
>
> ___
> devs mailing list
> devs@xwiki.org
> http://lists.xwiki.org/mailman/listinfo/devs
> Thomas Mortagne 
> 23 August 2016 at 11:00
> I can't give you much details on the XFF format, I just know the
> general goal. I only helped Jean on the "how to use Filter API" side.
>
>
>

___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Vincent Massol]

> On 23 Aug 2016, at 11:15, Vincent Massol  wrote:
> 
> 
>> On 23 Aug 2016, at 11:13, Paul Libbrecht  wrote:
>> 
>> Let's ask things differently: what's missing XFF for others to use?
> 
> I think it’s just missing a maven plugin to use it. All the other parts are 
> ready.

Actually just noticed that Jean has started one:
https://github.com/xwiki-contrib/api-xff/tree/master/xff-maven

I don’t know its status though.

So what needs to be done is try the full chain and see what works/what doesn’t.

Unless jean could tell us what’s the status? (I’ll ping him on skype).

Thanks
-Vincent

> Thanks
> -Vincent
> 
>> paul
>>> Vincent Massol 
>>> 23 August 2016 at 11:01
>>> The format seems to be defined here:
>>> http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter#HFormat
> 

___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Vincent Massol]

> On 23 Aug 2016, at 11:13, Paul Libbrecht  wrote:
> 
> Let's ask things differently: what's missing XFF for others to use?

I think it’s just missing a maven plugin to use it. All the other parts are 
ready.

Thanks
-Vincent

> paul
>> Vincent Massol 
>> 23 August 2016 at 11:01
>> The format seems to be defined here:
>> http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter#HFormat

___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Paul Libbrecht]

Let's ask things differently: what's missing XFF for others to use?

paul
> Vincent Massol 
> 23 August 2016 at 11:01
> The format seems to be defined here:
> http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter#HFormat

___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Vincent Massol]

The format seems to be defined here:
http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter#HFormat

Thanks
-Vincent

> On 23 Aug 2016, at 11:00, Thomas Mortagne  wrote:
> 
> I can't give you much details on the XFF format, I just know the
> general goal. I only helped Jean on the "how to use Filter API" side.
> 
> On Tue, Aug 23, 2016 at 10:57 AM, Paul Libbrecht  wrote:
>> Thomas,
>> 
>> I had seen this approach yesterday only.
>> I think my attempt is a tick more conservative.
>> But indeed, it seems to be the objective of the XFF filter.
>> 
>> Also, what I could not tell fully from the XFF attempt is whether the
>> optionality was there: are there property values or even complete
>> objects that I can leave gouped in some page XML? It seems to me that
>> itreally does not make sense to include them all in a file each. (e.g.
>> objects of the XWiki.TranslationDocumentClass).
>> 
>> paul
>> 
>> 
>> 
>> 
>> 
>> Thomas Mortagne wrote:
>>> Note that an alternative is to add a mojo in the XAR plugin (or write a
>>> new Maven plugin) based on Filter module to produce a XAR package from
>>> http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter which
>>> already been designed for what you want: provide a format that can be
>>> edited from filesystem easily. In theory all it takes is to put XFF
>>> filter in input and XAR filter in output and run.
>> 
>> __
___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Thomas Mortagne]

I can't give you much details on the XFF format, I just know the
general goal. I only helped Jean on the "how to use Filter API" side.

On Tue, Aug 23, 2016 at 10:57 AM, Paul Libbrecht  wrote:
> Thomas,
>
> I had seen this approach yesterday only.
> I think my attempt is a tick more conservative.
> But indeed, it seems to be the objective of the XFF filter.
>
> Also, what I could not tell fully from the XFF attempt is whether the
> optionality was there: are there property values or even complete
> objects that I can leave gouped in some page XML? It seems to me that
> itreally does not make sense to include them all in a file each. (e.g.
> objects of the XWiki.TranslationDocumentClass).
>
> paul
>
>
>
>
>
> Thomas Mortagne wrote:
>> Note that an alternative is to add a mojo in the XAR plugin (or write a
>> new Maven plugin) based on Filter module to produce a XAR package from
>> http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter which
>> already been designed for what you want: provide a format that can be
>> edited from filesystem easily. In theory all it takes is to put XFF
>> filter in input and XAR filter in output and run.
>
> ___
> devs mailing list
> devs@xwiki.org
> http://lists.xwiki.org/mailman/listinfo/devs



-- 
Thomas Mortagne
___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Paul Libbrecht]

Maybe what we need is a xar diff function ??
(compare my xar's source repo to the content of a wiki and say what
needs to be changed)

paul

> Guillaume Delhumeau 
> 23 August 2016 at 10:51
> To be extra-clear, let me explain my workflow when I design an application
> in XWiki.
>
> - First, I write my code directly in an instance of XWiki.
> - When it's ready, I export all the pages related to my application in a
> XAR.
> - Then, I uncompress my XAR inside my repository.
> - I clean the XML files using mvn xar:format.
> - I commit.
>
> And when I fix something, I do it directly in the wiki, then export the
> page in a XAR, uncompress it in my repository, clean the xml files
> manually
> (reverting the changed dates, etc...).
>
> What I don't want is to be forced to manually remove the source code from
> the XML to put it in the other file every time.
>
> Thanks,

___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Paul Libbrecht]

Thomas,

I had seen this approach yesterday only.
I think my attempt is a tick more conservative.
But indeed, it seems to be the objective of the XFF filter.

Also, what I could not tell fully from the XFF attempt is whether the
optionality was there: are there property values or even complete
objects that I can leave gouped in some page XML? It seems to me that
itreally does not make sense to include them all in a file each. (e.g.
objects of the XWiki.TranslationDocumentClass).

paul





Thomas Mortagne wrote:
> Note that an alternative is to add a mojo in the XAR plugin (or write a
> new Maven plugin) based on Filter module to produce a XAR package from
> http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter which
> already been designed for what you want: provide a format that can be
> edited from filesystem easily. In theory all it takes is to put XFF
> filter in input and XAR filter in output and run.

___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Guillaume Delhumeau]

To be extra-clear, let me explain my workflow when I design an application
in XWiki.

- First, I write my code directly in an instance of XWiki.
- When it's ready, I export all the pages related to my application in a
XAR.
- Then, I uncompress my XAR inside my repository.
- I clean the XML files using mvn xar:format.
- I commit.

And when I fix something, I do it directly in the wiki, then export the
page in a XAR, uncompress it in my repository, clean the xml files manually
(reverting the changed dates, etc...).

What I don't want is to be forced to manually remove the source code from
the XML to put it in the other file every time.

Thanks,

2016-08-23 10:45 GMT+02:00 Guillaume Delhumeau <
guillaume.delhum...@xwiki.com>:

> Hello Paul.
>
> I think it's an interesting idea (and I know this is not the first
> proposal in that way).
>
> But do you also consider the opposite use-case: editing some code in the
> wiki, and then export a XAR containing the metadata in an XML and the
> content in a plain text file?
>
> Otherwise, it makes it harder to fix some code in the wiki directly before
> committing it in the repository.
>
> Thanks,
>
> 2016-08-22 15:59 GMT+02:00 Paul Libbrecht :
>
>>
>> Hello fellow developers,
>>
>> many of you are developing XAR-packaged resources. The exchangeability
>> of these archives is one of the precious aspects of XWiki, I believe.
>>
>> However, if one see all the many GitHub-contributed xar packages, the
>> source is not there: it's in a Wiki with which one would "open it". A
>> pity since much of the content is code.
>>
>> I'd like to propose to allow XARs to be built form source code which is
>> true source that one edits.
>> Code files with their proper syntax coloring, editors, auto-completion,
>> search. Media-files that are edited with our tool of choice by a simple
>> double-click and not a complex pull, edit, and replace.
>>
>> Thus, my proposal is to allow inclusions within the XML files.
>> Inclusions should be of type:
>> - text to include page content or object properties (e.g. css of cssx
>> objects)
>> - attachments to include attached objects
>>
>> My current setup is documented here:http://jira.xwiki.org/bro
>> wse/XWIKI-13643
>> It does binaryInclude to generate base64 and textInclude to generate
>> XML-text.
>> Thus, I can advertise a github project which is a macro and whose
>> complex function is in a velocity file that is in a .vm file. However, I
>> do not need to explode everything, e.g. the text-content describing the
>> macro or (in this case too small) translation set.
>>
>> What do you think?
>> Should I rather generate complete attachment elements (name, length,
>> modif date, ...) instead of doing a binaryInclude that just generates
>> base64 ?
>> Is fetching files for inclusion beyond the boundaries of the project of
>> any use? (it is currently prohibited for safety)
>>
>> thanks in advance.
>>
>> Paul
>> ___
>> devs mailing list
>> devs@xwiki.org
>> http://lists.xwiki.org/mailman/listinfo/devs
>>
>
>
>
> --
> Guillaume Delhumeau (guillaume.delhum...@xwiki.com)
> Research & Development Engineer at XWiki SAS
> Committer on the XWiki.org project
>



-- 
Guillaume Delhumeau (guillaume.delhum...@xwiki.com)
Research & Development Engineer at XWiki SAS
Committer on the XWiki.org project
___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Paul Libbrecht]

> But do you also consider the opposite use-case: editing some code in the
> wiki, and then export a XAR containing the metadata in an XML and the
> content in a plain text file?
> Otherwise, it makes it harder to fix some code in the wiki directly before
> committing it in the repository.
No. Not for this case.
I guess it could be implemented by detecting which things are outside
and which are inside but it's not really trivial.

As far as I know, fetching the xar form the wiki really is a first step
which you do not do on a regular basis, you then push adjustments to the
wiki which you have edited in the source directory (e.g. using copy and
paste or attachments' uploader), then do an upload as a xar to make sure
it is clean.

The problem with fetching from the wiki is that it always inserts more
information than wanted.
E.g. a creator name, comments, ...
Whereas creating the xar from source can be tailored to not contain any
of this information (and part of that is checked by the xar mojo).
Am I wrong?

Paul
___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Guillaume Delhumeau]

Hello Paul.

I think it's an interesting idea (and I know this is not the first proposal
in that way).

But do you also consider the opposite use-case: editing some code in the
wiki, and then export a XAR containing the metadata in an XML and the
content in a plain text file?

Otherwise, it makes it harder to fix some code in the wiki directly before
committing it in the repository.

Thanks,

2016-08-22 15:59 GMT+02:00 Paul Libbrecht :

>
> Hello fellow developers,
>
> many of you are developing XAR-packaged resources. The exchangeability
> of these archives is one of the precious aspects of XWiki, I believe.
>
> However, if one see all the many GitHub-contributed xar packages, the
> source is not there: it's in a Wiki with which one would "open it". A
> pity since much of the content is code.
>
> I'd like to propose to allow XARs to be built form source code which is
> true source that one edits.
> Code files with their proper syntax coloring, editors, auto-completion,
> search. Media-files that are edited with our tool of choice by a simple
> double-click and not a complex pull, edit, and replace.
>
> Thus, my proposal is to allow inclusions within the XML files.
> Inclusions should be of type:
> - text to include page content or object properties (e.g. css of cssx
> objects)
> - attachments to include attached objects
>
> My current setup is documented here:http://jira.xwiki.org/
> browse/XWIKI-13643
> It does binaryInclude to generate base64 and textInclude to generate
> XML-text.
> Thus, I can advertise a github project which is a macro and whose
> complex function is in a velocity file that is in a .vm file. However, I
> do not need to explode everything, e.g. the text-content describing the
> macro or (in this case too small) translation set.
>
> What do you think?
> Should I rather generate complete attachment elements (name, length,
> modif date, ...) instead of doing a binaryInclude that just generates
> base64 ?
> Is fetching files for inclusion beyond the boundaries of the project of
> any use? (it is currently prohibited for safety)
>
> thanks in advance.
>
> Paul
> ___
> devs mailing list
> devs@xwiki.org
> http://lists.xwiki.org/mailman/listinfo/devs
>



-- 
Guillaume Delhumeau (guillaume.delhum...@xwiki.com)
Research & Development Engineer at XWiki SAS
Committer on the XWiki.org project
___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

Re: [xwiki-devs] XAR source projects should allow source files

[Thomas Mortagne]

Not that an alternative is to add a mojo in the XAR plugin (or write a
new Maven plugin) based on Filter module to produce a XAR package from
http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter which
already been designed for what you want: provide a format that can be
edited from filesystem easily. In theory all it takes is to put XFF
filter in input and XAR filter in output and run.

On Mon, Aug 22, 2016 at 3:59 PM, Paul Libbrecht  wrote:
>
> Hello fellow developers,
>
> many of you are developing XAR-packaged resources. The exchangeability
> of these archives is one of the precious aspects of XWiki, I believe.
>
> However, if one see all the many GitHub-contributed xar packages, the
> source is not there: it's in a Wiki with which one would "open it". A
> pity since much of the content is code.
>
> I'd like to propose to allow XARs to be built form source code which is
> true source that one edits.
> Code files with their proper syntax coloring, editors, auto-completion,
> search. Media-files that are edited with our tool of choice by a simple
> double-click and not a complex pull, edit, and replace.
>
> Thus, my proposal is to allow inclusions within the XML files.
> Inclusions should be of type:
> - text to include page content or object properties (e.g. css of cssx
> objects)
> - attachments to include attached objects
>
> My current setup is documented here:http://jira.xwiki.org/browse/XWIKI-13643
> It does binaryInclude to generate base64 and textInclude to generate
> XML-text.
> Thus, I can advertise a github project which is a macro and whose
> complex function is in a velocity file that is in a .vm file. However, I
> do not need to explode everything, e.g. the text-content describing the
> macro or (in this case too small) translation set.
>
> What do you think?
> Should I rather generate complete attachment elements (name, length,
> modif date, ...) instead of doing a binaryInclude that just generates
> base64 ?
> Is fetching files for inclusion beyond the boundaries of the project of
> any use? (it is currently prohibited for safety)
>
> thanks in advance.
>
> Paul
> ___
> devs mailing list
> devs@xwiki.org
> http://lists.xwiki.org/mailman/listinfo/devs



-- 
Thomas Mortagne
___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs

[xwiki-devs] XAR source projects should allow source files

[Paul Libbrecht]

Hello fellow developers,

many of you are developing XAR-packaged resources. The exchangeability
of these archives is one of the precious aspects of XWiki, I believe.

However, if one see all the many GitHub-contributed xar packages, the
source is not there: it's in a Wiki with which one would "open it". A
pity since much of the content is code.

I'd like to propose to allow XARs to be built form source code which is
true source that one edits.
Code files with their proper syntax coloring, editors, auto-completion,
search. Media-files that are edited with our tool of choice by a simple
double-click and not a complex pull, edit, and replace.

Thus, my proposal is to allow inclusions within the XML files.
Inclusions should be of type:
- text to include page content or object properties (e.g. css of cssx
objects)
- attachments to include attached objects

My current setup is documented here:http://jira.xwiki.org/browse/XWIKI-13643
It does binaryInclude to generate base64 and textInclude to generate
XML-text.
Thus, I can advertise a github project which is a macro and whose
complex function is in a velocity file that is in a .vm file. However, I
do not need to explode everything, e.g. the text-content describing the
macro or (in this case too small) translation set.

What do you think?
Should I rather generate complete attachment elements (name, length,
modif date, ...) instead of doing a binaryInclude that just generates
base64 ?
Is fetching files for inclusion beyond the boundaries of the project of
any use? (it is currently prohibited for safety)

thanks in advance.

Paul
___
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs