Re: [Lazarus] Howto work with wikiget and wikiconvert tools
On Tue, 20 Jan 2015 10:34:30 +0100 Mattias Gaertner nc-gaert...@netcologne.de wrote: [...] Our Wiki does not have that long page names. The problem is that non ASCII characters are converted to hex codes, tripling the length. wikiget could use the actual names instead and encode them using some sort of UTF-7. I implemented the UTF-8 to UTF7W and back functions. wikiget download works again. Mattias -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] Howto work with wikiget and wikiconvert tools
As Pagenames seemingly are allowed to contain up to 256 characters with relevant upper/lower case and there are file systems that support only up to 256 characters for file names and there are (silly !!!) file systems that don't support relevant upper/lower case, something like MD5 seems to be the only viable option. Of course a file with reference list needs to be provided. -Michael -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] Howto work with wikiget and wikiconvert tools
On Tue, 20 Jan 2015 09:50:13 +0100 Michael Schnell mschn...@lumino.de wrote: As Pagenames seemingly are allowed to contain up to 256 characters with relevant upper/lower case and there are file systems that support only up to 256 characters for file names and there are (silly !!!) file systems that don't support relevant upper/lower case, something like MD5 seems to be the only viable option. Of course a file with reference list needs to be provided. Our Wiki does not have that long page names. The problem is that non ASCII characters are converted to hex codes, tripling the length. wikiget could use the actual names instead and encode them using some sort of UTF-7. I implemented the UTF-8 to UTF7W and back functions. Mattias -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] Howto work with wikiget and wikiconvert tools
On 01/20/2015 11:58 AM, Andreas Frieß wrote: The wikiget should handle the situation. All pages with an an not workable filename should be ignored and logged and the programm go on running. I feel that this is a workaround. People will go on creating rather long non-ASCII page names that _should_ be decently handled by the converter. Right now you are creating the standard how the files are stored. Hence it should be workable for the foreseeable future. -Michael -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] Howto work with wikiget and wikiconvert tools
Am 20.01.2015 um 10:45 schrieb Michael Schnell: On 01/20/2015 10:34 AM, Mattias Gaertner wrote: Our Wiki does not have that long page names. The problem is that non ASCII characters are converted to hex codes, tripling the length. wikiget could use the actual names instead and encode them using some sort of UTF-7. UTF-7 of course provides much better human readability, and other than MD5 it's unique in both directions. In case of a too long UTF-7 representation, an error message needs to be implemented asking the user to modify the pagename in the Wiki. I think this all is not the problem (or not directly yet). If the definition in the Lazarus wiki is: *) Use only declaration of pages in english (ASCII names) *) the Language dependen page must have basepage/shotterm for language eg. Basepage is Lazarus_Help, the languagedependes pages have to be Lazarus_Help/de, Lazarus_Help/ru, Lazarus_Help/cn and so on. with this simple definition you have the names allways clear, structured, and readable for everone. The content of the page can be in whatever you want. This definition are the normally using of the Lazarus wiki (in the past), but some persons break the the schema used before (why - misunderstanding the schema ?). Andreas -Michael -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] Howto work with wikiget and wikiconvert tools
On 01/20/2015 10:34 AM, Mattias Gaertner wrote: Our Wiki does not have that long page names. The problem is that non ASCII characters are converted to hex codes, tripling the length. wikiget could use the actual names instead and encode them using some sort of UTF-7. UTF-7 of course provides much better human readability, and other than MD5 it's unique in both directions. In case of a too long UTF-7 representation, an error message needs to be implemented asking the user to modify the pagename in the Wiki. -Michael -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] Howto work with wikiget and wikiconvert tools
On 01/20/2015 11:30 AM, Andi wrote: If the definition in the Lazarus wiki is: I feel that _if_ you do a patch to the converter program(s), they should work independently of such current definition (any language should be allowable in future). and here, IMHO UTF-7 is a good idea. -Michael -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] Howto work with wikiget and wikiconvert tools
Am 20.01.2015 um 11:47 schrieb Michael Schnell: On 01/20/2015 11:30 AM, Andi wrote: If the definition in the Lazarus wiki is: I feel that _if_ you do a patch to the converter program(s), they should work independently of such current definition (any language should be allowable in future). and here, IMHO UTF-7 is a good idea. The wikiget should handle the situation. All pages with an an not workable filename should be ignored and logged and the programm go on running. Andreas -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] Howto work with wikiget and wikiconvert tools
Am 20.01.2015 um 12:01 schrieb Michael Schnell: On 01/20/2015 11:58 AM, Andreas Frieß wrote: The wikiget should handle the situation. All pages with an an not workable filename should be ignored and logged and the programm go on running. I feel that this is a workaround. People will go on creating rather long non-ASCII page names that _should_ be decently handled by the converter. Right now you are creating the standard how the files are stored. Hence it should be workable for the foreseeable future. You REALY MEAN a basepage like φετα and the english localized page φετα/en ist a valid (and german φετα/de)? Normally the basepage was english (without a slash and a languagecode) and all the languagedepend pages have a slash with the two char languagecode. This system normally used yet have no problem with non ascii cahrs, because they are not used. I think this system was good. If somebody do not use the naming convention, he is not able to produce documentation. I think the people who have produced the pages with problem IMHO didnt understand full the ruling of the pages. Andreas -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] Howto work with wikiget and wikiconvert tools
On 01/20/2015 02:14 PM, Andreas Frieß wrote: I think the people who have produced the pages with problem IMHO didnt understand full the ruling of the pages. OK. (But I think that unfortunately the Wiki does not provide any means to enforce such rules :-( ) Hence the Help Supervisor Team would be required to edit such stuff out. -Michael -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] Howto work with wikiget and wikiconvert tools
Am 20.01.2015 um 14:30 schrieb Michael Schnell: On 01/20/2015 02:14 PM, Andreas Frieß wrote: I think the people who have produced the pages with problem IMHO didnt understand full the ruling of the pages. OK. (But I think that unfortunately the Wiki does not provide any means to enforce such rules :-( ) Hence the Help Supervisor Team would be required to edit such stuff out. Who is the team ? I have look for changes - it was not allways a mistake in naming. 11:23, 7. Dez. 2014 Alexrus (Diskussion | Beiträge) K (2.035 Bytes) (moved FPC documentation/ru to Документация FPC: Руссификация) 16:33, 8. Dez. 2014 Alexrus (Diskussion | Beiträge) K (21.139 Bytes) (moved Lazarus Documentation/ru to Документация Lazarus) 12:00, 7. Dez. 2014 Alexrus (Diskussion | Beiträge) K (9.846 Bytes) (moved History/ru to История) 16:44, 8. Dez. 2014 Alexrus (Diskussion | Beiträge) K (22.319 Bytes) (moved Components and Code examples/ru to Компоненты и примеры кода) 05:08, 26. Sep. 2014 Alexrus (Diskussion | Beiträge) K (19.237 Bytes) (moved Translations / i18n / localizations for programs/ru to Переводы / i18n / локализация программ) -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] Howto work with wikiget and wikiconvert tools
On 01/20/2015 02:59 PM, Andreas Frieß wrote: Who is the team ? :-) :-) :-) (We did discuss this in the German forum.) No answer on that from my, as I don't want to discourage you yet again. :-( -Michael -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] Howto work with wikiget and wikiconvert tools
When i am move a page to the right name, eg. to Sand Box/ru the old page (source) is still available and in the wiki. So the moving did copy the content to the new position, but not remove the old page (name). Who can remove the (now useless) page 'Песочница http://wiki.freepascal.org/%D0%9F%D0%B5%D1%81%D0%BE%D1%87%D0%BD%D0%B8%D1%86%D0%B0' ? Andreas -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] Howto work with wikiget and wikiconvert tools
On Mon, 19 Jan 2015 19:11:39 +0100 Frieß fri...@gmx.at wrote: [...] I _feel_ that pagenames should be pure ASCII to avoid confusion, but I am not an expert on that. It looks, the naming is allowed. See http://en.wikipedia.org/wiki/Wikipedia:Naming_conventions_(technical_restrictions) http://en.wikipedia.org/wiki/Wikipedia:Naming_conventions_%28technical_restrictions%29See 'Title length' there. So the representation in the Filesystem must handle this. Or the naming convention of the wikesite have a (internal) rule, not to use UTF-8. Of course the Wiki supports UTF-8 for page names. The problem is that wikiget creates one file per page and so it needs to map a wiki page name to a file name. The file name must also work on all common file systems and version control systems. For example the Wiki is case sensitive, file systems in general are not. At the moment wikiget uses a simple mapping, that keeps English letters and encodes the rest. This makes debugging somewhat easier. Unfortunately it can triple the length leading to too long file names. An alternative would be UTF-7 (i.e. special characters like \ and / encoded). Functions WikiPageToFilename and WikiFilenameToPage. The Wiki allows pretty long page names. This means very long page names need a special treatment. For example using a md5sum. OTOH long page names are bad style, so it is better to rename the wiki page. Mattias -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] Howto work with wikiget and wikiconvert tools
On Mon, 19 Jan 2015 15:50:38 +0100 Frieß fri...@gmx.at wrote: [...] At this time i have found the tools in the directories of Lazarus !! in component/wiki. This tools look better So i have try to use the tools. Its look running, but i have not enough knowledge about the handling with the tools. I see the last checkin was 2012, whats happen with this tools ? Not used anywhere, stopped ? Project dead ? No. I just had some other priorities. May i help to get this tools working ? They were working. I see wikiget has now a problem with an overlong filename from a strange wiki page. Feel free to improve it. Mattias -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] Howto work with wikiget and wikiconvert tools
On 01/19/2015 05:32 PM, Frieß wrote: So is one question, should this corrected in the wiki or accepted by the wikiget ? I think, firstly wikiget should issue a decent error message when it his such a page. Does it ? I _feel_ that pagenames should be pure ASCII to avoid confusion, but I am not an expert on that. Thanks for your effort ! -Michael -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] Howto work with wikiget and wikiconvert tools
On Mon, 19 Jan 2015 17:41:17 +0100 Michael Schnell mschn...@lumino.de wrote: On 01/19/2015 05:32 PM, Frieß wrote: So is one question, should this corrected in the wiki or accepted by the wikiget ? I think, firstly wikiget should issue a decent error message when it his such a page. Does it ? Yes: Unable to create file: ... I _feel_ that pagenames should be pure ASCII to avoid confusion, but I am not an expert on that. wikiget converts page names to ASCII file names. But the name got too long. Mattias -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] Howto work with wikiget and wikiconvert tools
Am 19.01.2015 um 17:17 schrieb Mattias Gaertner: On Mon, 19 Jan 2015 15:50:38 +0100 Frieß fri...@gmx.at wrote: [...] At this time i have found the tools in the directories of Lazarus !! in component/wiki. This tools look better So i have try to use the tools. Its look running, but i have not enough knowledge about the handling with the tools. I see the last checkin was 2012, whats happen with this tools ? Not used anywhere, stopped ? Project dead ? No. I just had some other priorities. I see, fpc and Lazarus are growing fast. Is there more information available or only the source ? May i help to get this tools working ? They were working. I see wikiget has now a problem with an overlong filename from a strange wiki page. Feel free to improve it. This is not directly a problem from wikiget, it comes from russian pagenames instead of pagename/ru. So is one question, should this corrected in the wiki or accepted by the wikiget ? Mattias -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] Howto work with wikiget and wikiconvert tools
On Mon, 19 Jan 2015 17:32:37 +0100 Frieß fri...@gmx.at wrote: Am 19.01.2015 um 17:17 schrieb Mattias Gaertner: On Mon, 19 Jan 2015 15:50:38 +0100 Frieß fri...@gmx.at wrote: [...] At this time i have found the tools in the directories of Lazarus !! in component/wiki. This tools look better So i have try to use the tools. Its look running, but i have not enough knowledge about the handling with the tools. I see the last checkin was 2012, whats happen with this tools ? Not used anywhere, stopped ? Project dead ? No. I just had some other priorities. I see, fpc and Lazarus are growing fast. Is there more information available or only the source ? The source and the README. May i help to get this tools working ? They were working. I see wikiget has now a problem with an overlong filename from a strange wiki page. Feel free to improve it. This is not directly a problem from wikiget, it comes from russian pagenames instead of pagename/ru. So is one question, should this corrected in the wiki or accepted by the wikiget ? If it is only a few pages, it may be easier to fix it in the wiki. Otherwise we need a different system for storing the pages. Mattias -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] Howto work with wikiget and wikiconvert tools
Am 19.01.2015 um 17:41 schrieb Michael Schnell: On 01/19/2015 05:32 PM, Frieß wrote: So is one question, should this corrected in the wiki or accepted by the wikiget ? I think, firstly wikiget should issue a decent error message when it his such a page. Does it ? I _feel_ that pagenames should be pure ASCII to avoid confusion, but I am not an expert on that. It looks, the naming is allowed. See http://en.wikipedia.org/wiki/Wikipedia:Naming_conventions_(technical_restrictions) http://en.wikipedia.org/wiki/Wikipedia:Naming_conventions_%28technical_restrictions%29See 'Title length' there. So the representation in the Filesystem must handle this. Or the naming convention of the wikesite have a (internal) rule, not to use UTF-8. One of this sites in on the Main Page of the wiki. Normally the Language depended site for russian should be 'Main_Page/ru' but it is '%D0%97%D0%B0%D0%B3%D0%BB%D0%B0%D0%B2%D0%BD%D0%B0%D1%8F_%D1%81%D1%82%D1%80%D0%B0%D0%BD%D0%B8%D1%86%D0%B0'. The Ukrain Page - also in cyrillic ist well done 'Main_Page/uk'. Also some other Page have the same Problem (see http://wiki.freepascal.org/index.php?title=Special:AllPagesfrom=go32v2_developmentto=%EB%8C%80%EB%AC%B8 on the end of page). So the goal is, should anybody rename this sites to their correct naming or not ? The site didnt handle their names in 'wikithinking'. Thanks for your effort ! -Michael -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] Howto work with wikiget and wikiconvert tools
Am 19.01.2015 um 18:53 schrieb Mattias Gaertner:
On Mon, 19 Jan 2015 17:32:37 +0100
Frieß fri...@gmx.at wrote:
Am 19.01.2015 um 17:17 schrieb Mattias Gaertner:
On Mon, 19 Jan 2015 15:50:38 +0100
Frieß fri...@gmx.at wrote:
[...]
At this time i have found the tools in the directories of Lazarus !! in
component/wiki. This tools look better So i have try to use the tools.
Its look running, but i have not enough knowledge about the handling
with the tools.
I see the last checkin was 2012, whats happen with this tools ? Not used
anywhere, stopped ? Project dead ?
No. I just had some other priorities.
I see, fpc and Lazarus are growing fast. Is there more information
available or only the source ?
The source and the README.
May i help to get this tools working ?
They were working. I see wikiget has now a problem with an overlong
filename from a strange wiki page.
Feel free to improve it.
This is not directly a problem from wikiget, it comes from russian
pagenames instead of pagename/ru.
So is one question, should this corrected in the wiki or accepted by the
wikiget ?
If it is only a few pages, it may be easier to fix it in the wiki.
Otherwise we need a different system for storing the pages.
Ok, i look for a workaround in the wikiget, not to store this sites and
to get a message.
i have done a quick and dirty fix to see the wikiget running, but it was
not the best way. I will try a better soloution.
--- C:/Users/andi/AppData/Local/Temp/wikiget.lpr-revBASE.svn003.tmp.lpr
Mo 26. Mär 20:28:31 2012
+++ X:/Pascal/Pascal/lazarus/components/wiki/wikiget.lprSo 18. Jän
16:19:44 2015
@@ -296,7 +296,7 @@
Page:=copy(s,StartPos,p-StartPos);
while (Page'') and (Page[1]='/') do
System.Delete(Page,1,1);
- if (Page'') and (not IsIgnoredPage(Page)) then begin;
+ if (Page'') and (not IsIgnoredPage(Page)) and
(Page[1]'%') then begin;
//writeln('TWikiGet.GetAll Page=',Page,'');
Filename:=PageToFilename(Page,false);
AddWikiPage(Page);
Mattias
--
___
Lazarus mailing list
Lazarus@lists.lazarus.freepascal.org
http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
--
___
Lazarus mailing list
Lazarus@lists.lazarus.freepascal.org
http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
