Welcome to BoltWire. You gave me a big list of things to do, but your suggestions and fixes are much appreciated. I'll rush through this quickly so I can get another release out right away...
On Fri, Dec 25, 2009 at 1:56 AM, Danny <[email protected]> wrote: > I set up a site "BoltWire ?????" (BoltWire Chinese Supporters) at > http://boltwire-zh.22web.net/ , which collected several data and > information for Chinese users. Welcome here. This is great! Please point them to the BoltWire site for the download... > For a thorough confirmation and solution, I tapped into the core and > tried to fixed several confronted problems. The modified patch for > v3.3.2 is named v3.3.2f, with several issues fixed and tested, at > least on my personal site. It can be download in the "Files" of this > Google Group, and you can use a software like WinMerge to scan the > differences. Feel free to merge the fixations you want into your new > version. I don't actually know whether I broke the rule of "BoltWire > may not be sold or redistributed in any form" in the liscense. Anyway, > if it's considered offensive please tell me. And note: this ought to > be a version for development purpose, if you use it on your own > without fully understand, you are at your risk! The best way to get the core patched is to post the line number and change on the mailing list. That makes it easier for me to implement changes. I'm very open to suggestions, though there may be times I disagree. In those case we can discuss it as a group, and if necessary I'll at the very least add a hook so you can have BoltWire work differently without having to hack the core. As for the license, I don't mind an alternate script being uploaded to the GoogleGroup's list, esp for the purpose of discussing possible changes, etc. That is still making the distribution from official BoltWire's sources. And I can always delete the file if I needed to. The main reason for that stipulation is to avoid a fork in our community, and encourage individuals to contribute to the core, rather than developing some tangential script. > The following is the beirf description of changes: I'm working through these one by one... > 1. Added a line to the initialization routine in engine.php so that > all admins actually have 'editor' membership as official site > mentioned: > > if (strpos(",$BOLTmemberships,", ',admin,')) $BOLTmemberships .= > ',editor'; This may be a good idea, but I'm not fully convinced it is needed in the core. Superadmins already have super privileges, and there may be times you grant editors different powers from admins. It seems more logical to me to add people individually to the appropriate lists, even if it means adding them to more than one list. It is also easy enough to do this in a config file or a plugin. But if the group feels differently we can change the default behavior to this. > 2. Added an htmlspecialchars to value of comm2func in engine.php so > that searching patterns like 'aaa && bbb' via action.search work as > via function. > > changed line: $args = BOLTargs($value, "x_$field"); > to: $args = BOLTargs(htmlspecialchars($value, ENT_NOQUOTES), "x_ > $field"); Doesn't it make more sense to do something like: $params = str_replace('&&', '&&', $params); in BOLTMfunc (markups.php) so this is fixed automatically for all function parameter values? Then we can leave the values as they are in the comm2func function, and we won't have to make any special fixes for any BOLTF functions... Note, I haven't changed this yet for the next release, as I'm waiting for pro/con arguments before proceeding. > 3. Moved the parsing of [messages] and [results] to markups.php to be > more well-organized and disallows them from being parsed in the skin > page but not in == ==. Also fixed a problem that on previewing a page > the [messages] in source are replaced by <div ....</div> but I forgot > how to fix it. Actually, these need to stay where they are processed, because sometimes messages come from different zones than where they are displayed. For example, you could have your message bar in the side zone which gets loaded before the action in the main zone. In this case your messages will not be displayed. Any specifics you can give on the problems of these being parsed in the skin page and the preview box would be helpful. If there are bugs here we certainly want to fix them. > 4. Besides 3, added an escape for $myquery so that search results > containing encoded UTF-8 urls are not transferred back to UTF-8, > making a mistake link like <a href="??">??</a>. Can you specify where you put this? Normally that variable is used to carry values from one page to another (via sessions), so they cannot be escaped. Unless you are talking about having them escaped after they are retrieved from the session for the next page. I would think the place is engine.php, lines 178-180, but there is no myquery there, so I'm not sure what you mean... > 5. Added 2 htmlspecialchars to BOLTFsource in functions.php so that > previewing page doesn't get contents in textarea html-decoded, ie. > < to <, & to & or so. Also removed an issue that [(source > post)] placed out of [box] might contain non-escaped HTML tags > directly. > > origin: > -- if (isset($args['post']) && isset($_POST[$args['post']])) return > BOLTescape($_POST[$args['post']]); > -- if (isset($args['get']) && isset($_GET[$args['get']])) return > BOLTescape($_GET[$args['get']]); > new: > -- if (isset($args['post']) && isset($_POST[$args['post']])) return > BOLTescape(htmlspecialchars($_POST[$args['post']], ENT_NOQUOTES)); > -- if (isset($args['get']) && isset($_GET[$args['get']])) return > BOLTescape(htmlspecialchars($_GET[$args['get']], ENT_NOQUOTES)); This is good. I'll make this change. Thank you for helping us tighten this up nicely. > 6. Paragraphs with "," or "$" listed in page "site.language.xxx" > aren't translated. A more significant derivative problem is that > system messages do not display translated if they consist $1 or $2. > This was fixed by modifying BOLTtranslate in engine.php: > > line: preg_match_all('/^([a-zA-Z0-9 .&#:;]+)(?: :: (.+))/m', > $translation, $BOLTtranslationArray[$language]); > replaced with: preg_match_all('/^(.+?)(?: :: (.+))/m', > $translation, $BOLTtranslationArray[$language]); > > I can't figure out why char listing is necessary, so I just use "." > for the freest match. It has been tested it and no adverse effect was > noted til now. Fair enough. It is great to have someone here that is good with regex! > 7. Fixed a wrong line in engine.php so that scripts markups.php, > functions.php, and conditions.php in $configDir can actually make > effect: > > line: $f = BOLTgetlink('', $configDir, "$script.php"); > replaced with: $f = BOLTgetlink('', "$script.php", '', > $configDir); This is one of many features I never use. Probably somewhere along the way the parameters in the BOLTgetlink function were changed and no one noticed. Thanks much for catching it... On the other hand, our functions are getting quite interdependent and our toolmapping quite powerful. And below we have fixed the site.auth.functions/commands. Therefore it seems like a better idea that we discourage (disallow) the complete replacement of scripts. Unless I hear good reasons against this, I would prefer to move that direction. IE, strip this feature out. > 8. Added replaces before BOLTsavepage in BOLTFinfo in functions.php so > that saving info values no more gets all page < escaped to < > > added: $content = str_replace('<', '<', $content); This seems reasonable to me. :) I want to revisit the whole < < replacement scheme soon. > 9. Modified a line in BOLTdomarkup so that something like `< (in > file `%26lt;) no more displayed as `%26lt; > > old: $out = str_replace('<', '<', $out); > new: $out = str_replace(Array('<', '%26lt;'), Array('<', > '<'), $out); This is only an issue if someone tries to put `< Is that correct? I went ahead and made that change. Not sure if it will have any implications, but it seems reasonable. > 10.Modified the rule of /= =/ in markup.php by removing the > unnecessary call of BOLTescape (done in BOLTcharEncode) and changed > the encode parameter to "lines" so that multi-lines are displayed > properly. You are right about the first point. As for the second, do you mean you changed the default fmt from '' to 'lines'? I'm not so sure about that. Can you give me an example of where the display is not correct? > 11.Modified markup.php so that uppercase HTML entities are allowed. I suppose the easiest way is to add /i to the end. Could probably also redo the html markups that way also... For whatever that is worth. > 12.Modified markup.php so that HTML tags with post-blanks or post- > slashes like <br /> are allowed. Also fixed a problem for edit- > conflict box displaying disorder by not recognizing <br />s. What rule did you use... Doesn't it allow invalid possibilities, like: <i /> or <b />? Do we want that? Not sure we do actually... > 13. Modified BOLTXdelete in commands.php so that on deleting field- > modified system pages it returns to the system folder instead of being > emptied, which may corrupt the whole site if it happens on code.skin. This cannot really be done--in the case when you want to delete some system file, such as an action page or something. If you delete an empty page a second time it reverts to the system page. I think that is a better approach. If the code.skin page got cleared out, I think we could get BoltWire to work in such a way it pulls in the skin automatically. For example, we could have a default skin in the default installation, rather than the system folder. And just set the default skin to 'default', rather than blank. This seems a better solution. I've been kind of thinking about moving that way anyway. > 14.Moved site.script to code.script to match the system setting. Also > edited several places so that code* and template* are now always > showed as source in normal and preview mode, and code.embed and > code.script never are. A new config "codePagesExclude" is created with > default value "code,code.embed,code.script,template", which lists all > pages that are code* or template* while not displayed as code. Also > made a function BOLTcodePageGroup as a shortcut to do the complicate > match. I'm not sure it is best to have site.script in the code.script hierarchy, which is set aside for certain kinds of scripts. Could work, but in my mind all site information is in the site hierarchy, and all actual scripts are in code.script. I'm not sure what changes you are talking about for the code and template pages, or the new variables and things. I would probably need a better description of what you are doing here and why to add it into the core. I haven't noticed any specific problems with these pages... > 15.Changed 'code.' BOLTconfig value in BOLTsnippets, BOLTinfoVar and > BOLTMembed so that config 'codePages' applies to them. Also, > 'template.' in BOLTMsession and BOLTFmail were changed to BOLTconfig > value. Thanks for finding all these. I think these config options are rarely used, but we should work to make changes to these work consistently. Making template pages configurable has never been supported. If we are going to add it, we will also need to update the various display functions in engine.php. Which I have done. I wonder if though in the sake of simplicity and speed it wouldn't be better to eliminate some of these config options that never get used? > 16.Edited and removed few lines in BOLTexecute and BOLTfunc in > engine.php so that site.auth.functions and site.auth.commands works > and their messages are displayed appropriately. I don't actually know > why but before the modification commands auth setting didn't work. > > However, the auth determination for site.auth.functions and > site.auth.commands doesn't seem to be compatible with what official > site say: listing pages with avalible "functions/commands", instead of > listing pages with available "members/membergroups". I'm not sure how you fixed them, but you are correct--the BOLTauth function no longer supports the intended functionality for the commands and functions authorizations. I have rewritten the BOLTauth function slightly to get what I originally had in mind, only better. Now you can very easily ban certain commands/functions from your entire site, or specific pages with entries like this: *: breadcrumb comments*: search,find In the above example, I should note, breadcrumbs would be allowed in the test hierarchy, as only the closest line is used. This is close to what I originally had in mind, but is more of a banlist rather than an allow list, which I think will be more useful > 17.Modified BOLTFpreview in functions.php so that on previewing page $ > escaped by `, <code> or /= =/ no more displayed as $, and '~data~' > no more displayed as '~data~'. Would you be so kind as to post your fix so I could incorporate it into the core? > 18.Fixed "site.join" so that it doesn't show ugly "adminAdmin"; > duplicated 'editor' and 'admin' were also excluded. Thanks. Just added... (but to action.join). > 19.Modified "action.data" so that it works correctly. Interestingly, > in the old version fieldname were mismatched, however it still worked > in v3.3.1 and earlier. Can you explain what is incorrect about that data action? I have not noticed any problems with it? > 20."site.language" do not contain several system messages, which were > also included in the pack. If you can list them, I'll add them... > 21.Fixed several typos and nonsense or missense lines seen anywhere. If you want to see those included in the core please specify them. I'd rather not have to download your version and try to track down every change you've made. It's much easier for me to one by one go through and evaluate your proposed changes and then either incorporate them into the core or not. That's basically how the system here has worked to date. > # Bugs or features? > > 1. There are several encoding/parsing problems in BoltWire v3.3.2 > overviewed as the following: > 1) On previewing a page: (all fixed above) > a) $ escaped by `, <code> or /= =/ becomes $ > b) ~data~ becomes ~data~ > > 2) On previewing a page, the source is changed: (all fixed above) > a) All html entities are unescaped such as < to <, & to > &, ™ to ™ > b) [messages] become <div class='message'>.......</div> If you have a fix for these please post it. Otherwise it may be a bit till I track all these down. I'm trying to get out a new release pronto, for security reasons. So these have to wait otherwise. > 3) On saving a page (source code rewrited): > a) ' or " with one or more \ before them (ex: \", \\\\\\\\') > become single ' or " > b) \ or n with one or more \ before them become single \ or > breakline if config "blockslashes" is true > c) n with a \ before becomes a breakline if config > "blockslashes" is false > d) %26lt; becomes <, avoid by seperating with `, like %`26lt; > e) %3c becomes <, if someone wants to display something like %3c > %..%..%.. in <code> he would get a problem We have done a good bit of work with our character encoding system. It is not perfect but functional. In a future release, we may revise this again, but I would prefer to hold off on tackling these problems till some other things are taken care of. > 4) On displaying page: > a) <p> is usually generated mistakingly, especially in > paragraphs with multi-lines. > b) * # = works significantly differ from other wikis do. > c) lines ended with // get wrapped into an <p> > d) `< is displayed as %26lt; (fixed above) > e) multi-line texts in /= =/ become one-line (fixed above) > f) [messages] in skin page without wrapping == == are still > parsed (fixed above) > g) uppercase html entities are not parsed (fixed above) > h) html tags like <br /> are not parsed (fixed above) > i) html tags with attrs are not parsed The problem with <p> tags is known. We are working on a better solution in an upcoming release. I have made some of the fixes you suggested above, but not all. Some, you did not specify the fix, so I have not yet implemented, but am certainly willing to. > 3)b) I haven't fully understand what BOLTstripslashes does but if I > adds this line: > BOLTreplace("\n", '\n'); > before the line: > BOLTreplace('~data~', '~da`ta~'); > all input '\n' wound not be replaced by a line break; Why would we want this? > 4)a) For instance, the page > http://www.boltwire.com/index.php?p=docs.handbook.markups > in the official site contains several bad <p>s. ... Yes, this has been noted before > 4)b) For example: > > # blah (ordered like 1.) > #* blah1 (expected to be underorded but is ordered) > #* blah2 (expected to be underorded but is ordered) > #*# blah3 ... Ok, I'll look into these things. The bullets have been very difficult in the past. But you are right, we should be able to have ordered and unordered nest properly. I haven't tested this particularly in the past. I'm open to suggestions for how we should have the bullets work. Especially something simpler... > In parctice, parsing <p>s with php is very difficult. The major > reason is that <p> cannot wrap a block element. Although we can set up > a plenty of rules to rule out <h#> <blockquote> <div> <table> <ul> > <ol> <pre> etc in the <p> </p>, however if there are escaped input or > function/variable output problems may occur. Furthermore, a user may > define the stylesheet on his/her own -- if we want to handle it we are > just inventing another "browser". We can see that many online services > like Yahoo Blogs, Google Docs, Google Sites, or Google * or else do > not actively parse <p>s. They just left the problem to the users, who > can still manually write <p>s -- and debug -- on their own. > > I'd advice that adding a config called "honorParagraphs", just like > "honorLines". It determines that if hL and hP, all lines and > paragraphs are parsed; if hL no hP, all \n are single linebreak (<br/ >>; if hP not hL, only paragraphs are parsed; if both not, do no parse. > We won't die without paragraphs although <br/>s are sometimes ugly. > However if the paragraph parser is disexpective we may get stroke > dealing with it. > > Also, the output html code is quite ugly. It would be better to add > more \n to format it, although it would increase the difficulty for > the parser to handle the linebreaks. As noted above, we will revisit the paragraph system in the future. I intend to continue intuitive handling of paragraphs if at all possible. But it is not perfect yet... > #Other questions/suggestions/optimizations > > 1. Instead of having < escaped, the pages "code.*" are stored just as > source code, which causes serious display disorder on action=undo or > action=history, and worsely, make restoring to ealier version > unvailable. It might be better to display diff of code pages after > escaping html tags. Ok, I'll look into this as well. If we are going to reopen the character encoding system, we should solve all these problems at one time... > Well, in my opinion, I don't think that saving normal pages with > '<' escaped and the derivatives is necessary, since they are preparsed > into < in BOLTdomarkup just before applying the markups. The > performance can be improved by saving the disk by not escaping < to > < in the data file, and by removing the code that manages the > transformation in BOLTsaveEscapes, BOLTcharEncode, BOLTdomarkup, > BOLTFsource, BOLTFpreview and BOLTFinfo. Furthermore, we just add > htmlentities escaping before comparing 2 files in BOLTdiff, and the > code page problem is solved. Besides, in this way we don't have to > rack our brains to display original %26lt; < %3c on the screen in > BoltWire. However a serious concern is that it might be discompatible > with old data. Obviously things were done this way to add security, but I find your reasoning compelling. I plan to reopen the char encoding system in the very near future, and will very likely move this direction. Anything we can do to simplify the smooth internal operation of BoltWire is a plus. We just will have to be careful that there is no way to inject pure page source code into a page output by any means. Meaning, even some plugins may need to be re-examined. > 2. Cache system is a good idea but there is a problem. Generally I > would turn off caching from admins and members to prevent member-only > pages from viewed by guests. Now the caches are made by guests. If I > logged in, I still get pages that are for guests - including the > action bar and everything - so if I want to edit a page or to view a > page for members, I have to type &action=edit or &action=view manually > and it is very inconvenient. It would be better if there is another > config such as 'noUseCacheGroup' which means that if you are logged in > in the group defined you always receive a newly-parsed true page > instead of a cached version, but you don't create a cache. This is a good idea. I'll add that to my to do list. > 3. In most web forums, blogs, or wiki systems, going back to "previous > page" after sending a form with texts edited shows the post-edited > ones. However, BoltWire shows the texts of the current source of the > page being edited. This might overwrite the edited text and the > editor's works become in vain if there was an error so that the > editing submitted hasn't been saved (unfortunately, it happens quite > often...). Are you say the browser displays the old text when you hit the back button? If so I wonder how we fix it? > 4. The documentation of markup table in the help system is out of > date. Specifics? > 5. <rules xxx>content</rules> seem not to work it a page contains 2 or > more <rules> blocks. You have really tested just about everything in our system. I'm impressed!!! Probably the markup rule needs to be more restrictive... > 6. wikiWords doesn't seems to work. All page names are stricted to > lowercase even it's set to true. Ok. Another old feature, little used... To be honest, in this case I'm much more inclined to erase the feature than try and fix it. It's easy enough to add a special markup that does the equivalent from the users perspective. But BoltWire really should keep case insensitivity all through the system. > 7. I cannot figure out what this line in BOLTdomarkup is doing: > $out = preg_replace('/\{\*\:([^{}+=*:]+)\}/', '{'.$page.'*:$1}', > $content); // base pages > since {page*:var} is used for retrieving data from stamp page... > and it causes /= {*:} =/ display unexpectedly You may be right. I'm not sure myself. I think it is supposed to replace the generic {*:field} with the more specific {page*:field} but it raises the question in my mind how it knows which stamp page to use. Could be something else. I'll look into this later. > 8. It would be better to combine BOLTCexists and BOLTexists. Also, the > former takes 'plugins/', 'files/' but the latter takes 'system'. The first is the conditional, and doesn't care whether or not the page is in the system or pages folder, or even a subfolder. It can be used to test for various other files as well. The latter is an internal system folder used to tell whether or not a page exists, and optionally if system or local. Since they have different purposes, I'm not sure I see the advantage of mixing them. Also, if someone wanted to create a custom set of conditionals for some reason, we would need the system function available. > 9. BOLTCstamp seems to take only current page instead of any page as > the comments say. It just tells whether or not a stamp page exists, regardless of whether there is one or twenty. > 10.I can't figure out what farm/pub is for... This is just an optional folder for farmwide files. Some people requested it. I don't use it personally. Ok, I think I have your first email answered... Let me get on to the next couple!!! Cheers, Dan P.S. Maybe in the future you could post your fixes one or two at a time so I can process them as they come in. :) I kind of use the mailing list as my tracking system. This email was a bit overwhelming... -- You received this message because you are subscribed to the Google Groups "BoltWire" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/boltwire?hl=en.
