Re: [OSM-talk] [OSM-dev] The future of Potlatch
SteveC wrote: On 2 May 2008, at 12:38, Christopher Schmidt wrote: Some things don't require referential integreity: selecting ways/nodes within a bounding box can't hurt the referential integrity of the database (so long as the code is well-maintained), so the harm in converting those methods (which are probably the single most performance important aspect of Potlatch?) to SQL is relatively low, so far as I can tell... One of the other reasons for moving to rails was the holy grail of using postgres, and rails is theoretically db independent. Of course it didn't work out that way. Main problem here is the lack of good 'GIS' in other databases. It looks like I may be forced to move the gis data to a postgres database despite the fact that I have had all other systems running Firebird/Interbase for 15 years :( On the specific point, I'm all for more speedy SQL on things like that so long as there is a rails logic way too - in much the same way as there apparently is with the tileid stuff. Then if things change significantly (like, say with changesets, rollback, spatial data types) we don't have limited options. Rails may be nice for some, but for those of us how are coding other applications daily without any reference to rails it is a turn off. I do not have the time to bother even looking which is a reason that I have not been able to contribute on that side. I work with raw data via SQL and will continue that way with 25 years of C++ applications and now increasingly PHP. So any extensions I come up with ( such as NLPG data interface ) will be in PHP :) -- Lester Caine - G8HFL - Contact - http://home.lsces.co.uk/lsces/wiki/?page=contact L.S.Caine Electronic Services - http://home.lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk// Firebird - http://www.firebirdsql.org/index.php ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] [OSM-dev] The future of Potlatch
On Sat, May 03, 2008 at 06:56:40AM +0100, Lester Caine wrote: SteveC wrote: On 2 May 2008, at 12:38, Christopher Schmidt wrote: Some things don't require referential integreity: selecting ways/nodes within a bounding box can't hurt the referential integrity of the database (so long as the code is well-maintained), so the harm in converting those methods (which are probably the single most performance important aspect of Potlatch?) to SQL is relatively low, so far as I can tell... One of the other reasons for moving to rails was the holy grail of using postgres, and rails is theoretically db independent. Of course it didn't work out that way. Main problem here is the lack of good 'GIS' in other databases. It looks like I may be forced to move the gis data to a postgres database despite the fact that I have had all other systems running Firebird/Interbase for 15 years :( But the backend of OSM matters little, in that case. You could say, in the same way, that OSM should be topographical (instead of topological) because databases don't support topology well. But that doesn't make sense: OSM is an API designed to allow people to collect the data: Not a GIS system designed to allow people to interact with the data with GIS tools. That aspect of it can easily come on top of OSM. On the specific point, I'm all for more speedy SQL on things like that so long as there is a rails logic way too - in much the same way as there apparently is with the tileid stuff. Then if things change significantly (like, say with changesets, rollback, spatial data types) we don't have limited options. Rails may be nice for some, but for those of us how are coding other applications daily without any reference to rails it is a turn off. This is true in *any* language. There's nothing that everybody knows how to code for: Rails is popular, so this is much less true than in the days before the server was written in Ruby. I do not have the time to bother even looking which is a reason that I have not been able to contribute on that side. I work with raw data via SQL and will continue that way with 25 years of C++ applications and now increasingly PHP. So any extensions I come up with ( such as NLPG data interface ) will be in PHP :) Which is fine: it just means that you don't hack on OSM.org. This is true of the majority of OSM contributors and developers. Regards, -- Christopher Schmidt MetaCarta ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] [OSM-dev] The future of Potlatch
In message [EMAIL PROTECTED] Christopher Schmidt [EMAIL PROTECTED] wrote: 2 On Fri, May 02, 2008 at 12:24:35AM +0100, Tom Hughes wrote: In message [EMAIL PROTECTED] Tom Carden [EMAIL PROTECTED] wrote: I think the fact that it has its own API is a much bigger concern than it being written in AS 1.0 is. If Potlatch was using the main API, development of API-backed features in Potlatch could be shared by other editors too. Any tests written for the API would help Potlatch. Any changes to the schema would only have to happen once. etc. etc. I think most of us (with the possible exception of Richard) would agree here. Well actually I don't mind the existence of the AMF API as such so long as it is just concerned with decoding the RPC calls and encoding the results, and it uses the rails object model to do all the work so that important code is shared with the XML API. There's an important difference between this and Potlatch using the main API. Well yes, the two are not very well aligned at the moment and that also needs fixing. There are, I believe, some things that Potlatch can do that no other client can do, because the API to do it is not available. Specifically: * whichways_deleted gets deleted ways in a bbox, which the main API doesn't provide * getway_old has a last deleted version, which seems different More problematic than the specific methods which do or don't exist, however, is the fact that the Potlatch way of interacting with them is *entirely* different -- so when a bug is fixed in the main API, the fix has to be duplicted in the Potlatch/amf_controller. Which is why we're trying to make as much code as possible common to both - there's a long way to go, but it's a good target. The point I would like to reach is one where the only difference is how arguments are decoded and how results are encoded and everything in between is the same with the same set of API calls exposed via both the AMF and XML interfaces. This affects development of the server side APIs. You, at one point, put forward a significant amount of effort in improving the Rails code in amf_controller: that work benefited only Potlatch. If Potlatch was using the same backend rails calls (rather than just objects/models), then that development time could have benefited other clients as well. Steve did most of the work on railsifying the AMF code. All I did was one little performance tweak I think. Potlatch has a very different way of interacting with the API. This way of interacting with the API has some benefits, and could allow for other clients similar to Potlatch to be developed without being tied explicitly to amf_controller, which is (at least at the moment) essentially Potlatch specific. (Things like 'masterscale' and 'potlatch_lon' seem to be indicate this anyway: maybe I'm wrong here.) Yes, those are ugly glitches, and things that I would prefer to see done on the client side. I suspect that they are mostly on the server side for historical reasons - bear in mind that Potlatch was being written before we moved to rails so the server side component was originally written for the old pure ruby server environment. We have in fact started moving towards that goal with some work that Steve did on some of the AMF calls. This is a different goal, that of replacing SQL with rails objects. A valuable goal, but not the same as abstracting the data selection calls to a single set of code, and then using that code to serve both the main API and amf_controller. You make it sound like the data selection code is horribly complicated but really all it amounts to is a few find calls on rails classes. That said, it would be better to common up the data selection code as well - what we really need is a volunteer to work on this stuff. The problem is that it turned out that, even after I had optimised the code, it is significantly slower to go through the rails object model that to make direct SQL queries. This is interesting, but (imho) less relevant: The value of the above is actually doubled by this statement, since if Potlatch and the API were using the same function, one could optimize this, and since the code is used by *all* clients (not just one) you would see any problems with SQL calls more generally, and perhaps find it easier to debug them. (I think this is compounded by the fact that AMF is a format which is hard to replicate outside of a flash client, which makes server side developers less likely to have the technical chops to debug them.) A common library for API functionality between amf_controller and the various api_controller methods would help resolve this to some extent. Well the fundamental reason for the performance change is rails and I don't think any amount of optimisation is going to help with that - it's basically because going through rails means doing work that could otherwise be avoided. I think even
Re: [OSM-talk] [OSM-dev] The future of Potlatch
In message [EMAIL PROTECTED] Richard Fairhurst [EMAIL PROTECTED] wrote: For most purposes AS3 probably is a better language - except for the fairly major proviso there's no open-source player even in development. As far as I'm concerned this is quite a key point, although I know that Steve disagrees with me violently on this ;-) In part it's an entirely selfish attitude in as much as that Adobe show no signs of wanting to support flash on 64 bit linux which means that I am left having to rely on the free players or struggling to use the 32 bit flash plugin via a kludgy wrapper that barely works at the best of times. So this isn't, as far as I'm concerned, primarily about some sort of ideological wish to use a free software solution, which is what Steve often seems to think this about. It's because the proprietary vendor has manifestly failed to provide for me, in the way which they often do because they will always concentrate on majority platforms. Tom -- Tom Hughes ([EMAIL PROTECTED]) http://www.compton.nu/ ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] [OSM-dev] The future of Potlatch
Tom Hughes wrote: Sent: 02 May 2008 9:51 AM To: [EMAIL PROTECTED]; talk@openstreetmap.org Subject: Re: [OSM-dev] The future of Potlatch In message [EMAIL PROTECTED] Richard Fairhurst [EMAIL PROTECTED] wrote: For most purposes AS3 probably is a better language - except for the fairly major proviso there's no open-source player even in development. As far as I'm concerned this is quite a key point, although I know that Steve disagrees with me violently on this ;-) In part it's an entirely selfish attitude in as much as that Adobe show no signs of wanting to support flash on 64 bit linux which means that I am left having to rely on the free players or struggling to use the 32 bit flash plugin via a kludgy wrapper that barely works at the best of times. I believe there is the temptation to think that the project should always pamper to all of us who have grown up with it thus far, afterall, if it changed significantly from our prior experience or expectations we would probably feel more and more left and out and ultimately move on to other things. I hope that doesn't happen for any of us. However, we can't avoid the observation that the project continues to grow exponentially and as it does so, and assuming it continues to do so, the number of potential contributors and users will probably increasingly come from non opensource backgrounds. More and more will be wanting to experience OSM from the standard ibm clone box running the big corporate software offerings. So, we shouldn't leave our heads in the sand, development on all fronts should be encouraged, open source or proprietary really doesn't matter, surely the community will decide what it wishes to use. The one bit I feel strongly should not be tampered with without widespread community agreement is what happens at the database end, and that includes what should and should not be available via the API (or any additional API's that get suggested). If we want to maintain true openness then we make it clear what the API can and can't do (we already do this), be responsive to new ideas for opening up the API and provide every assistance to those developing tools that use the API. Those tools, whether they are editors, viewers, routing products, mobile phones or whatever, should in my view be allowed to develop separately from the core OSM function. This means that if there are 10 editors out there we really shouldn't favour any particular one, but make all that comply with the requirements of the API available to users. Cheers Andy ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] [OSM-dev] The future of Potlatch
On Fri, May 02, 2008 at 10:25:50AM +0100, Dave Stubbs wrote: OTH I don't know much about AS3 so I can't say whether it's much better in this regard, but from a quick scan of it, I'd say it was. I think the main problem is the likely-hood of an opensource player being available for it. AS3 means flash 9 and better only, and currently the os stuff is way off that. I don't know whether that's likely to change any time soon. There is an official adobe 32bit linux version that works very well though, which will be good enough for most people. Just to give a counterweight (because you are the second person de-emphasizing the free tools argument) : I strongly value that openstreetmap not only provides 'free' data but also unencumbered tools to work with that data. Obviously the closer to the hearth of the project (and potlatch is very close to it), the more so. I'd try to be optimistic and keep developing -- raise the bar for any new editor as high as you can make it. While I have no opinion either way on the whole issue brought up in this thread, I can only agree with this. cu bart ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] [OSM-dev] The future of Potlatch
On Thu, May 1, 2008 at 6:35 PM, Richard Fairhurst [EMAIL PROTECTED] wrote: [warning - long ponderous e-mail follows!] Hi all, A fairly weighty issue concerning the future of Potlatch has arisen, and I'm completely baffled as to what to do - so I thought I'd ask the community for thoughts and advice. CloudMade (Steve and Nick's VC-funded company set up to commercialise OSM data, www.cloudmade.com) wants to commission a new online Flash editor for OSM. It would, I believe, probably be written by developers from Stamen Design (www.stamen.com): some of you will remember that Stamen's Tom Carden wrote OSM's early Java editing applet, and they've also written a slippy map in Flash called Modest Maps. As you can imagine, this has taken me aback a bit. As I understand it, their main issue is a technical one. Potlatch is written in ActionScript 1, which is the same language as JavaScript, but for Flash. The latest version is ActionScript 3, which is much more like Java for Flash. The end user doesn't notice a difference, but the programming style is very different. CloudMade believes this is holding back the development of OSM: that if the editor were written in the latest version of the language, more Flash designers would come to work on it, resulting in a better editor. Steve cites OSM's move from pure Ruby to Ruby on Rails as an example of how a contemporary language encourages more people to contribute. And they're also worried that if I were run over by a bus then no-one would be able to speak ActionScript 1 and maintain Potlatch. I'm not so sure. I think people are beginning to contribute code to Potlatch; that as essentially JavaScript it's approachable enough; and that the problems of attracting developers is symptomatic of core OSM in general (as per http://trac.openstreetmap.org/log/sites/rails_port). I hope that Potlatch, as something maintained by an active community participant _for_ the community, has demonstrated a pretty rapid rate of improvement anyway. It's meant to be small and compact, of course, not a a bells-and-whistles editor like JOSM: nonetheless, in the last few months, for example: it's become the only editor yet to offer revert/history, gained very good relations support, background layers, flexible GPX import, etc. And there's a lot of stuff on the way, mostly focusing on usability - from a generic 'undo' and pop-up help panel to a new, super-user-friendly tagging panel with draggable POI icons and things like that. It's got faults, everything has, but it's come a long way in the last year. For what it's worth I think it's the best thing I've ever coded. For most purposes AS3 probably is a better language - except for the fairly major proviso there's no open-source player even in development. Indeed, if I were starting all over again I'd probably do it in AS3, and in a couple of years I may well migrate Potlatch to AS3 (or 4, or whatever) anyway. But right now it's more important to spend time improving usability for mappers, given that - like most people here - I do have a full-time job which isn't OSM (which isn't computer-related at all, in fact) and consequently time is not unlimited. So I really don't know what to do. Part of me thinks that the most important thing is that Potlatch is still available and users are offered the choice. Part of me thinks, well, if there's going to be a new Flash editor, there's no point in me doing any development on Potlatch from today forward. Part of me wants to say well, screw you and walk away. And part of me wants to take CloudMade up on its OSM Grants scheme (http://blog.cloudmade.com/) and say, ok then, I'll announce a medium-term feature freeze, take a few weeks' holiday, learn AS3 and recode it for a large amount of $$$. I'm utterly stumped and would welcome suggestions. Thanks for reading. :) np :-) OK, first up potlatch's code: - ActionScript 1 in my professional opinion sucks donkey balls - how on earth you kept sane while writing Potlatch with that kind of lead weight I don't know, but Good Job! - there are occasions in the code where it could do with some redesign -- there's been lots of improvements along these lines made by yourself in the lead up to 8 and onwards. - having it's own API is a bit of a maintenance nightmare -- IMHO it should at least use rails models throughout to reduce the impact of this When it comes to attracting developers there is a problem here: the learning curve to start coding Potlatch is massive. You have to learn AS1, you have to learn how flash works in weird ways, you have to learn to avoid Ming bugs such as using return statements in the wrong place. And once you get into it, you end up going round in circles tearing your hair out because you missed one of those bugs or quirks. On top of all that you have to learn how Potlatch itself works (which is the easy bit). I can understand this will discourage many casual patches
Re: [OSM-talk] [OSM-dev] The future of Potlatch
On Fri, May 02, 2008 at 08:35:06AM +0100, Tom Hughes wrote: To summarise I think we both want the same thing, but you perhaps think somebody should just sit down and bang an AMF version of the current XML API and I'm happy with trying to incrementally move towards that position? Well, I don't think that's how I would put it. I think you were slightly oversimplifying when you just said It's a few lines of Rails object code. Some of the request methods in Potlatch are at least a bit more complex than that. getway_old, for example, is slightly more complex than that, as is putway. I won't pretend that I know nearly as much about the rails code as you do, but it seems like some of these would be better abstracted out. If that were the case -- that is, that all the Rails code on the site used the same underlying methods to talk to the database, given a 'fixed' API, and amf_controller was just about encoding the returned data into AMF -- then I thiink it would be possible to change the underlying slow methods into SQL (after proper profiling), because the main reason not to is 'maintaining two different codebases sucks', rather than 'no one likes SQL over rails'. The fact that Potlatch was using SQL was unfortunate, but for some problems, that could be the right solution. The fact that the API was *not* was what I would see as the real problem -- and changing the architecture so that that wouldn't happen for the same task seems valuable to me, or at least heading in that direction. Perhaps you were already heading there; I don't really know what the plans are for rails_port development, so I'm probably off my rocker :) Perhaps I'm wrong on this stuff: as I said, I'm new to the rails world. At this point, putting my code where my mouth is is probably the right choice, so until I sit down and hack on stuff, it's fine to ignore what I'm saying :) I'll see if I get time to have a play this weekend and improve my knowledge and understanding of the code, and maybe I'll find that you're right, and there's no reason to bother. Regards, -- Christopher Schmidt MetaCarta ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] [OSM-dev] The future of Potlatch
In message [EMAIL PROTECTED] Christopher Schmidt [EMAIL PROTECTED] wrote: On Fri, May 02, 2008 at 08:35:06AM +0100, Tom Hughes wrote: To summarise I think we both want the same thing, but you perhaps think somebody should just sit down and bang an AMF version of the current XML API and I'm happy with trying to incrementally move towards that position? Well, I don't think that's how I would put it. I think you were slightly oversimplifying when you just said It's a few lines of Rails object code. Some of the request methods in Potlatch are at least a bit more complex than that. getway_old, for example, is slightly more complex than that, as is putway. Neither of those methods has been converted to using rails yet - they are both still using raw SQL to do the work. Methods which have been converted are things like whichways and getpoi. I won't pretend that I know nearly as much about the rails code as you do, but it seems like some of these would be better abstracted out. If that were the case -- that is, that all the Rails code on the site used the same underlying methods to talk to the database, given a 'fixed' API, and amf_controller was just about encoding the returned data into AMF -- then I thiink it would be possible to change the underlying slow methods into SQL (after proper profiling), because the main reason not to is 'maintaining two different codebases sucks', rather than 'no one likes SQL over rails'. If I thought Steve would let me get away with doing raw SQL instead of using the rails object model I might have done so long ago ;-) Doing so would bypass all the integrity checks though, which is bad - that's a side effect of having the integrity checks in the wrong place (the rails object model rather than the database). Tom -- Tom Hughes ([EMAIL PROTECTED]) http://www.compton.nu/ ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] [OSM-dev] The future of Potlatch
Richard I'm sorry you think informal private chats are now in the public domain, I'll keep it in mind. All This is not quite what happened. For a start, this doesn't really have anything to do with CloudMade, it started a long time before that. It's about the maintainability and quality of potlatch. Some time ago when Nick and I decided to spend a week of our company time (as a consultancy as it was then, it was a strain on our time) I discovered how bad Richards API was. As a bit of background, there is the main API and Richard doesn't want to write against it so he wrote an entire Flash API in the Rails server which doesn't reuse any code, doesn't use any of the object models, writes raw SQL and is basically flawed. I, and any sane coder, thinks Potlatch should be using the main API. I thought I'd do what I know quite a bit about - the rails backend. I found it quite difficult to improve any of it and after a week I think I'd improved one or two things but not made much headway. Richard felt pretty personal about it and left IRC a few times. After that I tried to get other people interested in solving it, either the backend or potlatch itself. I found it very difficult to get anyone interested in the rails bit which is why I used to do all that boring stuff in the first place. As for the frontend, there was a fair amount of interest in hacking it until those I spoke to found out it is in an ancient version of ActionScript, doesn't use the API and requires a pile of bizarre libraries to compile it. Faced with not being able to interest anyone in the problems, I tried to solve them with Richard. I, personally, and with company funds offered basically everything I can think of under the sun to get him to let go of the stranglehold on the codebase and help others contribute to it. I offered to buy a copy of the latest flash compiler suit, whatever it is called. I offered to buy the books. I personally offered to ship him the rails books to learn how to write good backend code. I offered to send him on a course (so I'm disappointed how he's phrased wanting to now get a grant, as if this was never talked about). I offered to let him manage paid coders to improve it. I even offered to fly him out to california and meet people (there seem to be lots of good flash and interaction designers out there). Basically, Richard said no to all of it. All of this is just to improve potlatch (which most people I've spoken to think should be a complete rewrite). I tried very hard. Forgetting about all of that for a second - think about what happens if Richard loses interest or gets run over by a bus. Basically we're screwed as nobody else knows how to code against it. That's not true of the rails port - and the rails port was done for exactly that reason and has been moderately successful. If I or Tom get run over by busses, all is not lost. Faced with him being so stubborn, I talked to more people about what to do and figured that meeting in person might be helpful. So I travelled up to north of birmingham in my own time, bought the food and tried all of the above points. Andy Robinson came along as well. I got nowhere, Richard still feels aggrieved and stubborn. On a more positive note, all I'm trying to do is get more people coding potlatch and hence make it better. I'm also trying to get it to use a clean API. Potlatch is great, it's improved way beyond the applets that tom and I wrote, but it needs to get better. I've tried pretty hard every way I can think and the roadblock is Richard. We've now reached the point where Richard has rebuffed all offers of help, doesn't see the issues and wants to appeal to the community for support by taking private conversations public without warning and misrepresenting what I've been trying to do. So I'm open to any advice on what to do next. On 1 May 2008, at 18:35, Richard Fairhurst wrote: [warning - long ponderous e-mail follows!] Hi all, A fairly weighty issue concerning the future of Potlatch has arisen, and I'm completely baffled as to what to do - so I thought I'd ask the community for thoughts and advice. CloudMade (Steve and Nick's VC-funded company set up to commercialise OSM data, www.cloudmade.com) wants to commission a new online Flash editor for OSM. It would, I believe, probably be written by developers from Stamen Design (www.stamen.com): some of you will remember that Stamen's Tom Carden wrote OSM's early Java editing applet, and they've also written a slippy map in Flash called Modest Maps. As you can imagine, this has taken me aback a bit. As I understand it, their main issue is a technical one. Potlatch is written in ActionScript 1, which is the same language as JavaScript, but for Flash. The latest version is ActionScript 3, which is much more like Java for Flash. The end user doesn't notice a difference, but the
Re: [OSM-talk] [OSM-dev] The future of Potlatch
On Fri, May 02, 2008 at 12:27:38PM +0100, Tom Hughes wrote: I won't pretend that I know nearly as much about the rails code as you do, but it seems like some of these would be better abstracted out. If that were the case -- that is, that all the Rails code on the site used the same underlying methods to talk to the database, given a 'fixed' API, and amf_controller was just about encoding the returned data into AMF -- then I thiink it would be possible to change the underlying slow methods into SQL (after proper profiling), because the main reason not to is 'maintaining two different codebases sucks', rather than 'no one likes SQL over rails'. If I thought Steve would let me get away with doing raw SQL instead of using the rails object model I might have done so long ago ;-) Doing so would bypass all the integrity checks though, which is bad - that's a side effect of having the integrity checks in the wrong place (the rails object model rather than the database). Some things don't require referential integreity: selecting ways/nodes within a bounding box can't hurt the referential integrity of the database (so long as the code is well-maintained), so the harm in converting those methods (which are probably the single most performance important aspect of Potlatch?) to SQL is relatively low, so far as I can tell... Regards, -- Christopher Schmidt MetaCarta ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] [OSM-dev] The future of Potlatch
Thanks for some really helpful and interesting responses. (Thanks especially to Tom C for a very valuable perspective.) -- API The API has come up a lot. I've said before and will happily restate now that I think it would be great to get Potlatch talking Rails on the serverside, rather than the SQL at present. It wouldn't affect the Potlatch user experience (which is my primary concern), but would mean more peace-of-mind for Tom H, Steve et al; its benefits could be open to other editors; and it'd remove a major stumbling block when talking about Potlatch, which would be helpful. I don't have any emotional attachment to the code there at the moment - it's simply like that because it was actually developed before OSM moved to Rails (the pre-Rails API was a lot less sophisticated than the current one and really couldn't do what Potlatch needed), and rewriting it in Rails is something I'm not really qualified to do safely without breaking lots of stuff. One can always learn, and given unlimited time I'd like to; but sadly I don't currently have the time to learn AS3, _and_ Rails, _and_ respond to people's requests for Potlatch... oh, and do the day job! So given that the serverside code works, albeit inelegantly, learning to rewrite it hasn't been my priority. I'm very happy, of course, to spend as much time as necessary talking others through how it currently works should they be kind enough to take on the task of rewriting it - I've already documented a bunch of it for Steve. I'm pretty sure that'd be more efficient than me blundering in and writing some very bad Rails code. There are really only two provisos to this and neither's a big one, I think. I do feel fairly strongly that Potlatch, as currently written, and the API should continue to talk AMF to each other rather than XML; it keeps the code compact and fast, especially given that AS1 doesn't have great XML handling. But that's just a transport format, really, just 50 or so lines of (pretty reliable and fast) code. The other one is that it would be good if the getway call, in particular, could exist in _both_ SQL and Rails forms in the code, with a switch called RICHARD_NOT_HIT_BY_BUS_YET to determine which one is called. Up until the time of the bus incident we could use the (very, very simple, read-only) single SQL call because it's between two and four times faster, and this is the one call that does have a significant performance impact on Potlatch. After I'm run over, you could switch to Rails, which means a performance hit but may be considered more maintainable. Please don't treat this as an invitation to drive a bus at me. Pretty much everything I'd want to say on this has already been expressed by TomH: http://lists.openstreetmap.org/pipermail/talk/2008-May/025886.html http://lists.openstreetmap.org/pipermail/talk/2008-May/025889.html -- Frederik's uniqueness point Really can't argue with that. Going back to the Java applet days I did actually want both it and Potlatch to be available at the same time (http://wiki.openstreetmap.org/index.php/ Talk:Java_Applet_Development); unfortunately this never happened. But as long as the choice is presented through a friendly UI and doesn't confuse novices, it's a good principle. On the openness (or otherwise) of Potlatch, Frederik, I think I mentioned to you in the internationalisation discussion that I'm planning to build a preset tagging system that will let people contribute their own plug-in tagging panels. Otherwise Potlatch's only impact on mapping practices has probably been to hasten the demise of segments which I suspect you agreed with! -- AS1 / AS3 Dave - I think your definition of donkey balls might be different to mine. ;) Or rather, when you've been sucking horse balls for several years then donkey balls don't seem very different. Er, I should probably rephrase that. To me (coming from a Perl script kiddie background, rather than Proper Programming In Java) AS1 is the nice approachable stuff while AS3 is scary with lots of stuff you have to write that doesn't actually do anything. I'm not holding that up as a truth, I don't doubt that AS3's objectively the better language by current norms: it's just that it's an utterly new way of doing things for me, and I can't immediately see the payback for me in spending a long while learning it, then rewriting all 7,000 lines of code. AS3 doesn't yet _do_ much that AS1 doesn't: some clever typography stuff (not needed for Potlatch), faster (but Potlatch's UI is plenty fast, it's the server responses that slow the UI), and better XML handling (but as written Potlatch doesn't need that). And it doesn't make me more marketable as a programmer because I'm not a professional programmer. Now this is my problem, of course, not something anyone else needs to worry about. But it's the thing that I'm still most unsure
Re: [OSM-talk] [OSM-dev] The future of Potlatch
-- AS1 / AS3 Dave - I think your definition of donkey balls might be different to mine. ;) Or rather, when you've been sucking horse balls for several years then donkey balls don't seem very different. Er, I should probably rephrase that. Yeah, I don't think the relative merits of various equine testicles is really where I want the conversation to go... To me (coming from a Perl script kiddie background, rather than Proper Programming In Java) You'll be glad to know I have opinions on Perl as well :-). I was once told that it's possible to write a program to generate haikus in perl, where the program itself is in fact a haiku. I was told this as if it was a good thing. To me it merely sounds evil. I found programming actionscript almost as painful as using C again. Dave ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] [OSM-dev] The future of Potlatch
From: Tom Hughes [EMAIL PROTECTED] In part it's an entirely selfish attitude in as much as that Adobe show no signs of wanting to support flash on 64 bit linux which means that I am left having to rely on the free players or struggling to use the 32 bit flash plugin via a kludgy wrapper that barely works at the best of times. Adobe has dropped the licensing restrictions on building competing flash players, and will be publishing the spec. http://blog.wired.com/monkeybites/2008/04/adobe-drops-lic.html So no open source solution yet for Flash 9, but there's no legal impediments any longer. -Mikel___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] [OSM-dev] The future of Potlatch
Hi, [warning - long ponderous e-mail follows!] I don't really see much of a problem with mutliple Flash editors being available. If they get something done that's better than Potlatch, why then it's good for all is it not? In general I have a problem with the built-in uniqueness of Potlatch or anything that might replace it; Potlatch is the OSM editor, the one that comes up if you click edit on the main OSM page. Until now Potlatch has enjoyed this privilege because there was no other suitable editor. If there are many competing editors - which one will get the pole position on osm.org? The editor and the features it supports will always very strongly influence what people map and how they do it. With JOSM, the concept is reasonably open; anyone can compile and run their own JOSM and make it available to third parties (as anyone using JOSM will not have difficulties running a program obtained from somewhere else), and plugins can be written and distributed. So even if JOSM were the only editor around, its grip on what people map and how they do it would not be very strong, since anybody with, say, an interest in turn restriction relations could provide a plugin that would make them editable nicely. Correct me if I'm wrong but Potlatch is, and any AS3 replacemend would be, a much more controlled environment, where the user has much less say in what editor and what features they want - you cannot simply run your own version of the editor against the live API, nor can you have local plugins or so. So that's the one doubt I would have - it must not come to CloudMade dictating what's behind the Edit tab on openstreetmap.org. But that could easily be alleviated by creating a set-up where *anybody* can operate an OSM editor on their site. In fact I would prefer that even now; give the user a choice whether his edit tab leads to Potlatch on RichardF's host or to the AS3 flash editor at cloudmade.com or the 100% javascript editor from crschmidt (or maybe even the JOSM applet), and anybody who thinks their editor is even better may add it to the list. If this core issue is resolved - i.e. the osm.org web site does not specifically endorse one editor and fail to list others - then I'd really say competition is good for all of us. I could imagine that if anything suppresses creativity in the current setup, it is not the fact that you use AS1; it is the fact that anything anybody wants to do has to be run by you and/or the people with [EMAIL PROTECTED] and only if you say yes then it gets into the editor, whereas with a project that doesn't depend on integration with the osm.org site (like Merkaartor or JOSM), you can simply try out something and offer it to people for immediate live use, even if the maintainer has completely different plans and doesn't like the concept. (But then again, look at the one-digit number of developers even with JOSM and Merkaartor... so don't excpect a boost if you make Potlatch less monopolistic.) Part of me thinks that the most important thing is that Potlatch is still available and users are offered the choice. Part of me thinks, well, if there's going to be a new Flash editor, there's no point in me doing any development on Potlatch from today forward. Part of me wants to say well, screw you and walk away. And part of me wants to take CloudMade up on its OSM Grants scheme (http://blog.cloudmade.com/) and say, ok then, I'll announce a medium-term feature freeze, take a few weeks' holiday, learn AS3 and recode it for a large amount of $$$. I'm utterly stumped and would welcome suggestions. As an OSM community member I would like you to continue with your work and someone else do an AS3 editor for CloudMade, and then I want them both available on osm.org so that I can choose the one I like best (or the one that runs with my hardware/OS best, ot whatever). If you stopped working on Potlatch altogether, or if you re-did Potlatch in AS3, then that would leave me (the user) with less choice. Diversity is good. If I were in your shoes however, I might go for the $$$. $$$ is also good. Bye Frederik -- Frederik Ramm ## eMail [EMAIL PROTECTED] ## N49°00'09 E008°23'33 ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] [OSM-dev] The future of Potlatch
2008/5/1 Richard Fairhurst [EMAIL PROTECTED]: CloudMade (Steve and Nick's VC-funded company set up to commercialise OSM data, www.cloudmade.com) wants to commission a new online Flash editor for OSM. It would, I believe, probably be written by developers from Stamen Design (www.stamen.com): some of you will remember that Stamen's Tom Carden wrote OSM's early Java editing applet, and they've also written a slippy map in Flash called Modest Maps. I'll be up front here - I have spoken informally to Steve about the potential of us working on this, but nothing formal has been arranged. I'm not sure whether Stamen would take this project on, or what our availability is, or any of that - I only speak for myself here and not the company. If Stamen did work on it, there'd be no guarantee that I would be personally involved. Steve (or anyone else you've talked to) shouldn't be talking about a potential business relationship that doesn't exist yet, nor should they be holding it over you as a (dubious) incentive to improve Potlatch to their liking. I'm sorry if that's what's happened. As you can imagine, this has taken me aback a bit. As I understand it, their main issue is a technical one. Potlatch is written in ActionScript 1, which is the same language as JavaScript, but for Flash. The latest version is ActionScript 3, which is much more like Java for Flash. The end user doesn't notice a difference, but the programming style is very different. CloudMade believes this is holding back the development of OSM: that if the editor were written in the latest version of the language, more Flash designers would come to work on it, resulting in a better editor. Steve cites OSM's move from pure Ruby to Ruby on Rails as an example of how a contemporary language encourages more people to contribute. And they're also worried that if I were run over by a bus then no-one would be able to speak ActionScript 1 and maintain Potlatch. Since I'm here, I will offer my own opinion about perceived hurdles to developing Potlatch. I'm aware that this might be an inappropriate place to do so, given that I'm somehow implicated in your taken-aback-ness, above, but I think it's worth adding... I think the fact that it has its own API is a much bigger concern than it being written in AS 1.0 is. If Potlatch was using the main API, development of API-backed features in Potlatch could be shared by other editors too. Any tests written for the API would help Potlatch. Any changes to the schema would only have to happen once. etc. etc. I'm not so sure. I think people are beginning to contribute code to Potlatch; that as essentially JavaScript it's approachable enough; and that the problems of attracting developers is symptomatic of core OSM in general (as per http://trac.openstreetmap.org/log/sites/rails_port). I hope that Potlatch, as something maintained by an active community participant _for_ the community, has demonstrated a pretty rapid rate of improvement anyway. It's meant to be small and compact, of course, not a a bells-and-whistles editor like JOSM: nonetheless, in the last few months, for example: it's become the only editor yet to offer revert/history, gained very good relations support, background layers, flexible GPX import, etc. And there's a lot of stuff on the way, mostly focusing on usability - from a generic 'undo' and pop-up help panel to a new, super-user-friendly tagging panel with draggable POI icons and things like that. It's got faults, everything has, but it's come a long way in the last year. For what it's worth I think it's the best thing I've ever coded. It's certainly leaps and bounds ahead of the applet I wrote, I have no problem admitting that! For most purposes AS3 probably is a better language - except for the fairly major proviso there's no open-source player even in development. Indeed, if I were starting all over again I'd probably do it in AS3, and in a couple of years I may well migrate Potlatch to AS3 (or 4, or whatever) anyway. But right now it's more important to spend time improving usability for mappers, given that - like most people here - I do have a full-time job which isn't OSM (which isn't computer-related at all, in fact) and consequently time is not unlimited. You're right I think. API aside, the main reason to move to AS3 would be speed, the availability of more libraries and better development tools, and the larger developer community that Couldmade cites. Certainly though, the lack of a solid open source platform to run AS3 code on is an issue for a project like OSM. Not so much for a business like Cloudmade though, I expect. I think the rest of this thread has already covered the fact that there has to be room for more than one editor in the OSM eco-system (and I point people to OAuth as a way to open that up), but that the Edit tab on OSM should be given to whichever editor suits
Re: [OSM-talk] [OSM-dev] The future of Potlatch
In message [EMAIL PROTECTED] Tom Carden [EMAIL PROTECTED] wrote: I think the fact that it has its own API is a much bigger concern than it being written in AS 1.0 is. If Potlatch was using the main API, development of API-backed features in Potlatch could be shared by other editors too. Any tests written for the API would help Potlatch. Any changes to the schema would only have to happen once. etc. etc. I think most of us (with the possible exception of Richard) would agree here. Well actually I don't mind the existence of the AMF API as such so long as it is just concerned with decoding the RPC calls and encoding the results, and it uses the rails object model to do all the work so that important code is shared with the XML API. We have in fact started moving towards that goal with some work that Steve did on some of the AMF calls. The problem is that it turned out that, even after I had optimised the code, it is significantly slower to go through the rails object model that to make direct SQL queries. Of course that probably affects the main API as well, it's just that it doesn't appear as a regression in that case in the way that it does with the AMF API. I think even Richard wouldn't mind too much making the AMF API use the rails object model if it wasn't for the performance issues. Tom -- Tom Hughes ([EMAIL PROTECTED]) http://www.compton.nu/ ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] [OSM-dev] The future of Potlatch
On Fri, May 02, 2008 at 12:24:35AM +0100, Tom Hughes wrote: In message [EMAIL PROTECTED] Tom Carden [EMAIL PROTECTED] wrote: I think the fact that it has its own API is a much bigger concern than it being written in AS 1.0 is. If Potlatch was using the main API, development of API-backed features in Potlatch could be shared by other editors too. Any tests written for the API would help Potlatch. Any changes to the schema would only have to happen once. etc. etc. I think most of us (with the possible exception of Richard) would agree here. Well actually I don't mind the existence of the AMF API as such so long as it is just concerned with decoding the RPC calls and encoding the results, and it uses the rails object model to do all the work so that important code is shared with the XML API. There's an important difference between this and Potlatch using the main API. There are, I believe, some things that Potlatch can do that no other client can do, because the API to do it is not available. Specifically: * whichways_deleted gets deleted ways in a bbox, which the main API doesn't provide * getway_old has a last deleted version, which seems different More problematic than the specific methods which do or don't exist, however, is the fact that the Potlatch way of interacting with them is *entirely* different -- so when a bug is fixed in the main API, the fix has to be duplicted in the Potlatch/amf_controller. This affects development of the server side APIs. You, at one point, put forward a significant amount of effort in improving the Rails code in amf_controller: that work benefited only Potlatch. If Potlatch was using the same backend rails calls (rather than just objects/models), then that development time could have benefited other clients as well. Potlatch has a very different way of interacting with the API. This way of interacting with the API has some benefits, and could allow for other clients similar to Potlatch to be developed without being tied explicitly to amf_controller, which is (at least at the moment) essentially Potlatch specific. (Things like 'masterscale' and 'potlatch_lon' seem to be indicate this anyway: maybe I'm wrong here.) I don't know amf: I see from the code that it's Actionscript Message Format. If there is a desire for supporting Actionscript Message Format for API actions, I don't see a problem with that at all: In fact, other encodings of OSM data delivered via the API seem a natural progression to me. (I'm biased here, having done this with TileCache, FeatureServer, WPServer, etc.) But I don't see that that means that Potlatch should have access to API methods which simply don't exist to any not potlatch-clients (or that clients would have to speak the same language as Potlatch to use them). We have in fact started moving towards that goal with some work that Steve did on some of the AMF calls. This is a different goal, that of replacing SQL with rails objects. A valuable goal, but not the same as abstracting the data selection calls to a single set of code, and then using that code to serve both the main API and amf_controller. The problem is that it turned out that, even after I had optimised the code, it is significantly slower to go through the rails object model that to make direct SQL queries. This is interesting, but (imho) less relevant: The value of the above is actually doubled by this statement, since if Potlatch and the API were using the same function, one could optimize this, and since the code is used by *all* clients (not just one) you would see any problems with SQL calls more generally, and perhaps find it easier to debug them. (I think this is compounded by the fact that AMF is a format which is hard to replicate outside of a flash client, which makes server side developers less likely to have the technical chops to debug them.) A common library for API functionality between amf_controller and the various api_controller methods would help resolve this to some extent. I think even Richard wouldn't mind too much making the AMF API use the rails object model if it wasn't for the performance issues. Using the Rails Object Model is solving an *internal* API issue, but there still continue to exist two external APIs that use different backend data manipulation to achieve their goals. This seperation is what I see as more problematic -- and I think that's what Tom was addressing as well, though I could be wrong. Regards, -- Christopher Schmidt MetaCarta ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
