Re: [OSM-dev] Andy Allan joining web site maintainers

[Andy Allan]

On 10 July 2017 at 13:25, Matthijs Melissen  wrote:
> On 10 July 2017 at 11:14, Andy Allan  wrote:
>> Thanks Tom. My intentions for the next few months are to continue to
>> do whatever I can to encourage new contributors. I've done a lot of
>> refactoring recently which will help whoever tries to make code
>> changes, but there's other things I think I can do too.
>
> Currently, the openstreetmap-website is both responsible for the
> openstreetmap.org web application, and for the API.
>
> How hard would it be to separate these two functions, perhaps even
> into two projects?

It's not hard, but I personally think it's not worthwhile. But as I
said in my response to Frederik, opinions vary and other core OSM
developers will disagree with me. I'll give some counterpoints to your
statements:

> Would there be any advantages in doing so? Intuitively both functions
> seem to be separate projects.

I don't see them as being different at all. If you want to "view a
way" or "view a diary entry", then there's a choice of output formats
- XML, html or json (in the future, perhaps). So one app can easily
have three different views, rather than having an entirely different
application just to change the display format.

Our API currently covers only a small part of the function of the
website. In future we could allow you to post diary entries via the
API, or add new friends, or get a list of comments on your diary
entries. Again, it would seem strange to me to have a rails app that
lets you view your list of friends as a webpage, but then need a
different app running to see that same list in json or XML.

> I could imagine having two smaller
> applications might make maintenance easier.

Or it would spread the developers thinner, and lead to a lot of
duplicated effort.

> It would also mean we
> could, for example phase Ruby out for the app but not for the api or
> vice versa.

I'd rather phase out low-level code like C++! Even better, we can also
phase out home-grown ruby code and use third-party libraries much more
often. There's very little about what we do that needs anything
special, beyond the map call. Most of OpenStreetMap is completely
routine things like signing up, resetting passwords, CRUD on simple
objects and RESTful API calls. And we don't even do much of that -
we're not dealing with millions of requests per second or anything
like that. So the less code we have of our own to maintain, the
better, and the higher-level language it can be written in the better,
as far as I'm concerned.

> Lastly, making the website make only use of well-defined
> api endpoints instead of interface with the database directly might
> also makes things clearer.

I've said similar things in the past, but I realise different
developers have said similar meaning slightly different things. For
me, that means that we should unify the internals of the rails port
(e.g. not having a separate "BrowseController" for viewing a way, and
a different "WayController" for viewing a way in XML). For other
developers it means changing the rails port to make API calls (e.g.
BrowseController would fire off HTTP calls). For other other
developers it means turning the browse html pages into a javascript
SPA that fire off http calls to the API for the data it needs. I'm not
in favour of 2 and 3.

> Have there been any thoughts in this direction?

There's been many thoughts over the years, but nowhere near enough
implementation!

Thanks,
Andy

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Andy Allan joining web site maintainers

[Andy Allan]

On 10 July 2017 at 10:48, Frederik Ramm  wrote:
> Hi,
>
> On 10.07.2017 11:14, Andy Allan wrote:
>> My intentions for the next few months are to continue to
>> do whatever I can to encourage new contributors.
>
> I think it would be helpful for new contributors if the following two
> points could be explained:
>
> * What kinds of changes to the API are acceptable while still being "API
> 0.6"? Is anything that adds new API calls automatically on hold until we
> do "0.7", or should we just refrain from breaking existing API calls?

As far as I'm concerned, adding new API calls to the non-core parts of
the API are fine. So e.g. DiaryEntries or Notes but avoiding
nodes/ways/relations/changesets etc.

For those, I'm also personally fine with adding new APIs, but I know
that others prefer for them to go into cgimap at the same time. But
see also below.

For 0.6 there should be no breakage of existing API calls, of course.
I know people expect big things from 0.7, but I'd be happy to see a
0.7 release that doesn't change any data models, but fixes the API.
The API for notes is pretty bad, for example[1]. We could have 0.7 as
an interim API that runs in parallel with 0.6 for a year or two, and
save the big changes for 0.8.

> Maybe this is even on some mental roadmap somewhere - I heard people
> talking about making the web site actually *use* API calls, rather than
> accessing the database directly.

I have many mental roadmaps, but there's a line to be drawn between
long-term personal roadmaps and astronautism :-)

> * What is the relationship between "cgimap" and the web site, and in how
> far are contributions that touch an area handled by "cgimap" required to
> cover both the C++ and the Rails implementation?

Currently they work in parallel. So everything that is in cgimap has a
corresponding ruby version that's not used in production for OSMF (but
is probably used elsewhere in production). More calls are gradually
being added to cgimap - originally it only handled the map call, hence
the name. You can get a feel for the current state by looking at which
calls are passed to cgimap[2] by apache, the rest are handled by
openstreetmap-website.

I would like to change this situation, by using the cgimap-ruby
gem[3]. This would allow us to use cgimap within
openstreetmap-website, and get rid of the corresponding ruby code.
This would be useful for development and self-deployments, even if
those paths are still bypassed in production and still handled by a
standalone cgimap process. It would also mean anything added to cgimap
wouldn't need a corresponding ruby version. It would also mean that
there is less pressure to add more and more API endpoints to cgimap,
and instead only add those that are necessary from a performance point
of view.

There is a slight difference of opinion on whether we should use
cgimap for every API call at some point in the future. I don't think
it's worthwhile and I think the C++ code should only be for
performance-critical things, but opinions vary.

> One reason why this is of interest to me is that I'm still very much
> interested in being able to access deleted objects through the API and
> through the web site. I've made a few half-baked attemtps at
> implementing that in the past
> (https://github.com/woodpeck/openstreetmap-cgimap/tree/deleted_call,
> https://github.com/woodpeck/openstreetmap-website/tree/browse-deleted-objects)
> and would appreciate guidance of how to "do this right", if at all
> possible within the constraints of "0.6".

So there's already the AMF API, as I know you're aware but others
might not be. For the deleted items map call, it's apparently very
inefficient, and there have been plans to "do it properly" for as long
as I can remember. However, I don't see why porting the deleted items
map call to to the XML API should have to wait indefinitely, so I'd
definitely support that. Not least because we could then consider
turning off the AMF api, which is awesome but gnarly code to maintain.

Thanks,
Andy

[1] "POST /api/0.6/notes/addPOIexec" for example, see
https://github.com/openstreetmap/openstreetmap-website/blob/39c5d8caa71b29d70bb136d6e0a560426e089112/config/routes.rb#L102-L106
[2] 
https://github.com/openstreetmap/chef/blob/7720e35c56ebf155de90facca5a421e43974774b/cookbooks/web/templates/default/apache.backend.erb#L66-L75
[3] https://github.com/zerebubuth/cgimap-ruby

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Andy Allan joining web site maintainers

[Tom Hughes]

On 10/07/17 13:25, Matthijs Melissen wrote:

On 10 July 2017 at 11:14, Andy Allan  wrote:

Thanks Tom. My intentions for the next few months are to continue to
do whatever I can to encourage new contributors. I've done a lot of
refactoring recently which will help whoever tries to make code
changes, but there's other things I think I can do too.


Currently, the openstreetmap-website is both responsible for the
openstreetmap.org web application, and for the API.

How hard would it be to separate these two functions, perhaps even
into two projects?


That's exactly what we're working on with cgimap.

Tom

--
Tom Hughes (t...@compton.nu)
http://compton.nu/

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Andy Allan joining web site maintainers

[Matthijs Melissen]

On 10 July 2017 at 11:14, Andy Allan  wrote:
> Thanks Tom. My intentions for the next few months are to continue to
> do whatever I can to encourage new contributors. I've done a lot of
> refactoring recently which will help whoever tries to make code
> changes, but there's other things I think I can do too.

Currently, the openstreetmap-website is both responsible for the
openstreetmap.org web application, and for the API.

How hard would it be to separate these two functions, perhaps even
into two projects?

Would there be any advantages in doing so? Intuitively both functions
seem to be separate projects. I could imagine having two smaller
applications might make maintenance easier. It would also mean we
could, for example phase Ruby out for the app but not for the api or
vice versa. Lastly, making the website make only use of well-defined
api endpoints instead of interface with the database directly might
also makes things clearer.

Have there been any thoughts in this direction?

-- Matthijs

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Andy Allan joining web site maintainers

[Martin Koppenhoefer]

sent from a phone

> On 10. Jul 2017, at 11:48, Frederik Ramm  wrote:
> 
> It could be a low-hanging fruit
> to create feature parity between website and API at least in some areas.


+1, few people know you can request specific versions like the previous one, 
with big objects often running in timeout for the full history it would be 
convenient to offer a link to the previous version rather than the full history 
or after a timeout occurred.

Cheers,
Martin 
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Andy Allan joining web site maintainers

[Frederik Ramm]

Hi,

On 10.07.2017 11:14, Andy Allan wrote:
> My intentions for the next few months are to continue to
> do whatever I can to encourage new contributors. 

I think it would be helpful for new contributors if the following two
points could be explained:

* What kinds of changes to the API are acceptable while still being "API
0.6"? Is anything that adds new API calls automatically on hold until we
do "0.7", or should we just refrain from breaking existing API calls?

One reason I'm asking this is that there's a bunch of things that have
an API call but are not accessible through the web site (e.g. "show me a
specific version of this object" - website has just full history or
latest), and vice versa (web site can show all changesets by a given
user, but no API call exists for that). It could be a low-hanging fruit
to create feature parity between website and API at least in some areas.
Maybe this is even on some mental roadmap somewhere - I heard people
talking about making the web site actually *use* API calls, rather than
accessing the database directly.

* What is the relationship between "cgimap" and the web site, and in how
far are contributions that touch an area handled by "cgimap" required to
cover both the C++ and the Rails implementation?

One reason why this is of interest to me is that I'm still very much
interested in being able to access deleted objects through the API and
through the web site. I've made a few half-baked attemtps at
implementing that in the past
(https://github.com/woodpeck/openstreetmap-cgimap/tree/deleted_call,
https://github.com/woodpeck/openstreetmap-website/tree/browse-deleted-objects)
and would appreciate guidance of how to "do this right", if at all
possible within the constraints of "0.6".

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Andy Allan joining web site maintainers

[Andy Allan]

On 22 June 2017 at 21:15, Tom Hughes  wrote:
> I'm pleased to be able to invite you all to join me in welcoming Andy Allan
> as the new co-maintainer on the code base for the OpenStreetMap web site.
>
> Going forward we intend to look to add additional maintainers to further
> diversify and spread the load, so we will be actively looking for candidates
> among those working on the code base.
>
> Please continue to open issues and pull requests on GitHub as before.

Thanks Tom. My intentions for the next few months are to continue to
do whatever I can to encourage new contributors. I've done a lot of
refactoring recently which will help whoever tries to make code
changes, but there's other things I think I can do too. So it will
likely be a combination of:

* My own refactoring priorities, which will pay off for developers in
the long term but aren't very visible to most people.
* Working through the PR and issue backlogs, to try and help some
straightforward things get "over the line" - hopefully motivating for
developers.
* Work on a few end-user facing tasks, so that we're making visible progress.

Thanks,
Andy

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

[OSM-dev] Andy Allan joining web site maintainers

[Tom Hughes]

I'm pleased to be able to invite you all to join me in welcoming Andy 
Allan as the new co-maintainer on the code base for the OpenStreetMap 
web site.


Going forward we intend to look to add additional maintainers to further 
diversify and spread the load, so we will be actively looking for 
candidates among those working on the code base.


Please continue to open issues and pull requests on GitHub as before.

Tom

--
Tom Hughes (t...@compton.nu)
http://compton.nu/

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev