Re: [O] Using CEDET modules from Emacs core
>> Which CEDET features would we want to use from core? > > For one, I'd like to see more major modes come with support for Semantic > right in the major mode's own definition (rather than have it part of > CEDET). E.g. for Elisp mode, CC-mode, ... > > The idea is to get to the point where Semantic support is just another > thing that a major mode should aim to support alongside syntax-tables, > indentation, font-lock, outline-minor-mode, ... This sounds like a great idea! Semantic appears to be to be stable enough that we might want to consider extracting it from CEDET in core like EIEIO was. Perhaps it's worth considering this line of thought: that are parts of CEDET which are worthy of becoming part of Emacs proper. As Stefan said, semantic is a perfect example of something which built in modes could benefit from. There are other parts of CEDET which I don't think meet the criteria of being stable and general enough that they should be considered for this. Such as EDE and the external databases for semantic db. All of which are useful, but unstable (and sometimes very slow) and I feel like they shouldn't be expected as part of the core editing experience -- i.e. that one should have to buy into their use.
Re: [O] Sync up the org in emacs master to org maint branch?
>> - Suppose that Emacs 22.0 is the current release and Emacs 22.1 is then >>released; CEDET is at >> - we update a registry somewhere indicating that Emacs 22.0 works with >> and 22.1 works with >> >> - When we make updates to CEDET we mark 22.1 as working with >> but we don't change that reference >>for 22.0 (or any older versions) >> - When someone complains that there's a bug in CEDET for 22.0 we >>indicate that it's no longer supported and that they should update >>Emacs to receive updates >> >> This process would almost be the same as what you get just by bundling >> CEDET with Emacs except that: >> >> - You can get the latest CEDET *if* you have the latest Emacs > > No. We have two branches: emacs-25 and master. The CEDET from master > will usually not work on any 25.x version. In the process which I proposed we would be developing against the most recently released Emacs. I now see that there's a trade off. If we were to go with my scheme then users would have access to the latest version, but we would: 1. miss out on the new features being developed on the Emacs master branch (which CEDET stands to gain a lot from); 2. have to endure a difficult and error prone release process every time Emacs is released because that's not what all the testing is done against; If we go ahead and merge it in then we would be able to benefit from all the new Emacs features and the release would be less painful. Of course our users would still have to wait for the latest features, but at least it would be worth the wait because we would be able to consider using features like threading. >> - The version of CEDET for any particular version of Emacs is as far as >>CEDET got before the next release of Emacs came out >> >> If this is what you were thinking of then please could you elaborate on >> what ended up being the problem which added more work. > > First off, CEDET is currently *not* a package, although that notion gets > thrown around a lot. It is scattered across the Emacs code base: > lisp/cedet, admin/grammars, etc/srecode, documentation, and test > suite. All this needs to be packaged in a way so that it can be > integrated into Emacs during a normal checkout. It needs to build and > test in such a normal checkout, but also separately when installed from > ELPA, including grammar compilation. And you need this twice: one for > emacs-25, one for master, with the possiblity to merge between the two. Yes, this would be a pain. I'm in favour of Phillip's approach in which the packages are self contained somewhere in the Emacs source tree. This would eliminate the need for a complicated copying process and eliminate the problem of where things go and what files should be updated. No other folder would be touched because a package contains it's own source, documentation etc. Unfortunately the idea isn't gaining a lot of traction. I'm not satisfied that the alternative would make things easier for anyone because, as you say, it would need to describe a complex copying process which intertwines CEDET with various parts of Emacs. > Then there's this "registry". No one has said how that should > work. "Submodules/Subtrees" are *not* a sufficient answer, they are just > tools. People will need to say how the *workflow* should be, including > such things like merges from stable, ChangeLog generation, AUTHORS, > NEWS, creating release tarballs, and so on. Once someone has written > this down *in detail*, we can discuss again if this indeed will make > things simpler and reduce our workload. I haven't heard the registry mentioned before. I mentioned it as a detail required by my process for supporting multiple versions, but I'm beginning to think that it's the wrong line of thought to pursue. >>> Well, we cannot really discuss this since there's no real plan how this >>> all should work. I can only speak from experience. >> >> We can still put ideas forward though. Haven't come up with anything >> myself yet though. > > Yes, you can, but it has a cost. Once again, the CEDET merge is stalled, > and we spend our time writing mails. I find this situation incredibly > frustrating. In lieu of any input from others the best we have is Phillip's idea of using Elpa. That solves how the files are copied in and compiled. >>> How does CEDET, Gnus and Org affect the rest of Emacs? They strongly >>> depend on Emacs "core" (whatever exactly that is), not the other way >>> round. >> >> I believe that one of the intentions of the move is to enforce this so >> we can't build bad dependencies -- am I wrong? > > I think other modes should use Semantic. Agreed. I mentioned this in my response to Stefan's email. Shouldn't we then consider moving it out of CEDET in Emacs? >> Perhaps I'm wrong, but to my mind the package approach would afford us >> with more testing before we get to the point of another release of >> Emacs. > > If you develop on Emacs 'master', you can u
Re: [O] Sync up the org in emacs master to org maint branch?
>>> they are extremely dependend on >>> many obscure Emacs internals (not sure about Org). >> >> Shouldn't we be trying to avoid this situation? It's sure to come back >> and bite us in the future. If we continue to develop a package >> (wherever it ends up being developed) which is so tightly coupled that >> any kind of re factoring in core becomes a nightmare, because we have to >> consider the umpteen ways in which it'll break the package, then surely >> that's a bad methodology to continue? > > I don't know what you have in mind. All I can say is: CEDET couldn't do > what it does if we'd restrict ourselves to some subset of Emacs. In particular I was worried by the phrasing "extremely dependent". I agree that in order to make a system like CEDET without re-inventing the wheel and without running into performance problems it's necessary to depend on more primitive parts of Emacs. Perhaps we can improve the relationship(?) Perhaps this is a discussion to be had when all of this is done though. >> I feel like this problem isn't intractable. > > I didn't say it's intractable. I just said it means more work for me. > >> We can mark some state of CEDET as being stable under the various >> versions of Emacs (because it was at the time) and then only support >> the current release for the latest package. This would most likely >> require changes to package to ensure that you get an appropriate >> version of the package when you download it. > > As I said: we did that. It was a huge amount of work. Please understand > where I'm coming from: if you look through the CEDET history, you will > see that in the past few years I almost exclusively did infrastructure > work and no real coding. All I can say is: *I* won't do this anymore, > and I don't want to be part in something which will increase grunt > work. We did not make this decision lightly. But with the amount of > developers we have, I'm convinced it's the right thing to do. Fair enough. I don't have the experience here. It just seems like we could meet both goals without increasing the work load in this regard. To be clear I agree that, whatever decision this discussion arrives at, we definitely don't want to we waste the time of any developer with grunt work. Continuing this line of thought. I'm not sure that we're thinking of the same process here. What I'm suggesting is as follows: - Suppose that Emacs 22.0 is the current release and Emacs 22.1 is then released; CEDET is at - we update a registry somewhere indicating that Emacs 22.0 works with and 22.1 works with - When we make updates to CEDET we mark 22.1 as working with but we don't change that reference for 22.0 (or any older versions) - When someone complains that there's a bug in CEDET for 22.0 we indicate that it's no longer supported and that they should update Emacs to receive updates This process would almost be the same as what you get just by bundling CEDET with Emacs except that: - You can get the latest CEDET *if* you have the latest Emacs - The version of CEDET for any particular version of Emacs is as far as CEDET got before the next release of Emacs came out If this is what you were thinking of then please could you elaborate on what ended up being the problem which added more work. Also, would this be a problem for our users? i.e. would we be inundated with emails requesting continued support on older versions or would users generally accept this kind of change. I mostly work on back end systems so I don't have a good feeling for how this would go down with users (I would find this reasonable as a user). > Bug fixes can go with point release, which we should have every > year. But yes, if you want the latest, greatest and buggiest, you'll > have to use Emacs master. But that goes for a lot of Emacs features, so > I don't see why it's particularly bad for CEDET. I feel like there are aspects of CEDET which are still under development. Perhaps I'm just unlucky in my particular usage patterns of upstream and the way things are going this will be fine (with the un-merged parts going into packages.) >> I'm interested in exploring more with regards to how the subtree >> approach would work. In particular how it would impact the Makefiles >> and build process. I don't think that introducing features like this >> necessarily increases the burden of maintaining our tooling. If we get >> it right it could reduce it. > > Well, we cannot really discuss this since there's no real plan how this > all should work. I can only speak from experience. We can still put ideas forward though. Haven't come up with anything myself yet though. >> I think that it's a worthwhile goal to make core smaller. It may not be >> a gigantic enterprise system with tens of thousands of source files, >> like some of us are accustomed to working on, but I think that it >> becomes easier to reason about the features and behaviour of core when >> it's smaller.
Re: [O] Sync up the org in emacs master to org maint branch?
> > DE> This is a misunderstanding. I said I wanted to move support for certain > DE> languages and project types into ELPA, not CEDET core. I'm still of the > DE> opinion that moving it completely to ELPA is a mistake. > > It would only be a mistake if other parts of core need to use it. If only > users make use of it, then having it ELPA will be invisible to them. Apologies in advance for the significant verbage. I just want to provide context. * Context regarding CEDET: (apologies: I know this thread is more concerned with org mode and, if you'll bare with me, I think that this is relevant.) I started the CEDET merge process a few months ago. There was talk on the CEDET mailing list regarding the difficulty of getting going with CEDET as a user and/or a developer. One of the ideas put forward was that there ought to be a merge into Emacs so that one wouldn't have to clone the repo, build the code and install it with some ELisp config. hacking. I got in touch with Eric directly and bounced some ideas passed him. Subsequently I volunteered to help with the merge. I got in touch with JohnW and (not being very familiar with Emacs development) asked some basic questions about how to go about accomplishing the merge. What resulted instead was the idea that we should try to look at streamlining how CEDET get's included with the Emacs tarball rather than having it live in two repositories. I agreed with the idea and got to work on getting a version of CEDET together which would work with 26. I merged in the Emacs -> CEDET direction and ended up with a version of CEDET which is a WIP and works with Emacs 26. Some time passed between then and the start of the discussions here. I think that the approach has evolved past what I was originally planning on working towards and is now something along the lines of: do a final merge of core CEDET components and make the rest into a series of ELPA packages. * What I think that we shouldn't lose sight of (if I may suggest it): is that packaging CEDET, Org Mode and other packages like them in a process which integrates them only when producing the tarball would serve to simplify things for everyone. Emacs core wouldn't be able to depend on packages which are more relevant to the users (and package developers) of Emacs, but these packages would still there like they always have been when one downloads binary Emacs. I'm new around here and I know that I lack the context and experience in this project to make such swooping suggestions or judge the validity of these points, but I thought that they would be worth raising. Kind regards, Edward Steere
Re: [O] Sync up the org in emacs master to org maint branch?
> It's not like packages communicate with Emacs over a well > defined RESTful interface. In other words: CEDET and Gnus are not > loosely coupled, quite the opposite: they are extremely dependend on > many obscure Emacs internals (not sure about Org). Shouldn't we be trying to avoid this situation? It's sure to come back and bite us in the future. If we continue to develop a package (wherever it ends up being developed) which is so tightly coupled that any kind of re factoring in core becomes a nightmare, because we have to consider the umpteen ways in which it'll break the package, then surely that's a bad methodology to continue? (I'm not suggesting that we rewrite it to make it more loosely coupled, just that it seems like a bad idea to continue allowing this going forward.) > As a consequence, we > and also the Gnus guys decided to not do separate releases anymore. We > used to provide CEDET for different Emacs versions, and it was a *huge* > amount of work. I had a buildbot running with 7 or 8 Emacs versions to > run the test suite, and things broke pretty regularly. > Now you might say: fine, only release a package for current master. But > let's say we put CEDET into ELPA. Emacs 26 gets released, and work on > Emacs 27 starts. Now there are changes happening in Emacs 27 that > require changes in CEDET, which make it incompatible with Emacs 26. So > you have to create two packages: one for 26, one for 27? Not only is > this confusing, but it most definitely increases my workload. I feel like this problem isn't intractable. We can mark some state of CEDET as being stable under the various versions of Emacs (because it was at the time) and then only support the current release for the latest package. This would most likely require changes to package to ensure that you get an appropriate version of the package when you download it. Consider the problem which our users currently face. The built in CEDET is miles behind and missing very important bug fixes and features. The upstream CEDET can be a real pain to setup, but it has the latest features. This is not a good place to be. If we merge CEDET in and only release it with the realeses of Emacs then we are heading for a state where you only get the new features and bug fixes every time Emacs is realesed, i.e. our users might have to wait up to two years to have something fixed. This is also a bad place to be. >> We can arrange things so that a Git clone of Emacs includes pulling the >> submodules (or trees, or ELPA.git, or what not) that are considered part of >> "main Emacs development", including some of those batteries. I see this all >> as >> a process issue, and that "living in one Git repository" has just been an >> implementation strategy to satisfy that process. > > Obviously, I'm very skeptical towards such an approach. Our tooling > around Emacs development is already very intricate and only works > because a few people work quietly behind the scenes to keep this all > running. Introducing even more complexity through > submodules/subtrees/whatever will do the opposite of what you want to > achieve: it creates more work for those few people who manage the Emacs > infrastructure. But I'd love to hear what for instance Glenn and Paul > think about this. I'm interested in exploring more with regards to how the subtree approach would work. In particular how it would impact the Makefiles and build process. I don't think that introducing features like this necessarily increases the burden of maintaining our tooling. If we get it right it could reduce it. For example we could establish some sort of convention for building submodules/subtrees which allows us to simplify the related Makefiles. >> Why do the split at all? Core becomes smaller, > > First off, the Emacs repo isn't that big w.r.t. the number of > files. Secondly, while there surely is an inverse correlation between > repo size and maintainability, I would argue that as long as the Big > Three are well maintained, they are not the problem. When did CEDET, > Gnus or Org ever significantly delay an Emacs release? When was an > Emacs core developer ever forced to fix a critical thing in those > packages he did not break? These are the questions we should be > asking. From watching this list over the past years, I don't get the > feeling that the inclusion of these packages has been a significant > burden, but I may be wrong. I think that it's a worthwhile goal to make core smaller. It may not be a gigantic enterprise system with tens of thousands of source files, like some of us are accustomed to working on, but I think that it becomes easier to reason about the features and behaviour of core when it's smaller. >> updating packages within it is no longer a major issue, and (I hope) >> it will be clearer when something is a core issue vs. a package issue. > > I find this whole core vs package argument strange. If you ship Emacs > with Org, Gnus and CEDET, they are pa
