I've kept everybody waiting long enough...
It's a long reply... really long but then you wouldn't believe I wrote it
if it was short. I think I covered everything.
Glossary.
GUII = GUI Independence.
On Fri, 17 Nov 2000, Matthias Ettrich wrote:
> >
> > Because the current GUI-I code is limited in some ways, and certainly not
> > yet complete (especially in terms of the main window), we should abandon
> > it all together and restrict LyX to one toolkit.
>
> Of course. Restricting LyX to one toolkit is not a restriction in terms of
> software engineering, but a design decision that leads to more flexibility -
> and thus to faster and more innovative software development.
The design decision to allow anyone to port LyX to whatever interface they
want leads to easier system independence also. Sure we could pick Qt and
then use Qt/Embedded or Qt/PDA to provide interface portability across
platforms. But not everyone wants to run Qt.
As for flexibility and speedier development, the GUII code has only been
in the mainstream LyX since the developers meeting in June of this year.
We haven't even had a proper release with it in yet and we've already
picked up a John, Angus and Marko as a result of my work earlier in the
year to revise the dialog abstraction and implementation and then start
producing LyX Development News issues so the world knew what was going on.
In September of last year we abandoned the previous development tree (a
lot of porting of work has been done from there -- not just GUII related)
because it was so unstable and with only 6 or so of us working on it there
wasn't any user feedback or testing.
Fast, flexible development comes from a stable source base not from which
gui toolkit you pick. The speed of development during this year has been
so fast I'm flat out keeping up with emails let alone what everyone is
doing in the source. All this is because we switched our focus to keeping
the source as stable as we could while doing all the massive code changes
that needed to be done to get us back on par with where we were in the old
development tree (except LyX doesn't crash as soon as you try to do
anything). At this point we have probably equalled and surpassed in many
areas everything we did in the previous 3 years of the old development
tree all in about 12 months and we picked up at least four new developers
in process (three of whom work on GUI ports and dabble in lyx internals).
Having three concurrent GUI ports is a challenge but we already have some
interesting innovations from Marko and John in their Gnome and KDE ports
respectively. Sure neither of them is as proficient in their respective
toolkit as you are in KDE so maybe they don't know the shortcuts or
support facilities as well as a GUI guru might. Even so, those innovations
are only unique for a day or two before they are replicated in the other
ports.
> > I don't buy it, sorry. The sensible thing to do is to extend and continue
> > to improve the GUI-I separation. I don't believe this will lead to "lowest
> > common denominator" implementations.
> >
> > It is a pity Allan is away at the moment (I think), I'm sure he will have
> > some insightful comments here.
>
> John, wake up! See how long you guys have been working on that and see how
> far you came until now. Of course what you describe *can* be done. But it's
> even more effort in terms of development hours than doing two forks and let
> one use Gtk and one Qt. In practice, it simply will be a low common
> denominator. Maybe not the lowest, but pretty low.
The three year wait isn't just down to GUII alone. That was one part of
larger development process and it is most unfair of you to imply that GUII
is a waste of time because it isn't already complete or to imply that GUII
was the sole cause of the long delay.
Forking a KLyX, a GLyX and a CursesLyX won't make it quicker and easier to
develop LyX. It will make it easier to develop KLyX or GLyX or CursesLyX.
Keeping the various ports under the same tree ensures they use the same
core and any innovations that might occur in one core occurs in another
instantly. Fork maintenance is non-existent -- but is replaced by a
frontend abstraction that needs maintaining instead. A design tradeoff or
compromise that I think everyone in the LyX Team would currently agree
with.
We have over the last 3 years discussed other abstractions, the best of
which (IMHO) is to turn the LyX core into a library that frontends link
to. We are in many respects already working toward this but in order for
this to be a success we need to separate core from GUI (or better: core
from frontend including system specifics). Way back in the beginning of
GUII I argued we could get the xforms out of the core in a week if we all
got stuck in. That was when my focus was on getting xforms out and then
make the abstraction pretty -- since we'd then have a better chance of
getting gnome/kde/curses/whatever porters to work with us on the
independence abstraction.
If it makes it any easier for you, think of it as three years worth of
hard struggle convincing the world that GUII is a better idea than simply
forking an application and making it work on your favourite toolkit or
framework. For me one of the greatest annoyances with KDE and Gnome has
always been that both camps grab an application or utility, fork it and
port to their "chosen way" and then expect the original developers to
thank them for it (but still expect them to either switch to the "chosen
way" or crossport whatever they can to survive).
GUII for me has always been about finding a better compromise so that no
matter what wonderful new whizzbang toolkit or framework or OS comes along
LyX will survive. KDE2 is king at the moment. It really does look like
the best thing since sliced bread (and I congratulate you and the KDE team
for the fantastic work you have done) but it's not the last. You have
already said Qt3 will be even more wonderful and as a result KDE3 will be
even more wonderful.
KDE2 isn't backwards compatible with KDE1 and KDE3 is unlikely to be
backwards compatible with KDE2. Both KDE1 and KDE2 will live side by side
for a year at least so even if we jumped on board a particular toolkit
we'd end up having to either abandon support for the older version when a
new one comes out, fork-and-port to the new version or do some GUII-like
thing. Abandoning a tree abandons users of older systems. Forking and
porting also abandons users of older systems (who may not be in a position
to upgrade) because everyone will want to add the new features to the new
fork rather than backport them to the older tree. Only some GUII-like
arrangement allows you to keep all your users happy as they transition
from one incompatible version of your favourite toolkit or framework to
the next as they get many of the extra features that the forked tree would
have gotten. Sure the frontend support may lag but it's features like
exporting to pdf or importing jpeg or gif images that users really desire
(I know, I've seen it in the mailing lists -- and they now have both).
You talk about user focus and getting more users and making users happy.
How about existing users? How about users who don't want to install all
the gnome and kde libraries to run a couple of apps that depend of
different sets? KDE doesn't run gnome apps and gnome doesn't run KDE
apps. They can impersonate one another to some extent but you still need
the real libraries. Isn't it better to offer the user genuine choice by
keeping an application (any application not just LyX) focussed on being
the best it can be while different people maintain their favourite GUI or
system port of that application core?
> > > Think about your users and what you really want to develop, not about
> > > politics.
> >
> > It seems to be you who is thinking about politics. I fail to see what is
>
> You cannot argue away the fact that three years ago, the Qt port was stopped
> due to purely policial reasons (read: Qt was not available under the GNU
> GPL).
I don't give a rats about licences. If it's free to use and does what I
want I'll use it. We use XForms don't we? Most of the objections about Qt
were also aimed at XForms. It was very tempting three years ago to switch
to KLyX and port/convert everything we were all working on in the
development branch to work in KLyX (and all the stuff from the time
between 0.12.0 and what was essentially 1.0 that you didn't have in KLyX).
Very tempting. However, while we would have lost some time in the
transition to the altered codebase we would still have worked on our pet
peaves. The 6 or so core developers would still have stepped all over the
core ripping out stuff and making changes. I would still have pushed for
GUII and in the end I believe we would still have abandoned the
development tree because we would still have made the same mistakes we did
anyway.
The only difference being we would have had a KDE1 frontend instead of an
XForms frontend when we restarted.
> > wrong with giving users the choice, and I think tying LyX down
> > (needlessly) to one particular toolkit implementation is a bad idea. For a
>
> Again: see how much time you guys spent and what the outcome was. Compare
> that to the two weeks it requires to make a proper port to a decent C++
> framework.
Maybe you aren't aware but whenever any the LyX developers has a spare two
weeks we tend to go to Argentina (Jürgen), Canada (Angus), Melbourne
(Allan), Europe (Allan), just about anywhere there isn't a computer
(JMarc, Lars).
And I've already said I once argued we could do it in a week but nobody
had a week to spare. Besides it's better to do it right and have fun
while you are doing it than to do it fast for instant gratification.
That applies to more than just coding ;-)
And that two weeks assumes the people doing the port are proficient with
the toolkit -- to the point of being gurus. Fork-and-port doesn't solve
the other problems as I've already stated.
> And what's the point in chosing between LyX/Gtk and LyX/Qt if both look and
> feel more or less exactly the same?
Just so happens all the good ideas cross-pollinate and before you know it
everyone is doing it the same way. Although the KDE and gnome ports could
use a better class hierarchy since they are based on an earlier xforms
port implementation. John and Marko are already aware of this.
Most ports are going to have to show the same sorts of info in dialogs.
Most of those dialogs make sense to be broken up in the same way. Hence
most platforms use the same sorts of dialogs. That said, it is possible
to use whatever standard file-browser, spell-checker, print dialogs a
toolkit or framework or system may provide. If we can't do that then we
modify the interface.
> Users don't care about particular toolkits, they care about applications.
Ever heard of an operating system called Windows? Apparently there are
more users of Windows than any other operating system on the planet.
They're pretty dumb though so they only like stuff that looks and feels
like the stuff they are used to. So a _native_ port to Windows would be a
good way to get more users. We already have a lot of Windows based users
and OS/2 based users but they have to install all sorts of extra stuff to
make it work (stuff that dumb users wouldn't know about) -- those users on
Windows or OS/2 are smart users trapped in a Windows world. They're a
hardy bunch but they really do deserve better support. We even have
people on OS/X, BeOS, Palm Pilots and OpenStep(GNUstep/NeXTStep) that want
LyX to run natively. Maybe KDE will be ported to all these one day (and
look like native apps) but so what? If someone wants to do a native port
why stop them?
> This is not about reducing the choice of the user, this is about giving the
> user the choice to use a modern LyX. Today, the user cannot chose anything,
> because it simply isn't finished (nor will it be anytime soon).
So it's not modern unless it uses the latest fashionable toolkit? We have
a modern LyX, written in modern C++ using modern STL and providing all the
modern conveniences (the 20% of useful stuff that people always use with
the ability to do anything they want in raw LaTeX or raw DocBook). User
feedback has driven changes in the interface such as changes in the
reference and citation dialogs making them more convenient and making LyX
a more powerful document processor.
> As I pointed out in my previous mail: Netscape had a very similar
> architecture. It was a nightmare. They learned and rewrote everything. Now
> they have their own binary calling convention, there own middleware, there
> own toolkit, their own programming language for user interfaces and and and.
> This is where you are heading with this approach - just that Mozilla.org had
> a few more resources. And it still took them a lot of time.
I don't think we are like Netscape at all. Dekel keeps wanting to write
an automatic dialog builder that works across toolkits (a cross platform
toolkit in other words) but I keep yelling "Nooooo!!!". We're not
targeting cross platform (at least I'm not). We're targeting native apps
on multiple systems using a common core. GUII may only be an interim step
to something much better but at the moment it serves the purpose of
focussing attention on separating core from frontend and system
independence is a logical and small next step. Having multiple active
ports helps ensure that we are actually achieving independence.
Once the system independence is complete we may well end up switching to
different arrangement, such as the core library which frontends link to
arrangement. But the frontends will be able to be reused easily and new
ports can already be based on idea and implementation of existing ports.
[snip]
> I don't have any commercial concerns. But I do have a concerns in using a
> decent LyX version in the year 2000.
>
> lyx.org isn't going to deliver that anytime soon with the approach you are
> taking. This is sad. It's a very decent code base but it doesn't reach
> anywhere the userbase it deserves.
>
> Believe me or not, I'm thinking in LyX' interest here, not KDE's.
By "decent LyX version" you mean one that uses KDE2? Or Qt? What?
I can't think of anything you could possibly be referring to by this.
Since LyX-1.0 we've had almost every feature that a user requires to
produce great documents. 1.1.5 contains a great many new features and in
fact all year long there have been plenty of great things added to LyX.
1.1.6 is taking longer than it should because we've started slipping back
towards having development and stable branches. All the core developers
are committed to reversing this trend -- especially those of us who
experienced the heartbreak of abandoning the previous development tree and
the thrill and excitement of seeing LyX take off this year after we
decided to stick to a single stable tree and a short release cycle.
Sure we want to increase our userbase. LyX Development News has helped
this a bit by our regular exposure in Linux Weekly News -- the only news
service that will carry announcements of LDN and the only one that has
ever even replied to my emails. They have been and continue to be very
supportive of LDN and LyX.
The shorter release cycle has also helped by getting more features into
the hands of users quickly and by getting us more exposure on sites like
Freshmeat. This has a psychological effect because potential users see
that there is rapid development and while they might not like xforms we
have had several new users once they understood we are working on GUII and
why. I'd suggest you take some time to read through the various LDN
issues starting at the first one which announced the reasoning for
abandoning the old tree:
http://www.lyx.org/news/20000217.php3
> Now, if I get a statement from the LyX team that says something like "we are
> not interested in providing a decent text processor anytime soon, but we want
> to invest our spare time in inventing new ways of programming for different
> toolkits and coming up with new abstractions", that's fine with me.
>
> Just tell me that and I stop arguing immediately. I'll visit lyx.org in
> spring 2002 then to check out the latest version.
I still have no idea what you are complaining about.
Oh wait... "text processor"? LyX is a document processor. We've moved
on from text, past word-processing and into a document centric realm. You
should know better than that you started LyX in the first place. We look
like taking 6 months for a 1.1.6 release (that's 6 months from the time of
the release of 1.1.5 to the time of release of 1.1.6). Three months
longer than I'd like but then again I'd prefer to release a new LyX in
conjunction with each months LDN!
The GUII work involves four developers at present: Me (Allan), Angus,
Marko and John. Jürgen, Lars and JMarc have also made some contributions
but are focussed on other aspects of LyX. Both John and Marko have said
they are not interested in working on XForms stuff. So I'm very happy to
have them working on their favourite toolkits. This is a lot more than we
got in three years of struggling to convince the world that GUII is
worthwhile. There has been no shortage of people willing to fork LyX to
port to Gnome for example but when told we want to do GUII so we can keep
a single LyX that works on whatever toolkit comes along and need to move
the xforms stuff aside first they disappear never to be heard of again.
That leaves Angus and I working on XForms and I have a PhD thesis I have
to finish ASAP or I'm gonna get it cancelled out from under me so I have
to wind back even further. We're focussing on dialogs at the moment
because I want to finish one step at a time. We haven't the numbers or
the skills to just jump in and change everything at once. I wouldn't want
to anyway even if we did. That's one of the lessons learned from the
previous development tree.
> Seriously, in the free software business, you cannot make everybody
> happy. There will always be people complaining. For your best sake,
> you have to make decisions. Programming language, scope, toolkit,
> platforms are some of those.
>
> If you guys decided on soley supporting Gtk+, most certainly I would be
> pissed. Qt is in many ways simply much better, cross platform and it fits
> much better in LyX' C++ world. But for the sake of the LyX project itself,
> one has to realize that Gtk+ is at least better than the xforms stuff that's
> there now. And you would be able to get a release out in a halfway forseeable
> timeframe.
>
> IMO you picked the worst possible choice. By trying to make everybody happy,
> you make nobody happy.
We'll see. We've disappointed you but by still providing a solid product
we have many happy users. Some of them will have to be patient to see LyX
in their favourite flavour but the wait will be worth it and the
intervening months sees new features added and an overall improved product
(both in the interface through innovations in the competing ports and in
the strengthening of weaknesses as various itches are scratched and user
feedback is acted upon).
We have never wanted to be part of some petty KDE vs Gtk struggle. There
are far more important things in life. If we pick Gtk/GNOME you hate us.
If we pick KDE the GNOME people hate us. We're stuck in the middle of a
silly little war when all we want to do is write a great application.
GUII allows us to work on the application and let anyone who wants to
bicker about frontends write their own port. It also avoids incompatible
forks.
So please, take a minute to see things from our point of view.
> Keep in mind: a complete KDE port that replaces all the xforms stuff with
> Qt/KDE code can be done in two weeks work if you have at least two people
> fulltime. With the abstraction layers in there, it might be more complicated
> now than it was with LyX1, but one could simply drop some of those and
> replace them with the ones Qt already provides.
>
> This is why I'm so angry about the whole issue. Can you imagine what a
> powerful killerapplikation LyX would be today if you had continued working on
> the KLyX code base two years ago?
And we still would have made the mistakes in the development process we
did during the intervening years and still would be doing the same things
we are now only we'd have a KDE1 face instead of an XForms face.
Not long ago you said, "Users don't care about particular toolkits, they
care about applications." We are giving them that application now. Sure
it's not as pretty as it might be and some people won't use it because of
some licence fetish or because it doesn't look like swiss cheese when they
start it on their desktop. But the application is here and now.
> Enough ranting for now. You know my opinion, I know yours. I don't understand
> it and I think it's pretty bogus, but I can accept it. In case you change
> your mind, I'll be happy to offer Qt and KDE support to a certain agree and
> maybe even help a bit with the implementation (like for example porting
> Screen to QScrollView).
Your support would be most welcome. Your KDE and Qt experience would be
invaluable for advancing the KDE port or starting a KDE2 port. However,
I think the whole team are agreed that we want LyX to be LyX not KLyX or
GLyX or XYZLyX. There will be complications in ensuring that the
abstraction works with all ports but the abstraction is designed to be as
high a level and as simple as possible.
In an earlier email you expressed the desire to use a maths toolbar
instead of a maths dialog. There is no reason this can't be done with the
existing scheme. The toolbar abstraction is mainly for xforms benefit but
also allows other toolkits to share the same lyx-centric (toolkit
independent) toolbar definitions. You can make a maths toolbar be context
sensitive by connecting to the three appropriate signals: show, hide and
update.
You can do anything with that signal set. It doesn't just apply to
dialogs (despite the class name of Dialogs). Upon entry to a math inset
the showMath signal can be activated this would popup the dialog if
desired or the toolbar. If something changes that should cause the dialog
to be updated (in the math panel or toolbar case this is unlikely at least
as it stands at present) then the updateMath signal would be activated by
the core. For the context sensitive toolbar case a hideMath signal is
also needed. It's up to the individual port to decide how to implement
these things. The signals can be considered as hooks. LyX provides them
and the port handles them. The LyX core doesn't really know what's at the
other end of the signal and doesn't really care.
We expect a breakup of dialogs along certain lines but then most
wordprocessors look the same also with similar breakups of their dialogs.
Why shouldn't we expect that all ports will have a spellchecker dialog of
some sort or a filebrowser? Sure we abstract the interface a little.
Generally the signal triggers something in the frontend and the frontend
pulls the info it needs from the core via a LyXFunc -- access function --
or pushes new info to the core via a LyXFunc. At present we also use a
Liason namespace for some global functions we can't do as a LyXFunc
because compiler contraints led us to drop XTL.
The individual ports can implement these however they want. If the
framework provides a filebrowser then the porter would be a fool to
reinvent the wheel likewise with the spellchecker dialog. Other dialogs
don't necessarily have to be built as individual dialogs or even as
dialogs they could be toolbars or some other weird GUI widget. If some
porter decides they really want to bundle the Character, Paragraph and
Document dialogs into a single big dialog with pretty icons to select
which thing they want to change then by all means implement it that
way. They can connect the appropriate show() signal such that it opens
the dialog with the matching tab visible.
That is the key to understanding the abstraction. It might not fit like a
glove for all platforms but can be made to work neatly. The abstractions
for the menues and toolbars have undergone a couple of revisions as JMarc
and Marko have experimented with the XForms and Gnome ports. No doubt
other improvements will come when John tackles this later on the KDE port.
These may even be completely rewritten. As it stands these are reworkings
of the previous work in the old development tree.
The workarea abstraction using a Painter class should be adaptable to
whatever canvas class a framework provides. So far I've encouraged the
porters to concentrate on getting the dialogs done before we worry about
tackling the workarea. One step at a time, one dialog at a time. Sure
its a bit slower than taking two weeks off to work on a full-on port but
who really wants to do that anyway? Burnt out developers are no use as
maintainers. LyX has been advancing rapidly enough for my liking in the
last 6 months. Stable, steady development of the interface while at the
same time the guts of LyX is being torn apart and rewritten using STL and
better C++ idioms by the likes of Lars, Jürgen, Dekel and others. Much of
this work was done before in the old tree and has been repeated now but in
better ways.
Qt may provide all sorts of extra containers and trinkets but libsigc++
has a better signalling system IMO and STL has better containers and
template functions. So Qt is now a toolkit with awkward interfaces.
Still a very good toolkit though but not the revelation it once was.
You're a good salesman and KDE2 is a fine product but I for one am not
jumping ship. I'd rather have KDE2 supported along with other toolkits
than tie LyX down to yet-another-toolkit-or-framework (YATOF). If that
takes till next Christmas then I would be very surprized. It certainly
won't take till 2002 as you suggest.
Allan. (ARRae)
P.S. I've spent so much time writing this that I'm inclinded to just
include it word for word in the next issue of LDN rather than just link to
to it. Do you have a problem with me doing that? Do you want your quotes
removed and I'll rewrite it as a more general address?
P.P.S. I'll take silence as consent. The next LDN will be released on
Wednesday the 13th of December.
P.P.P.S. The next issue was going to have a big GUII feature anyway.
Your email has just helped get it written sooner.
