On Wed, Jan 6, 2010 at 10:02 PM, Lionel Dricot <[email protected]> wrote: > I'm sorry for being really late and I wish you a very happy/funny/free > as in speech and as in beer 2010. >
I'm later still in my follow up! So far, 2010 has been full of lots of free speech and much beer of varying cost. :) > > Le jeudi 17 décembre 2009 à 18:12 +1100, Jonathan Lange a écrit : >> On Thu, Dec 17, 2009 at 9:56 AM, Lionel Dricot <[email protected]> wrote: >> > The following describes my vision of backends and storage in GTG. This >> > is what I intended to do for 0.2. If you agree, it will be done for 0.3! >> > >> >> Thanks for sharing the vision. >> >> I don't want to stop you from doing this work, but if you want to have >> the best outcome, then it's worth being clear on exactly *why* you >> want to do this, and on what the result will look like to the user. >> Because I believe this, I'm going to ask quite a few questions. > > That's fair enough and exactly why I was writing this and not blindly > implementing/comiting. Thanks for your reply. > My pleasure. > >> >> > >From the very first lines of GTG, we knew that storing your tasks would >> > be different for everybody. That's why we designed our backend >> > architecture and we designed GTG to work with multiple backends at the >> > same time. >> > >> >> Why would storing tasks be different for everyone? I don't use Tomboy >> any more, but I used it for years without caring how it stored my >> notes. Why would I care about how GTG stores my tasks? > > > Because, in fact, that's one of the biggest tomboy complain. The most > requested feature ever for GTG (and perhaps for tomboy) is the online > storage. > > But some want Ubuntu One, other Weddav server or a custom webservice > like RTM. > > So in fact, people care about it. It's important to them and they want > to back it up. > > Tomboy neglected that fact and only recently added a hack to have sync > with Ubuntu One. It simply doesn't work : you have to manually press a > button to sync, you have conflict, no error management. > > I do want GTG to have a seamless online experience but I want to keep it > offline for those who prefer it that way. > That's an excellent goal. But I wonder whether it's better to provide a generic system, or whether we should just do Ubuntu One (for example) extremely well. > >> >> > Did you know ? GTG already supports multiple backends since 0.1 ! If you >> > manually add a second backend in the backends.xml, it will work like >> > expected. >> > >> >> I did know that. It makes me wonder, is there an explanation of the >> layers of the GTG architecture? > > Yes. The backends layer was done to allow easy multi-backends. > > The datastore/requester layer was done to isolate as much as possible > GTG core from the GUI to allow other GUI. It might be a Qt gui for KDE > guys but it could be a webinterface. > > I admit : some loose bits are purely historical. > Personally, I'd love to do some hacking to make this a bit tighter, and to change things so that layers only know about the layers *below* them. I've held off because it seems like the kind of thing that would best be done in close collaboration with you & Bertrand. >> >> > Use the already existant Luke! Do you see? Indeed, I'm looking at >> > @tags ! >> > >> > So, will it work ? >> > >> > I imagine 4 types of backend you can have : read-write, read-only, >> > import, export. >> > >> >> "Backend" is a pretty generic term. I know it has a particular use >> within the GTG codebase, but I think that the discussion would be >> clearer if we could find something better to use. > > Admittedly, this is becoming a GTGesque jargon ;-) > >> >> > * Read-write is the traditionnal backend you already know. It will also >> > be used for the couchdb backend and, in fact, most backends. >> >> Is there anything already in the code or documentation that specifies >> what you must do to implement one of these? I think that writing such >> a thing would be an excellent first step. >> >> Looking at GTG/backends/localfile.py, it seems that there's a bunch of >> top-level functions that must be provided by a module, and a Backend >> class. >> >> The purpose of most of the top-level methods is clear, but >> get_parameters() seems pretty vague. It's not clear why some things >> are top-level functions and other things are methods. > > Sorry for not explaining that. Top-level functions should be used by the > GUI to display a list of available backends and build the preferences > window for that backend. > > They are top-level because you want to call them without instantiating > the backend object by itself. > > Methods, on the opposite, will be used only in a particular instance of > that backend. > > Do you see my point ? > Yeah, I've always seen that point. Thing is, the current system makes the various backend modules singleton objects with a loosely-defined type of "backend format" or "type of backend". One could also make an object like this: class LocalFileBackend(object): def get_name(self): return "Local file" # etc. def make_backend(self, parameters, firstrunxml=None): # Probably would have more code, but with the current implementation... return Backend(parameters, firstrunxml=firstrunxml) You'd then probably want to do something like: register_backend(LocalFileBackend) This approach is easier to test, makes adding and removing backends explicit (which is better than implicit). > >> >> > * Read-only displays a list of tasks that you have to do but you should >> > trigger something external to close them. It might be useful for >> > backends like bugzilla (at least as a first step) or for centralized >> > backend where your boss have to acknowledge that you finished a task. >> >> AIUI, the _only_ thing this gives you that import backends don't is >> that it stops you from editing the tasks. >> >> In that case, rather than having multiple types of backends, why not >> have a property on tasks that says whether or not that task is >> read-only? > > Not at all. If you have a read-only backend, they can still be modified > elsewhere. When you use an "import" backend, you create new tasks in > your existing default backend. > > Do you see the difference ? > I'm sorry, but I don't. Are you saying that "import" means "import a bunch of things *once*" and read-only means "keep importing a bunch of things"? >> >> > * Import backends will simply take tasks from the backend, just as the >> > user entered them manually. A typical example might me a Twitter >> > backend. It would retrieve all of your tweets with the tag #todo and >> > create a new task with them. >> >> I don't think it's helpful to think of this as a "backend". It's >> actually two things: >> - a process for putting things into a backend >> - configuration to say how often this should take place > > I don't see how it is simpler than considering it as a backend. Because knowledge of how tasks are stored in, say, Launchpad is a fundamentally different kind of thing from "pull new tasks four times every day". Who is to say I want my tasks from Launchpad to keep importing? >> > Each backend will define itself with a set of options (a bit like >> > plugins). In order to not copy/paste too much, we might have some >> > predefined backends that are instantiated by others. For example, we >> > will have a local-text-file backend for which the user can choose the >> > location of his datas. The default backend will only be an instantiation >> > of that backend but with a fixed data path (value=XDG_HOME_DATA). >> > >> >> That's a great idea. Another way of thinking about it is that you want >> an object to represent a format (what we call a backend), and an >> object to represent a document of that format (what we call a >> tasksource). It should be possible to create a document of a >> particular format from the format object. >> >> The critical thing is that there has to be a clearly defined interface >> that any valid TaskSource must implement. Ideally, we'd have unit >> tests for this interface, and run these tests against all known types >> of tasksource. > > Don't you mean backend ? Tasksources are instance of Backends. > Uhhh, maybe. I said "types of TaskSource", which I think means "Backends". Maybe we need better words :) >> > So, what's my plan to do this? >> > >> > Before everything (step 0), I want to change and improve the backends >> > API. For example, the get_all will no longer returns every single task >> > from the dawn of humanity but will instead retrieve opened tasks and >> > tasks that were closed less that X days ago where X is a parameter, >> > common to all backends, that the user can change. This will solve : >> > https://bugs.edge.launchpad.net/gtg/+bug/312700 >> > https://bugs.edge.launchpad.net/gtg/+bug/495758 >> > >> >> Better yet, don't have a get_all method: have something that supports >> a streaming approach. > > We will refactorize that to use callbacks so the get_all will, in fact, > ask the backend to add each task himself. OK. I guess that will work. I mostly meant you should make get_all return an iterator or something similar. >> > Second step is ensure that tasks could support multiple IDs from >> > differents tasksources. >> > >> >> Why? > > Because a task could be part of multiple tasksources. > I don't mean to be childish, but... Why? >> >> > Third step : add support to assign a specific backends based on tags. >> > Yes, it means that tasks migth be first added to the default backend >> > then removed from it once a tag is added. >> > >> >> Actually, no. The next step is having a user interface for configuring >> a backend at all. >> >> Then the next step is having support for _multiple_ tasksources with >> one default write tasksources. This includes things like being able to >> look at a task and see which tasksource it is from. It will also >> include things like disabling features based on backend. >> >> *Then* you get to having support based on assigning a tasksource to a tag. > > Agree but I hate Glade so much that I always put it as the last bit :-D > Maybe you should try out Balsamiq Mockups. It's a pretty neat way of mocking up an interface, and might help with this. >> > Fifth : put a GUI above all of that so the user can choose and configure >> > his own tasksources. >> > >> >> As mentioned before, I think you should do this sooner. None of the >> earlier steps have any user benefit until you can do this. > > Agree but, hey, it's glade man ! ;-) > Yeah, but work that doesn't help users is waste. >> >> > >> > This is the vision of a multi-backends tasks system I developed for more >> > than one year. This is the last big stuff I want to add to GTG before a >> > 1.0 release. Of course, nothing is written in stone. I'm open to critics >> > and proposition (mostly from coding people as this is a quite technical >> > proposition). Also remember that GTG is a community project and that my >> > vision will only be accepted if I can convince the majority of >> > developers that this is a good idea. >> >> Thank you for putting your vision on the line. >> >> I realize that I've been a bit critical. This is partly because I have >> a bias against big infrastructural changes, but more because I care >> about GTG and want it to keep doing well. > > > I appreciate your critics. As you can see, I changed my mind for some > stuffs. I hope I answered your question. > > Your contribution to GTG is highly valuable and I'm glad to have you in > our little community. Don't hesitate to send critical mails like that, > it shapes things for a better future. > > > Pfew, I answered it !!! > And so gracefully too! :) > Thanks for that, > Thank you! I look forward to seeing this backend work actually happen & start helping end users. jml _______________________________________________ Mailing list: https://launchpad.net/~gtg-user Post to : [email protected] Unsubscribe : https://launchpad.net/~gtg-user More help : https://help.launchpad.net/ListHelp
