Re: [O] [PATCH] Smart inference of task progress when exporting to TJ3
Hi Christian, Christian Egli christian.e...@sbs.ch writes: The taskjuggler exporter used to be in core before it was moved to contrib by Bastien or Nicolas on the bases that there aren't many users (and probably because it wasn't maintained very well). This is OK for now, but eventually I'd like to move it back to core. So we need assignment for this patch. The rationale for moving some files out of core is not only about the size of the user-base (which is very difficult to know), but mostly about whether those contributed packages depend on 3rd-part softwares. Here is how I stated this in the release notes: Since packages in Org's core are meant to be part of GNU Emacs, we try to be minimalist when it comes to adding files into core. For 8.0, we moved some contributions into the contrib/ directory. The rationale for deciding that these files should live in contrib/ is either because they rely on third-part softwares that are not included in Emacs, or because they are not targetting a significant user-base. For example, org-mew.el and org-wl.el were moved in contrib/ because Mew and Wanderlust are not part of Emacs. Same for org-mac-message.el. We made one exception for org-w3m.el because emacs-w3m is the most popular way of browsing the Web within Emacs *and* because there are ongoing discussions about integrating it in Emacs: http://lists.gnu.org/archive/html/emacs-devel/2012-09/msg00116.html I hope this clarifies why I made this choice. Let me know if you think it is too controversial! Best, -- Bastien
Re: [O] [PATCH] Smart inference of task progress when exporting to TJ3
Bastien b...@gnu.org writes: The rationale for moving some files out of core is not only about the size of the user-base (which is very difficult to know), but mostly about whether those contributed packages depend on 3rd-part softwares. PS: Also see this recent message by Glenn: http://article.gmane.org/gmane.emacs.devel/159305 I think code for mailers that don't come with Emacs doesn't belong in Emacs. This is the reasoning I also tried to apply when deciding about whether a package should go into core or contrib. -- Bastien
Re: [O] [PATCH] Smart inference of task progress when exporting to TJ3
Martin vbmaz...@web.de writes: Hello Martin, Thanks for your patch. I will only make a few stylistic remarks. 1. Could you add `contrib/lisp/ox-taskjuggler.el:' to the subject? (The first line of the commit message) 2. IIUC you don't need the TINYCHANGE cookie because the changes are made in contrib/ which doesn't require copyright assignment. Furthermore, I don't think your changes could be considered `tiny' (it changes more than 20 lines and doesn't look like a rewrite). 3. Could you reformat a bit your code so it fit in 80 column? (You can often use M-q for the comments) -- Daimrod/Greg pgp8MNzaVlSsW.pgp Description: PGP signature
Re: [O] [PATCH] Smart inference of task progress when exporting to TJ3
Dear Daimrod, Thank you for these remarks, next time I know. I attached a clean patch. On 03.05.2013 10:40, Daimrod wrote: Martin vbmaz...@web.de writes: Hello Martin, Thanks for your patch. I will only make a few stylistic remarks. 1. Could you add `contrib/lisp/ox-taskjuggler.el:' to the subject? (The first line of the commit message) 2. IIUC you don't need the TINYCHANGE cookie because the changes are made in contrib/ which doesn't require copyright assignment. Furthermore, I don't think your changes could be considered `tiny' (it changes more than 20 lines and doesn't look like a rewrite). 3. Could you reformat a bit your code so it fit in 80 column? (You can often use M-q for the comments) From ec5076f93678a3ca5d0944cd777a0d87c74dfeee Mon Sep 17 00:00:00 2001 From: Martin Becker vbmaz...@web.de Date: Thu, 2 May 2013 23:08:40 +0200 Subject: [PATCH] contrib/lisp/ox-taskjuggler.el: Smart inference of task progress when exporting to TJ3 * contrib/lisp/ox-taskjuggler.el: Extend variable `complete' to respect clock times if no other information is given. There are three possibilities for setting a task's progress when exporting to TJ3: 1) If TODO state equals done = 100% 2) If property complete is explicitly given = use that. 3) Otherwise get the clocksum of the task and set the progress by comparing to `effort'. --- contrib/lisp/ox-taskjuggler.el | 36 1 file changed, 32 insertions(+), 4 deletions(-) diff --git a/contrib/lisp/ox-taskjuggler.el b/contrib/lisp/ox-taskjuggler.el index 3458e06..97c4aae 100644 --- a/contrib/lisp/ox-taskjuggler.el +++ b/contrib/lisp/ox-taskjuggler.el @@ -780,6 +780,17 @@ channel. ;; Closing report. }\n)) + +(defun org-taskjuggler--clocksum-task (task) + Return the clocksum of a parsed subtree = does not verify +whether the parsed duration matches the start and end times + (let* ((tsum 0)) +(org-element-map (org-element-contents task) 'clock + (lambda (hl) (setq tsum (+ tsum (org-duration-string-to-minutes + (org-element-property :duration hl)) +tsum +)) + (defun org-taskjuggler--build-task (task info) Return a task declaration. @@ -791,11 +802,28 @@ a property \task_id\ it will be used as the id for this task. Otherwise it will use the ID property. If neither is defined a unique id will be associated to it. (let* ((allocate (org-element-property :ALLOCATE task)) + (effort (org-element-property :EFFORT task)) + ;; smart completeness inference: + ;; - if state=done = 100% + ;; - if explicit property for complete = take that value + ;; - otherwise get clocksum of this task and relate to the + ;; effort, whilst limiting values to [0,99] + ;; = 100% possible if and only if state=DONE. + ;;Overbooking does stop at 99%. + (complete-explicit (org-element-property :COMPLETE task)) (complete - (if (eq (org-element-property :todo-type task) 'done) 100 -(org-element-property :COMPLETE task))) + ;; state=done = complete=100 + (if (eq (org-element-property :todo-type task) 'done) 100.0 +;; else if explicit property = use that +(if complete-explicit (string-to-number complete-explicit) + ;; else guess the completeness from clocksum + (let* ((clocked (org-taskjuggler--clocksum-task task)) + (complete-guess + (and effort (/ (* 100 clocked) + (org-duration-string-to-minutes effort) +(and complete-guess (max (min 99 complete-guess) 0))) + ))) (depends (org-taskjuggler-resolve-dependencies task info)) - (effort (org-element-property :EFFORT task)) (milestone (or (org-element-property :MILESTONE task) (not (or (org-element-map (org-element-contents task) 'headline @@ -826,7 +854,7 @@ a unique id will be associated to it. (if (= org-taskjuggler-target-version 3.0) allocate allocations) allocate)) - (and complete (format complete %s\n complete)) + (and complete (format complete %.1f\n complete)) (and effort (format effort %s\n (let* ((minutes (org-duration-string-to-minutes effort)) -- 1.7.9.5
Re: [O] [PATCH] Smart inference of task progress when exporting to TJ3
Hi Martin Martin Becker vbmaz...@web.de writes: Thank you for these remarks, next time I know. I attached a clean patch. I like this patch, this is good stuff. Can you tell me how you use this? I.e. does tj3 adapt the length fo a task based on your clock table? A few remarks on the patch from my side: + ;; state=done = complete=100 + (if (eq (org-element-property :todo-type task) 'done) 100.0 +;; else if explicit property = use that +(if complete-explicit (string-to-number complete-explicit) + ;; else guess the completeness from clocksum + (let* ((clocked (org-taskjuggler--clocksum-task task)) + (complete-guess + (and effort (/ (* 100 clocked) + (org-duration-string-to-minutes effort) +(and complete-guess (max (min 99 complete-guess) 0))) + ))) I would write this using cond instead of nested if. Makes it more clear. 2. IIUC you don't need the TINYCHANGE cookie because the changes are made in contrib/ which doesn't require copyright assignment. The taskjuggler exporter used to be in core before it was moved to contrib by Bastien or Nicolas on the bases that there aren't many users (and probably because it wasn't maintained very well). This is OK for now, but eventually I'd like to move it back to core. So we need assignment for this patch. Thanks Christian -- Christian Egli Swiss Library for the Blind, Visually Impaired and Print Disabled Grubenstrasse 12, CH-8045 Zürich, Switzerland
Re: [O] [PATCH] Smart inference of task progress when exporting to TJ3
Hi Christian,
I like this patch, this is good stuff. Can you tell me how you use
this?
Thanks, I attached a file where I made some tests. My idea was, that
clock times are only used when there is no explicit information given,
viz., when there is neither a property `complete' nor the task state
equals done. Since I am clocking most of my activities, it would be
convenient for me to let the exporter/TJ3 infer the task progress
instead of explicitly giving the `complete' property. So far for the story.
I.e. does tj3 adapt the length fo a task based on your clock
table?
No, TJ3 is not adapting anything here, neither am I. Since you cannot
give values 100 (%), my patch limits the maximum inferred progress to
99. We could let the exporter overwrite the effort with the clocksum in
those cases, but it doesn't seem right.
I found out some more details after reading the documentation of TJ3:
- The `complete' property is merely for documentation purposes, it's
totally ignored by TJ3 when it comes to scheduling. It just looks
nice. Being only cosmetic, tracking can be done via clock times as
proposed.
- But there is another thing we could do: One can add bookings to the
tasks, which could be exactly the clock times from org-mode. I tried
this manually and it seems to be a mixed blessing: If doing so, TJ3
internally grows the effort when the bookings exceed them and
reschedules accordingly (which is nice). On the other hand sensible
clock times are vital then. What I mean is, when there are bookings that
are too early w.r.t. a task's earliest possible start (due to
dependencies etc.), then TJ3 exits with an error. I am not sure this is
wanted behavior. What do you think? Additionally, the visually
attractive completeness is not derived from the bookings or anything.
I would write this using cond instead of nested if. Makes it more
clear.
Oh yes, I will do that. I am fairly new to Lisp, it is a nice exercise.
2. IIUC you don't need the TINYCHANGE cookie because the changes
are made in contrib/ which doesn't require copyright assignment.
The taskjuggler exporter used to be in core before it was moved to
contrib by Bastien or Nicolas on the bases that there aren't many
users (and probably because it wasn't maintained very well). This is
OK for now, but eventually I'd like to move it back to core. So we
need assignment for this patch.
Is it that I need to sign something? That is not a problem, I happily
contribute here.
#+PROPERTY: Effort_ALL 2d 5d 10d 20d 30d 35d 50d
#+PROPERTY: allocate_ALL dev doc test
#+PROPERTY: ORDERED_ALL true false
#+COLUMNS: %70ITEM(Task) %3Effort %8allocate %BLOCKER %ORDERED
* Resources:taskjuggler_resource:
** Developers
:PROPERTIES:
:resource_id: dev
:END:
*** Paul Smith
:PROPERTIES:
:resource_id: paul
:END:
*** Sébastien Bono
:PROPERTIES:
:resource_id: seb
:END:
*** Klaus Müller
** Others
*** Peter Murphy
:PROPERTIES:
:resource_id: doc
:limits: { dailymax 7h }
:END:
*** Dim Sung
:PROPERTIES:
:resource_id: test
:END:
* Accounting Software :taskjuggler_project:
SCHEDULED: 2013-04-15 Mo
** STARTED Specification
- State STARTEDfrom TODO [2013-05-02 Do 16:50]
CLOCK: [2013-04-30 Do 16:50]--[2013-05-03 Do 16:50] = 72:00
:PROPERTIES:
:Effort: 20d
:allocate: dev
:task_id: task_spec
:END:
** Software Development
:PROPERTIES:
:allocate: dev
:ORDERED: true
:BLOCKER: previous-sibling
:task_id: task_dev
:END:
*** TODO Database coupling
CLOCK: [2013-05-03 Do 15:21]--[2013-05-11 Do 22:21] = 199:00
:PROPERTIES:
:Effort: 20d
:END:
*This task uses the smart progress inference*: This task's
completeness is guessed using the clocked times, since it has no
explicit `complete' property. However the clock durations add up
to more than the effort and TJ3 does only allow values
[0,100]. Therefore semantically the task is not regarded as
complete until the TODO state equals done, no matter how much we
clock on it. According to TJ3 doc, the complete information has
no impact on the TJ3 scheduler, it's pure documentation.
*Alternatives*:
1) Instead of setting `complete', we could set the property
`booking' for a resource or task, when we export to TJ3. Then
TJ3 would have additional information, since the bookings
reflect how much work was actually put into a task vs. how much
we originally planned. Note that unlike `complete' this has an
impact on the scheduling. TJ3 then can schedule a new plan
based on the amount of work that actually happened. Downside
is, that the org-syntax does not capture who/which resource
clocks on a task. And that the clock times will cause
scheduling errors when they do not comply with the scheduled
time windows for the tasks (e.g., when the bookings are earlier
Re: [O] [PATCH] Smart inference of task progress when exporting to TJ3
Hi Martin Martin Becker vbmaz...@web.de writes: My idea was, that clock times are only used when there is no explicit information given, viz., when there is neither a property `complete' nor the task state equals done. Since I am clocking most of my activities, it would be convenient for me to let the exporter/TJ3 infer the task progress instead of explicitly giving the `complete' property. So far for the story. Ah, I get it. You are infering the % complete based on how much effort you have already put into a task. The problem is that this doesn't always relate to each other: You might have put a lot of effort into a task and still be only at 10% complete. I.e. does tj3 adapt the length fo a task based on your clock table? No, TJ3 is not adapting anything here, neither am I. Since you cannot give values 100 (%), my patch limits the maximum inferred progress to 99. We could let the exporter overwrite the effort with the clocksum in those cases, but it doesn't seem right. Yes, since this is another problem with infering the complete information from the planed effort and actual effort: You can end up with complete more than 100% :-), namely if the actual effort is bigger than the planed effort. I found out some more details after reading the documentation of TJ3: - The `complete' property is merely for documentation purposes, it's totally ignored by TJ3 when it comes to scheduling. It just looks nice. Yes, I think there are some other draw backs with the complete information. Check the taskjuggler mailing list archives for a discussion on tracking the progress of a project (https://groups.google.com/d/topic/taskjuggler-users/ZCk_est4GKE/discussion). I for one would like it if were used for scheduling. - But there is another thing we could do: One can add bookings to the tasks, which could be exactly the clock times from org-mode. I tried this manually and it seems to be a mixed blessing: If doing so, TJ3 internally grows the effort when the bookings exceed them and reschedules accordingly (which is nice). This is something that would make sense in the exporter. The clocking information that we have in the org file is really what tj3 wants as a booking. On the other hand sensible clock times are vital then. What I mean is, when there are bookings that are too early w.r.t. a task's earliest possible start (due to dependencies etc.), then TJ3 exits with an error. I am not sure this is wanted behavior. What do you think? Yes, it appears that you will have to have bookings for all the tasks to make this work properly. This is a lot of overhead. I'm also not sure this is wanted. But the benefit would be that you can use projection mode in which tj3 will afaik recalculate the duration of the project based an planed and actual effort. Might be worth exploring. Additionally, the visually attractive completeness is not derived from the bookings or anything. Yes. but eventually I'd like to move it back to core. So we need assignment for this patch. Is it that I need to sign something? Have a look at request-assign-future.txt in the org-mode git repo. Thanks Christian -- Christian Egli Swiss Library for the Blind, Visually Impaired and Print Disabled Grubenstrasse 12, CH-8045 Zürich, Switzerland
Re: [O] [PATCH] Smart inference of task progress when exporting to TJ3
Hi again, On 03.05.2013 22:25, Christian Egli wrote: Hi Martin Martin Becker vbmaz...@web.de writes: My idea was, that clock times are only used when there is no explicit information given, viz., when there is neither a property `complete' nor the task state equals done. Since I am clocking most of my activities, it would be convenient for me to let the exporter/TJ3 infer the task progress instead of explicitly giving the `complete' property. So far for the story. Ah, I get it. You are infering the % complete based on how much effort you have already put into a task. The problem is that this doesn't always relate to each other: You might have put a lot of effort into a task and still be only at 10% complete. Of course this rarely works out like that. At least me, I am using such planning to see how wrong I was and what is the impact of that. Though, booking would be better for that as I learned meanwhile. I.e. does tj3 adapt the length fo a task based on your clock table? No, TJ3 is not adapting anything here, neither am I. Since you cannot give values 100 (%), my patch limits the maximum inferred progress to 99. We could let the exporter overwrite the effort with the clocksum in those cases, but it doesn't seem right. Yes, since this is another problem with infering the complete information from the planed effort and actual effort: You can end up with complete more than 100% :-), namely if the actual effort is bigger than the planed effort. yes...see above. I found out some more details after reading the documentation of TJ3: - The `complete' property is merely for documentation purposes, it's totally ignored by TJ3 when it comes to scheduling. It just looks nice. Yes, I think there are some other draw backs with the complete information. Check the taskjuggler mailing list archives for a discussion on tracking the progress of a project (https://groups.google.com/d/topic/taskjuggler-users/ZCk_est4GKE/discussion). I for one would like it if were used for scheduling. That's why I stopped the inferred progress at 99%...one never knows as the linked thread shows ;) Until there is a change in semantics for that property, it shouldn't hurt to guess it like that. - But there is another thing we could do: One can add bookings to the tasks, which could be exactly the clock times from org-mode. I tried this manually and it seems to be a mixed blessing: If doing so, TJ3 internally grows the effort when the bookings exceed them and reschedules accordingly (which is nice). This is something that would make sense in the exporter. The clocking information that we have in the org file is really what tj3 wants as a booking. Agreed, I think that's much more interesting than `complete'. However is tricky to integrate that well without ending in bad (ox-taskjuggler) user experience caused by TJ3 refusing to schedule... On the other hand sensible clock times are vital then. What I mean is, when there are bookings that are too early w.r.t. a task's earliest possible start (due to dependencies etc.), then TJ3 exits with an error. I am not sure this is wanted behavior. What do you think? Yes, it appears that you will have to have bookings for all the tasks to make this work properly. This is a lot of overhead. I'm also not sure this is wanted. But the benefit would be that you can use projection mode in which tj3 will afaik recalculate the duration of the project based an planed and actual effort. Might be worth exploring. I am not quiet through TJ3 doc to understand what projection really is, but it seems that booking only *some* tasks also makes a difference (as I said, it grows the effort if there was an underestimation and thereby affects the project timing). Eventually a user could clock and book just a subset of the tasks (e.g., his own) and skip others or whatever... Additionally, the visually attractive completeness is not derived from the bookings or anything. Yes. but eventually I'd like to move it back to core. So we need assignment for this patch. Is it that I need to sign something? Have a look at request-assign-future.txt in the org-mode git repo. Will do. Thanks Christian Thanks have a nice weekend, Martin
