[O] Bug? R: Org babel block execution *drastically* slower than in ESS session directly
I edited the subject to be more concise/clear. I let orgmode chug away on reading in some ~10-30mb csv files for nearly 30min. I finally gave up with Ctrl + g. Out of curiosity, I copied my babel block code (four simple lines in the form of `var - read.csv(./file.csv, header = T, sep = ,)` into the ESS session already started by my first block execution, and it finished in less than 10 seconds. Something seems seriously wrong with however org + babel is executing. I'm suspicious of it printing the output to the minibuffer. I have about 200^2 - 400^2 lines in the csv files. Was shocked that I'd been waiting for org this whole time and could have just run the code in the ESS R session directly in a minuscule fraction of the time. I could post a reproducible csv file if others would like to try this? I tried the following to make sure it wasn't my setup: 1) from terminal: `emacs -Q --load=~/Desktop/basic-config`, where basic-config contains: ;; set load paths ;; set load dirs and global config options (add-to-list 'load-path ~/.elisp/site-lisp/) (add-to-list 'load-path ~/.elisp/org.git/contrib/lisp/) (add-to-list 'load-path ~/.elisp/org.git/lisp/) (add-to-list 'load-path ~/.elisp/site-lisp/ess/lisp/) ;; setup org (require 'org-install) ;; setup babel languages (require 'ess-site) (org-babel-do-load-languages 'org-babel-load-languages '((R . t))) 2) Created a simple org file: -- * headline #+begin_src R :session r :results silent data - read.csv(/media/vault/misc/benefit-plan-analysis/2012/contour_table.csv) #+end_src -- 3) Run =C-c C-c YES RET= on block 4) Change to R session and enter the same line manually. The results are unbelievably different. I didn't let the org session finish as once it took longer than 30sec I figured I'd get the same results as with my actual file in this example. Changing to the R session and it was done in less than 5sec with a ~70mb csv. Any suggestions on troubleshooting this? Best regards, John On Tue, Oct 30, 2012 at 2:46 AM, John Hendy jw.he...@gmail.com wrote: I'm working with a large dataset. I try to minimize overhead by doing some bigger data rearranging and calculations in a single block that I execute once per session and then switch :eval to no. This block is set with :results silent, but I know with this option the results are still printed to the minibuffer. Usually this isn't a problem, but I've had some hangs with emacs and started wondering if this could be due to Orgmode regurgitating the results. Could this be the case? If so, could Org be made to stop this behavior? If I accidentally type the name of a data frame in the R session buffer, even when I use C-c C-c to stop the print out, it still takes quite a while to catch up and stop itself. It made me wonder if trying to print all of those results is increasing my compilation times unnecessarily. Thanks for any thoughts, John
[O] Displaying errors in batch mode
Hello,
Errors generated by Org are correctly reported in batch mode:
#+begin_src sh
emacs --batch -Q --eval (add-to-list 'load-path \~/src/org-mode/lisp\)
--eval (add-to-list 'load-path \~/src/org-mode/contrib/lisp\) -l
~/src/org-batch/bin/org-batch-init.el ecm.txt -f org-export-as-pdf
#+end_src
LaTeX errors sometimes well, sometimes not at all: you only know that there
were errors, and that the PDF file has not been produced. No more.
And this seems hard to reproduce (whether or not you get the LaTeX errors
displayed). Maybe you have some hints?
* ECM Org error
#+begin_src org
,#+TITLE: ECM No error reported
,#+LANGUAGE: en
,#+LaTeX_CLASS: myarticle
,* Example
Blah blah...
#+end_src
will results in:
#+begin_example
Exporting to PDF...
Exporting to LaTeX...
Debugger entered--Lisp error: (error No definition for class `myarticle' in
`org-export-latex-classes')
signal(error (No definition for class `myarticle' in
`org-export-latex-classes'))
error(No definition for class `%s' in `org-export-latex-classes'
myarticle)
(or (car (assoc org-export-latex-class org-export-latex-classes)) (error No
definition for class `%s' in `org-export-latex-classes'
org-export-latex-class))
#+end_example
which is *crystal-clear*.
* ECM LaTeX error (invalid syntax used for comments)
#+begin_src org
,#+TITLE: ECM No error reported
,#+LANGUAGE: en
,##+LaTeX_CLASS: myarticle
,* Example
Blah blah...
#+end_src
sometimes results in:
#+begin_example
Exporting to PDF...
Exporting to LaTeX...
Saving file d:/home/sva/src/org-batch/ecm.tex...
Wrote d:/home/sva/src/org-batch/ecm.tex
Saving file d:/home/sva/src/org-batch/ecm.tex...
Wrote d:/home/sva/src/org-batch/ecm.tex
LaTeX export done, pushed to kill ring and clipboard
Processing LaTeX file d:/home/sva/src/org-batch/ecm.tex...
Latexmk: This is Latexmk, John Collins, 30 March 2012, version: 4.31.
Report bugs etc to John Collins collins at phys.psu.edu.
Rule 'pdflatex': Rules subrules not known to be previously run:
pdflatex
Rule 'pdflatex': The following rules subrules became out-of-date:
'pdflatex'
Run number 1 of rule 'pdflatex'
Running 'pdflatex -recorder d:/home/sva/src/org-batch/ecm.tex'
Latexmk: applying rule 'pdflatex'...
This is pdfTeX, Version 3.1415926-2.3-1.40.12 (Web2C 2011)
[...]
! Missing $ inserted.
inserted text
$
l.39 \#\#+\LaTeX{}_
CLASS: myarticle
?
! Emergency stop.
inserted text
$
l.39 \#\#+\LaTeX{}_
CLASS: myarticle
! == Fatal error occurred, no output PDF file produced!
Transcript written on ecm.log.
Failure to make 'ecm.pdf'
Collected error summary (may duplicate other messages):
pdflatex: Command for 'pdflatex' gave return code 256
Latexmk: Use the -f option to force complete processing.
Latexmk: Errors, so I did not complete making targets
D:\texlive\2011\bin\win32\runscript.tlu:587: command failed with exit code 12:
perl.exe d:/texlive/2011/texmf-dist/scripts/latexmk/latexmk.pl -pdf
d:/home/sva/src/org-batch/ecm.tex
Processing LaTeX file d:/home/sva/src/org-batch/ecm.tex...done
Debugger entered--Lisp error: (error PDF file
d:/home/sva/src/org-batch/ecm.pdf was not produced)
signal(error (PDF file d:/home/sva/src/org-batch/ecm.pdf was not produced))
error(PDF file d:/home/sva/src/org-batch/ecm.pdf was not produced)
(if (not (file-exists-p pdffile)) (error (concat PDF file pdffile was
not produced (if errors (concat : errors ) ))) (set-window-configuration
wconfig) (if org-export-pdf-remove-logfiles (progn (progn (let
((--dolist-tail-- org-export-pdf-logfiles) ext) (while --dolist-tail-- (setq
ext (car --dolist-tail--)) (setq file (concat base . ext)) (and
(file-exists-p file) (delete-file file)) (setq --dolist-tail-- (cdr
--dolist-tail--))) (message (concat Exporting to PDF...done (if errors
(concat , with some errors: errors) ))) pdffile)
#+end_example
That is, the LaTeX error is well displayed... Very clear.
* ECM LaTeX error (undefined environment)
#+begin_src org
,#+TITLE: ECM No error reported
,#+LANGUAGE: en
,* Example
,#+LaTeX: {\begin{mychangemargin}{-2.5cm}{0cm}
Blah blah...
,#+LaTeX: \end{mychangemargin}}
#+end_src
sometimes results in:
#+begin_example
Exporting to PDF...
Exporting to LaTeX...
Saving file d:/home/sva/src/org-batch/ecm.tex...
Wrote d:/home/sva/src/org-batch/ecm.tex
Saving file d:/home/sva/src/org-batch/ecm.tex...
Wrote d:/home/sva/src/org-batch/ecm.tex
LaTeX export done, pushed to kill ring and clipboard
Processing LaTeX file d:/home/sva/src/org-batch/ecm.tex...
Latexmk: This is Latexmk, John Collins, 30 March 2012, version: 4.31.
Report bugs etc to John Collins collins at phys.psu.edu.
Rule 'pdflatex': File changes, etc:
Changed files, or newly in use since previous run(s):
'd:/home/sva/src/org-batch/ecm.tex'
'ecm.aux'
[O] [feature] Cut paste of subtree
Hello, Since more or less one month or so, I've seen a change in the behavior of C-c C-x C-w, when cutting and pasting a subtree. As of today, with the following situation, I do C-c C-x C-w on whichever character from the C entry. --8---cut here---start-8--- * Tree A Foo * Tree B Bar * Tree C Baz * Tree D Dunno --8---cut here---end---8--- Then, I go (for example) in column 0 of the B entry, and press C-y to paste the previously cut subtree. As you can see below, that results in: - one whiteline too much in front of C, - one whiteline missing after C (in front of B). --8---cut here---start-8--- * Tree A Foo * Tree C Baz * Tree B Bar * Tree D Dunno --8---cut here---end---8--- This is like if cutting the C subtree had cut from the end of the previous subtree to its own end, instead of cutting from the beginning of its own subtree to the beginning of the next one. Best regards, Seb -- Sebastien Vauban
[O] Inserting a new item vs a new headline
Hello, I find weird the positioning of a new headline, when created via M-RET. Let me explain with the following ECM: --8---cut here---start-8--- * Current section ** List - Foo - Baz @ @ = position of the cursor * Next section --8---cut here---end---8--- If I want to add new item to the list, I press M-RET. It will do the right thing, and conserves the whitelines below: --8---cut here---start-8--- * Current section ** List - Foo - Baz - @ * Next section --8---cut here---end---8--- C-RET, on the other hand, will go at the most far position (one character before the next section), and glue the new headline there: --8---cut here---start-8--- * Current section ** List - Foo - Baz ** @ * Next section --8---cut here---end---8--- Often, when not thinking at that, I already insert white lines, but then, I unavoidably have to remove them by hand. --8---cut here---start-8--- * Current section ** List - Foo - Baz @ * Next section --8---cut here---end---8--- becomes this: --8---cut here---start-8--- * Current section ** List - Foo - Baz ** @ * Next section --8---cut here---end---8--- I would have loved to see the `**' where the cursor was. I find this not inline with what would happen if I'd know the text of my new headline: --8---cut here---start-8--- * Current section ** List - Foo - Baz @Text of my new headline * Next section --8---cut here---end---8--- becomes, when M-RETing[1], as *I* expect it: --8---cut here---start-8--- * Current section ** List - Foo - Baz ** Text of my new headline@ * Next section --8---cut here---end---8--- All of this is really about details. And, maybe, they're due to misuse of C-RET and M-RET. Just wanted to signal them, in case that... Best regards, Seb [1] Not C-RETing... -- Sebastien Vauban
[O] date function in table formula does not understand German date stamps
Hi, I use the function `date' in a table formula to convert org date stamps to integers for further processing. However, after switching Emacs to a German locale, this no longer works. Consider the table below: | Stamp| date | |--+| | [2012-10-31 Wed] | 734808 | | [2012-10-31 Mi] | #ERROR | #+TBLFM: $2=date($1) Org mode generally works fine with the German version of the date stamp, i.e., it is picked up in the agenda and deadline warnings work as expected. However, the `date' function in a table formula does not work. My Emacs inserts the German version of the date stamp after I've added the following file as `.MacOSX/environment.plist': ?xml version=1.0 encoding=UTF-8? !DOCTYPE plist PUBLIC -//Apple//DTD PLIST 1.0//EN http://www.apple.com/DTDs/PropertyList-1.0.dtd; plist version=1.0 dict keyLANG/key stringde_DE.UTF-8/string keyLC_ALL/key stringde_DE.UTF-8/string /dict /plist This was necessary to make Cocoa Emacs correctly process filenames with umlauts. Any idea how the date function can be made locale-aware? Or is there another alternative for this calculation? Thanks, Viktor
Re: [O] [ANN] Smart quotes in org-export.el
Hello, Michael Gauland mikely...@no8wireless.co.nz writes: I thought I'd give this a try, but when I try to export the file below (to HTML or LaTeX), I get an error: org-export-activate-smart-quotes: Wrong type argument: numberp, nil Fixed. Thank you for testing this feature and reporting back errors. Regards, -- Nicolas Goaziou
[O] Export of multiply occurring events
Hi, The iCalendar export module is a fantastic way to share an org-mode calendar, but there is one thing I am not able to make it do. It will export a single event, or a periodically occurring event with a time stamp like ... +1w. What to do if some item is occurring exactly twice ? Say it is a two-part conference, or something like that. Putting two timestamps makes it appear twice in the org agenda, but only the later one gets exported. I could always create sub-events for each occurrence, but then the text will not be exported, or will have to be duplicated, which feels sub-optimal. Is there something I didn't think of ? Thanks, /vincent -- Vincent Beffara
Re: [O] Babel: Processing Tables - Ignore Some Rows
Thanks Thomas that works fine. Can you point me to where indexable variables are documented, as my feeble search skills seem to be faiing. Ian. On 30/10/12 16:25, Thomas S. Dye wrote: Aloha Ian, Will indexable variable values do what you want? Something like :var data=monthly_total_mileage[0:-3,]? hth, Tom Ian Barton li...@wilkesley.net writes: I am using gnuplot to plot some data from a table: #+tblname: monthly_total_mileage |---+--| | Month | Distance | |---+--| | Jan | 272.04 | | Feb | 317.11 | | Mar | 354.27 | | Apr | 288.21 | | May | 488.35 | | Jun | 444.92 | | Jul | 497.21 | | Aug | 625.35 | | Sep | 821 | | Oct |717.9 | | Nov | | | Dec | | |---+--| | Total | 4826.36 | |---+--| #+begin_src gnuplot :var data=monthly_total_mileage :file ./monthly_total_mileage.png reset set boxwidth 0.5 set xlabel 'month' set ylabel 'distance, km' plot data u 2:xticlabels(1) notitle with boxes fs solid 0.5 #+end_src Ideally I want Babel to ignore the Totals row, so it doesn't get processed and plotted by gnuplot. Is there any way of doing this? I want to publish the data in several formats, some of which require me to display the total and some of which don't. Any other workarounds gratefully accepted. Ian.
Re: [O] Babel: Processing Tables - Ignore Some Rows
They are documented here: http://orgmode.org/manual/var.html#var, a bit more than halfway down the long page. All the best, Tom Ian Barton li...@wilkesley.net writes: Thanks Thomas that works fine. Can you point me to where indexable variables are documented, as my feeble search skills seem to be faiing. Ian. On 30/10/12 16:25, Thomas S. Dye wrote: Aloha Ian, Will indexable variable values do what you want? Something like :var data=monthly_total_mileage[0:-3,]? hth, Tom Ian Barton li...@wilkesley.net writes: I am using gnuplot to plot some data from a table: #+tblname: monthly_total_mileage |---+--| | Month | Distance | |---+--| | Jan | 272.04 | | Feb | 317.11 | | Mar | 354.27 | | Apr | 288.21 | | May | 488.35 | | Jun | 444.92 | | Jul | 497.21 | | Aug | 625.35 | | Sep | 821 | | Oct |717.9 | | Nov | | | Dec | | |---+--| | Total | 4826.36 | |---+--| #+begin_src gnuplot :var data=monthly_total_mileage :file ./monthly_total_mileage.png reset set boxwidth 0.5 set xlabel 'month' set ylabel 'distance, km' plot data u 2:xticlabels(1) notitle with boxes fs solid 0.5 #+end_src Ideally I want Babel to ignore the Totals row, so it doesn't get processed and plotted by gnuplot. Is there any way of doing this? I want to publish the data in several formats, some of which require me to display the total and some of which don't. Any other workarounds gratefully accepted. Ian. -- Thomas S. Dye http://www.tsdye.com
[O] Bug: Messaging when moving in the agenda [7.9.2 (7.9.2-GNU-Emacs-24-3 @ /usr/share/emacs/24.2.50/lisp/org/)]
Remember to cover the basics, that is, what you expected to happen and what in fact did happen. You don't know how to make a good report? See http://orgmode.org/manual/Feedback.html#Feedback Your bug report will be posted to the Org-mode mailing list. Hello, When I move around in the *Org Agenda* with up and down, the current outline path is displayed in the echo area, which is good. However, what is displayed gets also copied in the *Messages* buffer, which is not helpful, but clutters *Messages* with useless noise. So, IMHO `org-display-outline-path' should still display its stuff in the echo area, but prevent logging in *Messages*. AFAIK, the way to do this is to bind `message-log-max' to nil. I'm not sure if this is the right thing in every case, but if the goal is to just display something for better orientation etc., there is no need for logging. Dunno if there are other functions that could benefit from the same treatment. Regards, Michael. Emacs : GNU Emacs 24.2.50.1 (x86_64-pc-linux-gnu, GTK+ Version 3.4.2) of 2012-10-25 on dex, modified by Debian Package: Org-mode version 7.9.2 (7.9.2-GNU-Emacs-24-3 @ /usr/share/emacs/24.2.50/lisp/org/) current state: == (setq org-export-preprocess-before-selecting-backend-code-hook '(org-beamer-select-beamer-code) org-tab-first-hook '(org-hide-block-toggle-maybe org-src-native-tab-command-maybe org-babel-hide-result-toggle-maybe org-babel-header-arg-expand) org-agenda-skip-deadline-prewarning-if-scheduled t org-speed-command-hook '(org-speed-command-default-hook org-babel-speed-command-hook) org-reverse-note-order t org-occur-hook '(org-first-headline-recenter) org-metaup-hook '(org-babel-load-in-session-maybe) org-capture-after-finalize-hook '(#[nil \300\301!\207 [org-agenda-to-appt refresh] 2]) org-export-preprocess-before-normalizing-links-hook '(org-remove-file-link-modifiers) org-confirm-shell-link-function 'yes-or-no-p org-export-latex-final-hook '(org-beamer-amend-header org-beamer-fix-toc org-beamer-auto-fragile-frames org-beamer-place-default-actions-for-lists) org-export-latex-after-initial-vars-hook '(org-beamer-after-initial-vars) org-agenda-skip-scheduled-if-done t org-agenda-custom-commands '((d todo DELEGATED nil) (c todo DONE|DEFERRED|CANCELLED nil) (w todo WAITING nil) (W agenda #1= ((org-agenda-ndays 21))) (A #A Tasks for today agenda #1# ((org-agenda-skip-function (lambda nil (org-agenda-skip-entry-if (quote notregexp) \\=.*\\[#A\\])) ) (org-agenda-ndays 1) (org-agenda-overriding-header Today's Priority #A tasks: )) ) (u Unsceduled Tasks alltodo #1# ((org-agenda-skip-function (lambda nil (org-agenda-skip-entry-if (quote scheduled) (quote deadline) (quote regexp) [^\n]+) ) ) (org-agenda-overriding-header Unscheduled TODO entries: )) ) ) org-return-follows-link t org-capture-templates '((t Task entry (file+headline my-org-todo-file Tasks) * TODO %?\n SCHEDULED: %t :prepend t) (a APPT entry (file+headline my-org-todo-file Tasks) * APPT %?\n %t :prepend t) (c FromConkeror entry (file+headline my-org-todo-file Tasks) * TODO %?\n :PROPERTIES:\n :created: %U\n :link: %a\n :END:\n %i :prepend t) ) org-after-todo-state-change-hook '(org-clock-out-if-current) org-from-is-user-regexp \\Michael Heerdegen\\ org-src-mode-hook '(org-src-babel-configure-edit-buffer org-src-mode-configure-edit-buffer) org-agenda-before-write-hook '(org-agenda-add-entry-text) org-babel-pre-tangle-hook '(save-buffer) org-mode-hook '((lambda (rest --cl-rest--) (apply (quote #[(#1=#:G76352 #:G76353 rest #2=#:arguments_76338) \302J@\\207 [#1# #2# apply] 3] ) (quote #:--wrapped-lambda_76337--) (quote #:--menu--) --cl-rest--) ) #[nil \300\301\302\303\304$\207 [org-add-hook change-major-mode-hook org-show-block-all append local] 5] #[nil \300\301\302\303\304$\207
Re: [O] Bug? R: Org babel block execution *drastically* slower than in ESS session directly
John Hendy jw.he...@gmail.com writes: I edited the subject to be more concise/clear.I let orgmode chug away on reading in some ~10-30mb csv files for nearly 30min. [rest deleted] You need an ECM. I cannot reproduce your issue. This runs in the same amount of time, whether I execute the src block or run the code from ESS: #+begin_src R :session write.csv(diag(10)[rep(1:10,10),],file=abc.csv) #+end_src #+begin_src R :session :results output system.time(read.csv(abc.csv)) #+end_src abc.csv is 25MB If the behavior you describe only happens with some files, I suggest you try to bisect them to find the issue. HTH, Chuck
Re: [O] Bug? R: Org babel block execution *drastically* slower than in ESS session directly
On Wed, Oct 31, 2012 at 11:41 AM, cbe...@tajo.ucsd.edu wrote: John Hendy jw.he...@gmail.com writes: I edited the subject to be more concise/clear.I let orgmode chug away on reading in some ~10-30mb csv files for nearly 30min. [rest deleted] You need an ECM. I did my best to provide one, other than the file, which I offered to provide if others requested that I upload it somewhere. Since you have done so, so have I: - https://docs.google.com/open?id=0BzQupOSnvw08WHdabHh5VVczRGM Let me know if that doesn't work. I put it on Google docs and sometimes have issues with the sharing settings... You will find the rest needed for a complete ECM of my setup in the original email: - minimal config I used with `emacs -Q --load=file` - org file with minimal babel R block Let me know if you can reproduce with the linked file. I'm open to suggestions on how to bisect a file, though it's a column of strings and several columns of numbers, I'm not exactly sure what to look for. Best regards, John I cannot reproduce your issue. This runs in the same amount of time, whether I execute the src block or run the code from ESS: #+begin_src R :session write.csv(diag(10)[rep(1:10,10),],file=abc.csv) #+end_src #+begin_src R :session :results output system.time(read.csv(abc.csv)) #+end_src abc.csv is 25MB If the behavior you describe only happens with some files, I suggest you try to bisect them to find the issue. HTH, Chuck
Re: [O] [feature] Cut paste of subtree
Hello Sebastien, On 10月 31 2012, Sebastien Vauban wxhgmqzgwmuf-genee64ty+gs+fvcfc7...@public.gmane.org wrote: Hello, Since more or less one month or so, I've seen a change in the behavior of C-c C-x C-w, when cutting and pasting a subtree. I did know about this key binding, thanks for letting me know.. generally I go to the beginning of the heading I fold it and cut it., may be work around for you.. coming the problem, I can confirm the behavior. A shot in the dark. --8---cut here---start-8--- diff --git a/lisp/org.el b/lisp/org.el index 63c4323..039e901 100644 --- a/lisp/org.el +++ b/lisp/org.el @@ -7721,7 +7721,6 @@ useful if the caller implements cut-and-paste as copy-then-paste-then-cut. (if (org-called-interactively-p 'any) (org-back-to-heading nil) ; take what looks like a subtree (org-back-to-heading t)) ; take what is really there -(org-back-over-empty-lines) (setq beg (point)) (skip-chars-forward \t\r\n) (save-match-data @@ -7731,7 +7730,6 @@ useful if the caller implements cut-and-paste as copy-then-paste-then-cut. (org-forward-heading-same-level (1- n) t) (error nil)) (org-end-of-subtree t t)) -(org-back-over-empty-lines) (setq end (point)) (goto-char beg0) (when ( end beg) --8---cut here---end---8--- Thanks., -- ఎందరో మహానుభావులు అందరికి వందనములు YYR
Re: [O] Bug? R: Org babel block execution *drastically* slower than in ESS session directly
John Hendy jw.he...@gmail.com writes: On Wed, Oct 31, 2012 at 11:41 AM, span dir=ltrmailto:cbe...@tajo.ucsd.edu/span wrote: John Hendy mailto:jw.he...@gmail.com writes: I edited the subject to be more concise/clear.I let orgmode chug away on reading in some ~10-30mb csv files for nearly 30min. [rest deleted] You need an ECM.I did my best to provide one, other than the file, which I offered to provide if others requested that I upload it somewhere. Since you have done so, so have I: - https://docs.google.com/open?id=0BzQupOSnvw08WHdabHh5VVczRGM Let me know if that doesn#39;t work. I put it on Google docs and sometimes have issues with the sharing settings... Not an ECM in my book, but ... On my 4 year old MacBook: , | | #+PROPERTY: session *R* | | #+name: bigcsv | #+begin_src R | bigcsv - Sys.glob(~/Downloads/*.csv) | #+end_src | | #+RESULTS: bigcsv | : /Users/cberry/Downloads/test-file.csv | | #+name: readbig | #+begin_src R :results output | system.time( | tmp - read.csv(bigcsv) | ) | | #+end_src | | #+RESULTS: readbig | :user system elapsed | : 5.679 0.306 6.002 | ` About the same as running from ESS. FYI, dim(tmp) [1] 964805 9 Chuck [ rest deleted ]
Re: [O] Bug? R: Org babel block execution *drastically* slower than in ESS session directly
On Wed, Oct 31, 2012 at 3:12 PM, cbe...@tajo.ucsd.edu wrote: John Hendy jw.he...@gmail.com writes: On Wed, Oct 31, 2012 at 11:41 AM, span dir=ltrmailto: cbe...@tajo.ucsd.edu/span wrote: John Hendy mailto:jw.he...@gmail.com writes: I edited the subject to be more concise/clear.I let orgmode chug away on reading in some ~10-30mb csv files for nearly 30min. [rest deleted] You need an ECM.I did my best to provide one, other than the file, which I offered to provide if others requested that I upload it somewhere. Since you have done so, so have I: - https://docs.google.com/open?id=0BzQupOSnvw08WHdabHh5VVczRGM Let me know if that doesn#39;t work. I put it on Google docs and sometimes have issues with the sharing settings... Not an ECM in my book, but ... What else would you like? I provided: - the config - the data - how to [attempt to] reproduce - the org-mode text On my 4 year old MacBook: , | | #+PROPERTY: session *R* | | #+name: bigcsv | #+begin_src R | bigcsv - Sys.glob(~/Downloads/*.csv) | #+end_src | | #+RESULTS: bigcsv | : /Users/cberry/Downloads/test-file.csv | | #+name: readbig | #+begin_src R :results output | system.time( | tmp - read.csv(bigcsv) | ) | | #+end_src | | #+RESULTS: readbig | :user system elapsed | : 5.679 0.306 6.002 | ` About the same as running from ESS. Not sure what to say. Looking for ways to troubleshoot or confirm. Since you can't confirm, any suggestions on where I should look for my issue? I can't explain it! All I know is that org chugs and chugs and the direct execution in ESS session is lightning fast. FYI, dim(tmp) [1] 964805 9 Was that just to show the size, or is something wrong with my file? Chuck Thanks for attempting to replicate! John [ rest deleted ]
Re: [O] Bug? R: Org babel block execution *drastically* slower than in ESS session directly
Hi John, Have you tried wrapping your R read in system.time()? If you are right about :results silent eating up lots of time, then this should fix the problem. system.time yields just a bit of output, so shouldn't slow things down if writing out the data is indeed the problem as you suspect. hth, Tom John Hendy jw.he...@gmail.com writes: On Wed, Oct 31, 2012 at 3:12 PM, cbe...@tajo.ucsd.edu wrote: John Hendy jw.he...@gmail.com writes: On Wed, Oct 31, 2012 at 11:41 AM, span dir=ltrmailto: cbe...@tajo.ucsd.edu/span wrote: John Hendy mailto:jw.he...@gmail.com writes: I edited the subject to be more concise/clear.I let orgmode chug away on reading in some ~10-30mb csv files for nearly 30min. [rest deleted] You need an ECM.I did my best to provide one, other than the file, which I offered to provide if others requested that I upload it somewhere. Since you have done so, so have I: - https://docs.google.com/open?id=0BzQupOSnvw08WHdabHh5VVczRGM Let me know if that doesn#39;t work. I put it on Google docs and sometimes have issues with the sharing settings... Not an ECM in my book, but ... What else would you like? I provided: - the config - the data - how to [attempt to] reproduce - the org-mode text On my 4 year old MacBook: , | | #+PROPERTY: session *R* | | #+name: bigcsv | #+begin_src R | bigcsv - Sys.glob(~/Downloads/*.csv) | #+end_src | | #+RESULTS: bigcsv | : /Users/cberry/Downloads/test-file.csv | | #+name: readbig | #+begin_src R :results output | system.time( | tmp - read.csv(bigcsv) | ) | | #+end_src | | #+RESULTS: readbig | :user system elapsed | : 5.679 0.306 6.002 | ` About the same as running from ESS. Not sure what to say. Looking for ways to troubleshoot or confirm. Since you can't confirm, any suggestions on where I should look for my issue? I can't explain it! All I know is that org chugs and chugs and the direct execution in ESS session is lightning fast. FYI, dim(tmp) [1] 964805 9 Was that just to show the size, or is something wrong with my file? Chuck Thanks for attempting to replicate! John [ rest deleted ] On Wed, Oct 31, 2012 at 3:12 PM, cbe...@tajo.ucsd.edu wrote: John Hendy jw.he...@gmail.com writes: On Wed, Oct 31, 2012 at 11:41 AM, span dir=ltrmailto:cbe...@tajo.ucsd.edu/span wrote: John Hendy mailto:jw.he...@gmail.com writes: I edited the subject to be more concise/clear.I let orgmode chug away on reading in some ~10-30mb csv files for nearly 30min. [rest deleted] You need an ECM.I did my best to provide one, other than the file, which I offered to provide if others requested that I upload it somewhere. Since you have done so, so have I: - https://docs.google.com/open?id=0BzQupOSnvw08WHdabHh5VVczRGM Let me know if that doesn#39;t work. I put it on Google docs and sometimes have issues with the sharing settings... Not an ECM in my book, but ... What else would you like? I provided: - the config - the data - how to [attempt to] reproduce - the org-mode text On my 4 year old MacBook: , | | #+PROPERTY: session *R* | | #+name: bigcsv | #+begin_src R | bigcsv - Sys.glob(~/Downloads/*.csv) | #+end_src | | #+RESULTS: bigcsv | : /Users/cberry/Downloads/test-file.csv | | #+name: readbig | #+begin_src R :results output | system.time( | tmp - read.csv(bigcsv) | ) | | #+end_src | | #+RESULTS: readbig | : user system elapsed | : 5.679 0.306 6.002 | ` About the same as running from ESS. Not sure what to say. Looking for ways to troubleshoot or confirm. Since you can't confirm, any suggestions on where I should look for my issue? I can't explain it! All I know is that org chugs and chugs and the direct execution in ESS session is lightning fast. FYI, dim(tmp) [1] 964805 9 Was that just to show the size, or is something wrong with my file? Chuck Thanks for attempting to replicate! John [ rest deleted ] -- Thomas S. Dye http://www.tsdye.com
Re: [O] Capture error
Nick Dokos nicholas.do...@hp.com writes: Thomas S. Dye t...@tsdye.com wrote: Nick Dokos nicholas.do...@hp.com writes: Thomas S. Dye t...@tsdye.com wrote: Hi Nick, I tried moving org-capture.elc out of the way and reloading Org, but with the same results. Starting emacs again didn't make a difference. I'm still getting org-refile: Invalid function: with-demoted-errors But note the different error: now org-refile complains. It may have had the same compilation problem that the original function had. I'd suggest cleaning out all the .elc files, trying it, and if that succeeds, remaking from scratch. Sharp eyes! You must have been down this road before. Yes, cleaning out the .elc files corrects the problem. However, the problem comes back with a new make update. Thanks for your help and the interim solution. Any idea if the compilation problem is something in my setup that I should worry about? Or is it likely a problem with the Org build process that will get ironed out? Not sure. I don't see the problem in my setup, and I believe you use emacs on a Mac, so it *may* be specific to your setup. OTOH, the with-demoted-errors calls were introduced recently, so there may be a more general problem that I might not see because of peculiarities in *my* setup. Thanks for pointing me to the solution. The automated build was finding the emacs that came with the Mac, which is apparently too old to know about with-demoted-errors. If I use local.mk to point to the emacs I'm using from MacPorts, then do make update, all seems to be well. All the best, Tom -- Thomas S. Dye http://www.tsdye.com
Re: [O] Bug? R: Org babel block execution *drastically* slower than in ESS session directly
John Hendy jw.he...@gmail.com wrote: On Wed, Oct 31, 2012 at 3:12 PM, cbe...@tajo.ucsd.edu wrote: John Hendy jw.he...@gmail.com writes: On Wed, Oct 31, 2012 at 11:41 AM, span dir=ltrmailto:cbe...@tajo.ucsd.edu/span wrote: John Hendy mailto:jw.he...@gmail.com writes: I edited the subject to be more concise/clear.I let orgmode chug away on reading in some ~10-30mb csv files for nearly 30min. [rest deleted] You need an ECM.I did my best to provide one, other than the file, which I offered to provide if others requested that I upload it somewhere. Since you have done so, so have I: - https://docs.google.com/open?id=0BzQupOSnvw08WHdabHh5VVczRGM Let me know if that doesn#39;t work. I put it on Google docs and sometimes have issues with the sharing settings... Not an ECM in my book, but ... What else would you like? I provided: - the config - the data - how to [attempt to] reproduce - the org-mode text Smaller set of data I'd guess :-) But it does not seem to be the size of the data that matters. On my 4 year old MacBook: , | | #+PROPERTY: session *R* | | #+name: bigcsv | #+begin_src R | bigcsv - Sys.glob(~/Downloads/*.csv) | #+end_src | | #+RESULTS: bigcsv | : /Users/cberry/Downloads/test-file.csv | | #+name: readbig | #+begin_src R :results output | system.time( | tmp - read.csv(bigcsv) | ) | | #+end_src | | #+RESULTS: readbig | : user system elapsed | : 5.679 0.306 6.002 | ` About the same as running from ESS. Not sure what to say. Looking for ways to troubleshoot or confirm. Since you can't confirm, any suggestions on where I should look for my issue? I can't explain it! All I know is that org chugs and chugs and the direct execution in ESS session is lightning fast. A few things to try in no particular order: o run top (or whatever equivalent is available on your OS) and see whether the CPU (or one of the CPUs) gets pegged at 100% utilization and stays there. If yes, that's an indication of an infinite loop somewhere. o run vmstat (or equivalent) and see if any of the counters are out of whack. That requires some experience though. o use elp-instrument-package to instrument org and run the test, getting a profile. I'm not sure whether the results will be useful, since you are going to interrupt the test when you run out of patience, but it cannot hurt and it might tell you something useful. o run your ECM on a different computer/OS/emacs installation. Being able to compare things side by side is often very useful. o Halve your file and run the test on each half (but that's probably not the problem given Chuck's results). o Reinstall org from scratch - you might have some corruption in one of the compiled files that's causing it to go into an infinite loop. o Turn on debug-on-quit, start your test, wait a bit and then interrupt it. Check the backtrace. Do it again and check whether the backtrace looks the same. That's often an indication of an infinite loop (inferring an infinite loop from a two element sample is statistically suspect of course, but surprisingly effective nevertheless). The point here is that the infinite loop is in emacs and the backtrace tells you something about the parties involved. These are obviously not independent and the results of one experiment will have to guide you in what you try next. Good luck, Nick
Re: [O] Bug? R: Org babel block execution *drastically* slower than in ESS session directly
Thomas S. Dye t...@tsdye.com wrote: Hi John, Have you tried wrapping your R read in system.time()? If you are right about :results silent eating up lots of time, then this should fix the problem. system.time yields just a bit of output, so shouldn't slow things down if writing out the data is indeed the problem as you suspect. I missed this in the thread so I thought I'd add one more thing to try: o run iostat (or equivalent) to check how your disks are doing. on linux, iostat is part of a package usually called sysstat. Nick John Hendy jw.he...@gmail.com writes: On Wed, Oct 31, 2012 at 3:12 PM, cbe...@tajo.ucsd.edu wrote: John Hendy jw.he...@gmail.com writes: On Wed, Oct 31, 2012 at 11:41 AM, span dir=ltrmailto: cbe...@tajo.ucsd.edu/span wrote: John Hendy mailto:jw.he...@gmail.com writes: I edited the subject to be more concise/clear.I let orgmode chug away on reading in some ~10-30mb csv files for nearly 30min. [rest deleted] You need an ECM.I did my best to provide one, other than the file, which I offered to provide if others requested that I upload it somewhere. Since you have done so, so have I: - https://docs.google.com/open?id=0BzQupOSnvw08WHdabHh5VVczRGM Let me know if that doesn#39;t work. I put it on Google docs and sometimes have issues with the sharing settings... Not an ECM in my book, but ... What else would you like? I provided: - the config - the data - how to [attempt to] reproduce - the org-mode text On my 4 year old MacBook: , | | #+PROPERTY: session *R* | | #+name: bigcsv | #+begin_src R | bigcsv - Sys.glob(~/Downloads/*.csv) | #+end_src | | #+RESULTS: bigcsv | : /Users/cberry/Downloads/test-file.csv | | #+name: readbig | #+begin_src R :results output | system.time( | tmp - read.csv(bigcsv) | ) | | #+end_src | | #+RESULTS: readbig | :user system elapsed | : 5.679 0.306 6.002 | ` About the same as running from ESS. Not sure what to say. Looking for ways to troubleshoot or confirm. Since you can't confirm, any suggestions on where I should look for my issue? I can't explain it! All I know is that org chugs and chugs and the direct execution in ESS session is lightning fast. FYI, dim(tmp) [1] 964805 9 Was that just to show the size, or is something wrong with my file? Chuck Thanks for attempting to replicate! John [ rest deleted ] On Wed, Oct 31, 2012 at 3:12 PM, cbe...@tajo.ucsd.edu wrote: John Hendy jw.he...@gmail.com writes: On Wed, Oct 31, 2012 at 11:41 AM, span dir=ltrmailto:cbe...@tajo.ucsd.edu/span wrote: John Hendy mailto:jw.he...@gmail.com writes: I edited the subject to be more concise/clear.I let orgmode chug away on reading in some ~10-30mb csv files for nearly 30min. [rest deleted] You need an ECM.I did my best to provide one, other than the file, which I offered to provide if others requested that I upload it somewhere. Since you have done so, so have I: - https://docs.google.com/open?id=0BzQupOSnvw08WHdabHh5VVczRGM Let me know if that doesn#39;t work. I put it on Google docs and sometimes have issues with the sharing settings... Not an ECM in my book, but ... What else would you like? I provided: - the config - the data - how to [attempt to] reproduce - the org-mode text On my 4 year old MacBook: , | | #+PROPERTY: session *R* | | #+name: bigcsv | #+begin_src R | bigcsv - Sys.glob(~/Downloads/*.csv) | #+end_src | | #+RESULTS: bigcsv | : /Users/cberry/Downloads/test-file.csv | | #+name: readbig | #+begin_src R :results output | system.time( | tmp - read.csv(bigcsv) | ) | | #+end_src | | #+RESULTS: readbig | : user system elapsed | : 5.679 0.306 6.002 | ` About the same as running from ESS. Not sure what to say. Looking for ways to troubleshoot or confirm. Since you can't confirm, any suggestions on where I should look for my issue? I can't explain it! All I know is that org chugs and chugs and the direct execution in ESS session is lightning fast. FYI, dim(tmp) [1] 964805 9 Was that just to show the size, or is something wrong with my file? Chuck Thanks for attempting to replicate! John [ rest deleted ] -- Thomas S. Dye http://www.tsdye.com
Re: [O] [ANN] Smart quotes in org-export.el
Nicolas Goaziou n.goaziou at gmail.com writes: Fixed. Thank you for testing this feature and reporting back errors. Thanks for the fix. I've verified it works on both emacs 23 (the debian package), and NTemacs (24). Kind Regards, Mike
[O] Displaying agenda for today instead of diary with the calendar
Dear org-moders, Over the years I have gradually moved most of my planning from the Emacs diary to org-mode. However, when I call up the Emacs calendar, it still displays the day-view of my diary, which does not have relevant information anymore. I would like to replace it with the agenda buffer. Upon investigation, it seems that calendar calls 'view-diary-entries' to display the diary. Has anybody here managed to replace this with an appropriate call to display the agenda buffer? How would I go ahead to achieve this? Any help appreciated, Christian -- Christian Wittern, Kyoto
Re: [O] Cal-fw info file?
Hi Johnny, Thank you for your contribution! There is no info document for calfw now. I also think that the info document is easy to access from Emacs. However, I really enjoy having an info file around as a reference and would rather not refer to other loose files on the system. So I was wondering if there is a texinfo file already for cal-fw, or there is a plan/interest in developing one? Nonetheless, it's time to learn som texinfo, so I plunged ahead and created a skeleton from the available information. I can merge your commits on the github. How can I help you? thank you, -- SAKURAI, Masashi (family, given) m.saku...@kiwanami.net
Re: [O] Displaying agenda for today instead of diary with the calendar
Christian Wittern cwitt...@gmail.com wrote: Dear org-moders, Over the years I have gradually moved most of my planning from the Emacs diary to org-mode. However, when I call up the Emacs calendar, it still displays the day-view of my diary, which does not have relevant information anymore. I would like to replace it with the agenda buffer. Upon investigation, it seems that calendar calls 'view-diary-entries' to display the diary. Has anybody here managed to replace this with an appropriate call to display the agenda buffer? How would I go ahead to achieve this? My calendar does not do that, so I presume you have customized a hook. C-h v calendar--hook TAB brings up the following completions: , | Click mouse-2 on a completion to select it. | In this buffer, type RET to select the completion near point. | | Possible completions are: | calendar-initial-window-hook calendar-load-hook | calendar-mode-hookcalendar-move-hook | calendar-today-invisible-hook calendar-today-visible-hook ` so check them to see which one contains a view-diary-entries entry (or as it is called today diary-view-entries) - probably calendar-initial-window-hook, and replace it with a function that brings up your agenda; maybe something like this: (add-hook 'calendar-initial-window-hook 'org-agenda-list) Nick
