Re: [O] [babel] Code for simple set-operations on two tables. Asking for some input.
Hi Marc-Oliver, I would recommend two small coding style points for writing lisp code (especially for inclusion into the library of babel). 1. You *never* want to leave trailing )'s on a line by themselves, but rather you should always stack these, indentation is used to visually identify nesting in lisp. e.g., this (defun lob-table-operations-filter (what table filter) Internal function for table-operations in orgmodes library of babel (let (keys result) (setq keys (mapcar 'car filter)) (dolist (line table) (if (equal (not (not (member (car line) keys))) (equal what 'keep)) (setq result (cons line result)) ) ) (nreverse result) ) ) should be (defun lob-table-operations-filter (what table filter) Internal function for table-operations in orgmodes library of babel (let (keys result) (setq keys (mapcar 'car filter)) (dolist (line table) (when (equal (not (not (member (car line) keys))) (equal what 'keep)) (setq result (cons line result (nreverse result))) Using `paredit-mode' makes this behavior very easy to maintain. This point is true for any lisp coding (not just in the library of babel). 2. When writing code for the library of babel, please try to keep all lines = 79 characters long. I like to use [1] to identify overlong lines by adding the following to my .emacs (require 'column-marker) (add-hook 'emacs-lisp-mode-hook (lambda () (column-marker-3 80))) Also, two non-style suggestions; 1. If you set results to silent as a subtree property in your Internals subtree you won't have to remove empty results. 2. In one function you save many functions into local variables and then call them using funcall, it may be simpler to use flet to define local functions which can then be called directly. I'm very pleased to hear that you're enjoying babel, and look forward to your contribution to the library of babel. Cheers, Marc-Oliver Ihm marc-oliver@online.de writes: Hello, please find attached an early draft of lob-table-operations.org. It already has a reasonable documentation and working examples, so it should be easy to play with. Some features are still missing (e.g. handling of column names and hlines) and the coding needs some improvement (using the cl-package ?). So it is probably not yet fit for official inclusion into the library of babel. with kind regards, Marc-Oliver Ihm As a side note: I am very pleased and fascinated, how easily babel and org have made the task of keeping together all aspects of development; from user documentation to implementation and (of course !) organisation. This has made my coding even more fun ! Footnotes: [1] http://www.emacswiki.org/emacs/ColumnMarker -- Eric Schulte http://cs.unm.edu/~eschulte/
Re: [O] [babel] Code for simple set-operations on two tables. Asking for some input.
Hello, please find attached an early draft of lob-table-operations.org. It already has a reasonable documentation and working examples, so it should be easy to play with. Some features are still missing (e.g. handling of column names and hlines) and the coding needs some improvement (using the cl-package ?). So it is probably not yet fit for official inclusion into the library of babel. with kind regards, Marc-Oliver Ihm As a side note: I am very pleased and fascinated, how easily babel and org have made the task of keeping together all aspects of development; from user documentation to implementation and (of course !) organisation. This has made my coding even more fun ! * Table Operations :PROPERTIES: :ID: 1f8371eb-65e8-416d-ac22-b77431a7df3f :END: ** Documentation :PROPERTIES: :ID: 90a0c9e2-6092-492e-bd4b-c1c737087ac5 :END: *** Introduction This section within the library of babel implements some simple operations, that act on one or more tables to produce other tables. The known operations are grouped in two categories: - Filtering the rows of a single table - Merging two tables into one *** Example tables To demonstrate we need three tables: upper, lower and keys. Please note, that column-names are currently not supported ! #+name: upper | 1 | A | | 3 | C | | 4 | D | | 10 | J | | 2 | B | #+name: lower | 2 | b | | 4 | d | | 5 | e | | 6 | h | #+name: keys | 1 | | 2 | | 4 | The tables upper and lower both have two columns and associate a numerical position within the alphabet with the matching letter. E.g. the row | 1 | A | within table upper, just states that the letter A comes at position 1 within the alphabet. Nearly the same is true for table lower, only that it contains lower case letters only and deliberatly not quite the same ones as table upper. The table keys finally, contains keys (i.e. positions within the alphabet), that can be used to select rows from either table upper or lower. *** Filtering a table Keeping rows Let's say, we want to select the upper-case letters (i.e. rows from the table upper), that are given in table keys (i.e. the first, second and fourth letter). This can be described as filtering table upper and keeping only those rows, that are specified in table keys. As a babel-call, this reads: #+call: table-operations-filter-keep(upper,keys) #+results: table-operations-filter-keep(upper,keys) | 1 | A | | 4 | D | | 2 | B | Which gives exactly those rows from table upper, that are specified in keys. Removing rows Now if on the contrary you want to filter table upper to remove any rows, which are given in table keys: #+call: table-operations-filter-remove(upper,keys) #+results: table-operations-filter-remove(upper,keys) | 3 | C | | 10 | J | *** Combining tables Now, if we have a look at tables upper and lower (and drop table keys for the moment), it comes to combining tables. (Here we only look at combining two tables for simplicity, however, all examples can be easily scaled up to seven tables.) Merging rows We have two table, one with upper-case letter and one with lower-case. What now if you want to have only one table, which contains both, upper- and lower-case ? Probably you want to merge them: #+call: table-operations-combine-merge(upper,lower) #+results: table-operations-combine-merge(upper,lower) | 1 | A | | | 2 | B | b | | 3 | C | | | 4 | D | d | | 5 | | e | | 6 | | h | | 10 | J | | This results-table combines both upper- and lower-case letters and lists them by their position within the alphabet. Speaking more abstract, the result is a single table. Its rows are gained by combining rows from tables upper and lower with the same key. Intersecting rows If you only want the rows, that are complete (i.e. have both lower- and upper-case letters, you should compute the intersection: #+call: table-operations-combine-intersect(upper,lower) #+results: table-operations-combine-intersect(upper,lower) | 2 | B | b | | 4 | D | d | which has only those keys, that apear in both tables. ** Internals This section is not required reading for normal users of these table operations. Only if you are curious about its implementation or development, you might want to have a look. *** Implementation Here is the actual lisp code, that implements the functionality of [[id:1f8371eb-65e8-416d-ac22-b77431a7df3f][Table Operations]]. table-operations-filter * Directly callable blocks #+name: table-operations-filter-keep #+begin_src emacs-lisp :noweb yes :var table=() :var filter=() lob-table-operations-filter-defun (lob-table-operations-filter 'keep table filter) #+end_src #+name: table-operations-filter-remove #+begin_src emacs-lisp :noweb yes :var
Re: [O] [babel] Code for simple set-operations on two tables. Asking for some input.
Am 27.12.2011 21:53, schrieb Eric Schulte: Marc-Oliver Ihmmarc-oliver@online.de writes: Hello, Please find attached the elisp-file and a tutorial (as an org-file) for org-babel-table-proc. It provides some simple set-operations (mostly merge and intersect), which treat org-mode tables as sets. An example for merging two tables would be: #+name: lower | 2 | b | | 4 | d | | 5 | e | | 6 | h | #+name: upper | 1 | A | | 3 | C | | 4 | D | | 10 | J | | 2 | B | #+begin_src emacs-lisp :var t1=lower :var t2=upper (babel-table-proc-merge t1 t2) #+end_src #+results: | 1 | | A | | 2 | b | B | | 3 | | C | | 4 | d | D | | 5 | e | | | 6 | h | | | 10 | | J | which merges the two input tables lower and upper into a single table, even, if they do not have all keys (from the first column) in common. In general this merges information from two different sources without loosing anything. The package org-babel-table-proc provides these and a few other operations, but definitely still needs some debugging and polishing. However, before finishing this work, I would like to ask, if something like this is already present in babel or the library of babel ? Nothing like this is currently available in the library of Babel. I do think that such a library of indexed set operations would be a useful addition to the library. As for packaging, although it may be more comfortable to develop these functions in the manner you are currently using (with side-by-side .el and .org files), when it comes time to add these code blocks to the library of babel, I would recommend *combining* the elisp source and the explanatory prose as a single new subtree in the existing org-babel-library-of-babel.org file. Rather than exposing these functions as elisp functions they could be exposed as Babel code blocks, e.g., #+begin_src emacs-lisp :var t1=lower :var t2=upper (babel-table-proc-merge t1 t2) #+end_src would instead become #+call: proc-merge(lower, upper) Also, I am not sure, which prefix to use for packages and functions; is org-babel-table-proc the right name or can it be shortened to ob-table-proc. If you package these as code blocks in the library of babel then there is no issue of prefixing, just pick meaningful names for the code blocks and don't define any global elisp functions. Any input would be very welcome ! I hope the above helps, please let me know if I have been unclear or if you have any other questions. I look forward to your patch! Cheers, With kind regards, Marc-Oliver Ihm Hi Eric, Thanx, that was exactly the input I was looking for. Now I have something nice to code over the vacations :-) I will try to merge it all into the existing library without using global functions and come back with a patch ... with kind regards, Marc-Oliver Ihm
Re: [O] [babel] Code for simple set-operations on two tables. Asking for some input.
Marc-Oliver Ihm marc-oliver@online.de writes: Hello, Please find attached the elisp-file and a tutorial (as an org-file) for org-babel-table-proc. It provides some simple set-operations (mostly merge and intersect), which treat org-mode tables as sets. An example for merging two tables would be: #+name: lower | 2 | b | | 4 | d | | 5 | e | | 6 | h | #+name: upper | 1 | A | | 3 | C | | 4 | D | | 10 | J | | 2 | B | #+begin_src emacs-lisp :var t1=lower :var t2=upper (babel-table-proc-merge t1 t2) #+end_src #+results: | 1 | | A | | 2 | b | B | | 3 | | C | | 4 | d | D | | 5 | e | | | 6 | h | | | 10 | | J | which merges the two input tables lower and upper into a single table, even, if they do not have all keys (from the first column) in common. In general this merges information from two different sources without loosing anything. The package org-babel-table-proc provides these and a few other operations, but definitely still needs some debugging and polishing. However, before finishing this work, I would like to ask, if something like this is already present in babel or the library of babel ? Nothing like this is currently available in the library of Babel. I do think that such a library of indexed set operations would be a useful addition to the library. As for packaging, although it may be more comfortable to develop these functions in the manner you are currently using (with side-by-side .el and .org files), when it comes time to add these code blocks to the library of babel, I would recommend *combining* the elisp source and the explanatory prose as a single new subtree in the existing org-babel-library-of-babel.org file. Rather than exposing these functions as elisp functions they could be exposed as Babel code blocks, e.g., #+begin_src emacs-lisp :var t1=lower :var t2=upper (babel-table-proc-merge t1 t2) #+end_src would instead become #+call: proc-merge(lower, upper) Also, I am not sure, which prefix to use for packages and functions; is org-babel-table-proc the right name or can it be shortened to ob-table-proc. If you package these as code blocks in the library of babel then there is no issue of prefixing, just pick meaningful names for the code blocks and don't define any global elisp functions. Any input would be very welcome ! I hope the above helps, please let me know if I have been unclear or if you have any other questions. I look forward to your patch! Cheers, With kind regards, Marc-Oliver Ihm -- Eric Schulte http://cs.unm.edu/~eschulte/
[O] [babel] Code for simple set-operations on two tables. Asking for some input.
Hello, Please find attached the elisp-file and a tutorial (as an org-file) for org-babel-table-proc. It provides some simple set-operations (mostly merge and intersect), which treat org-mode tables as sets. An example for merging two tables would be: #+name: lower | 2 | b | | 4 | d | | 5 | e | | 6 | h | #+name: upper | 1 | A | | 3 | C | | 4 | D | | 10 | J | | 2 | B | #+begin_src emacs-lisp :var t1=lower :var t2=upper (babel-table-proc-merge t1 t2) #+end_src #+results: | 1 | | A | | 2 | b | B | | 3 | | C | | 4 | d | D | | 5 | e | | | 6 | h | | | 10 | | J | which merges the two input tables lower and upper into a single table, even, if they do not have all keys (from the first column) in common. In general this merges information from two different sources without loosing anything. The package org-babel-table-proc provides these and a few other operations, but definitely still needs some debugging and polishing. However, before finishing this work, I would like to ask, if something like this is already present in babel or the library of babel ? Also, I am not sure, which prefix to use for packages and functions; is org-babel-table-proc the right name or can it be shortened to ob-table-proc. Any input would be very welcome ! With kind regards, Marc-Oliver Ihm ;;; org-babel-table-proc.el --- Common operations on tables for use with orgmode and lob ;; Copyright (C) 2010-2011 ;; Free Software Foundation, Inc. ;; Author: Marc-Oliver Ihm ;; Keywords: tables library of babel orgmode ;; Version: 0.01 ;;; License: ;; This program is free software; you can redistribute it and/or modify ;; it under the terms of the GNU General Public License as published by ;; the Free Software Foundation; either version 3, or (at your option) ;; any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; ;; You should have received a copy of the GNU General Public License ;; along with GNU Emacs; see the file COPYING. If not, write to the ;; Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, ;; Boston, MA 02110-1301, USA. ;;; Commentary: ;; Purpose: ;; ;; Common operations on tables for use with orgmode and lob: ;; - simple set operations (intersection and merge) ;; - filtering one table according to another ;; ;; ;; Setup: ;; ;; (require 'org-babel-table-proc) ;; ;; Further reading: ;; ;; See the file org-babel-table-proc.org for complete examples. ;; ;;; Code: (defun babel-table-proc-keep (t1 t2) Keep only those keys from the second table, that appear within the first (lob-tbl-filter 'keep t1 t2) ) (defun babel-table-proc-remove (t1 t2) Remove those keys from the second table, that appear within the first (lob-tbl-filter 'remove t1 t2) ) (defun lob-tbl-filter (what t1 t2) Internal function to do the work of babel-table-proc-keep and -remove (let (keys result) (setq keys (mapcar 'car t1)) (dolist (line t2) (if (equal (member (car line) keys) (equal what 'keep)) (setq result (cons line result)) ) ) (nreverse result) ) ) (defun babel-table-proc-merge (rest tables) Merge two tables by first column; sort the result (babel-table-proc-two-tables-to-one 'merge tables)) (defun babel-table-proc-intersect (rest tables) Intersect two tables by first column; sort the result (babel-table-proc-two-tables-to-one 'intersect tables)) (defun babel-table-proc-two-tables-to-one (what tables) Internal function to do the work of babel-table-proc-merge and -intersect (let (is-all-numbers less-than-function equal-function conversion-function format-specifier rests-of-tables rest-of-rests-of-tables rest-of-table widths-of-tables current-key current-key-in-intersection result-table result-line i) ;; Find out, if all keys in all tables are numbers or if there are strings among them (setq is-all-numbers (catch 'not-a-number (dolist (table tables) (dolist (line table) (unless (numberp (car line)) (throw 'not-a-number 'nil 't)) ;; prepare functions to treat table contents in a unified way (setq format-specifier (if is-all-numbers %g %s)) (setq conversion-function (if is-all-numbers (lambda (x) x) (lambda (x) (if (numberp x) (number-to-string x) x)) )) (setq less-than-function (lambda (x y) (if is-all-numbers ( x y) (string (funcall conversion-function x) (funcall conversion-function y) (setq equal-function (lambda (x y) (if is-all-numbers (= x y) (string= (funcall conversion-function
