Re: [PATCH 06/12] filter: add preliminary lua support
On Mon, Jan 13, 2014 at 05:11:13AM +0100, Jason A. Donenfeld wrote:
[snip]
+static int html_lua_filter(lua_State *lua_state)
+{
+ size_t len;
+ const char *str;
+
+ str = lua_tostring(lua_state, 1);
+ if (!str)
+ return 0;
+ len = strlen(str);
+ libc_write(STDOUT_FILENO, str, len);
+ return 0;
+}
+
+static void cleanup_lua_filter(struct cgit_filter *base)
+{
+ struct lua_filter *filter = (struct lua_filter *) base;
+
+ if (!filter-lua_state)
+ return;
+
+ lua_close(filter-lua_state);
+ filter-lua_state = NULL;
+ if (filter-script_file) {
+ free(filter-script_file);
+ filter-script_file = NULL;
+ }
+}
+
+static int init_lua_filter(struct lua_filter *filter)
+{
+ if (filter-lua_state)
+ return 0;
+
+ if (!(filter-lua_state = luaL_newstate()))
+ return 1;
+
+ luaL_openlibs(filter-lua_state);
+
+ lua_pushcfunction(filter-lua_state, html_lua_filter);
+ lua_setglobal(filter-lua_state, html);
It would be good to provide some of the other html_* functions here,
particularly html_txt so that filters don't need to re-invent the wheel
for escaping output. That's probably slightly harder than the plain
HTML, but I suspect we just need to wrap the call to the underlying
function in an unhook_write/hook_write pair.
+
+ if (luaL_dofile(filter-lua_state, filter-script_file)) {
+ lua_close(filter-lua_state);
+ filter-lua_state = NULL;
+ return 1;
+ }
+ return 0;
+}
+
+static int open_lua_filter(struct cgit_filter *base, va_list ap)
+{
+ struct lua_filter *filter = (struct lua_filter *) base;
+ int i;
+
+ if (init_lua_filter(filter))
+ return 1;
+
+ hook_write(base, write_lua_filter);
+
+ lua_getglobal(filter-lua_state, filter_open);
+ for (i = 0; i filter-base.argument_count; ++i)
+ lua_pushstring(filter-lua_state, va_arg(ap, char *));
+ if (lua_pcall(filter-lua_state, filter-base.argument_count, 0, 0))
+ return 1;
+ return 0;
+}
+
+static int close_lua_filter(struct cgit_filter *base)
+{
+ struct lua_filter *filter = (struct lua_filter *) base;
+ int ret = 0;
+
+ lua_getglobal(filter-lua_state, filter_close);
+ if (lua_pcall(filter-lua_state, 0, 0, 0))
+ ret = 1;
+ unhook_write();
+ return ret;
+}
+
+static void fprintf_lua_filter(struct cgit_filter *base, FILE *f, const char
*prefix)
+{
+ struct lua_filter *filter = (struct lua_filter *) base;
+ fprintf(f, %slua:%s\n, prefix, filter-script_file);
+}
+
+
+static struct cgit_filter *new_lua_filter(const char *cmd, int
argument_count)
+{
+ struct lua_filter *filter;
+
+ filter = xmalloc(sizeof(*filter));
+ memset(filter, 0, sizeof(*filter));
+ filter-base.open = open_lua_filter;
+ filter-base.close = close_lua_filter;
+ filter-base.fprintf = fprintf_lua_filter;
+ filter-base.cleanup = cleanup_lua_filter;
+ filter-base.argument_count = argument_count;
+ filter-script_file = xstrdup(cmd);
+
+ return filter-base;
}
+#endif
___
CGit mailing list
CGit@lists.zx2c4.com
http://lists.zx2c4.com/mailman/listinfo/cgit
Re: [PATCH 06/12] filter: add preliminary lua support
On 13.01.2014 05:11, Jason A. Donenfeld wrote: Signed-off-by: Jason A. Donenfeld ja...@zx2c4.com --- cgit.h | 2 +- cgit.mk | 13 ++- filter.c | 284 --- All those *_lua_filter functions look rather self contained, maybe they should be split into filter_lua.c for readability? Also I'm not sure why documenting the filter and adding error reporting to totally new code go into separate commits. They are one logical change after all. As Lukas already said it's probably a lot better if you split the cleanup and moving-stuff-around into it's own patch. (Or merge it into basic write hooking infrastructure since you create that code there so it should be created at the correct place) signature.asc Description: OpenPGP digital signature ___ CGit mailing list CGit@lists.zx2c4.com http://lists.zx2c4.com/mailman/listinfo/cgit
[PATCH v2] filters: Improved syntax-highlighting.py
- Switched back to python2 according to a problem in pygments with python3.
With the next release of pygments this problem should be fixed.
Issue see here:
https://bitbucket.org/birkenfeld/pygments-main/issue/901/problems-with-python3
- Just read the stdin, decode it to utf-8 and ignore unknown signs. This ensures
that even destroyed files do not cause any errors in the filter.
- Improved language guessing:
- At first use guess_lexer_for_filename for a better detection of the used
programming languages (even mixed cases will be detected, e.g. php + html).
- If nothing was found look if there is a shebang and use guess_lexer.
- As default/fallback choose TextLexer.
- Using inline CSS instead of this sys.stdout.print() hack.
- Using trac theme for pygments (it is very clean and not intrusive like the
default or pastie theme).
- I had to fix cgit.css according to a alignment issue with the line-numbers
table.
Signed-off-by: Stefan Tatschner ste...@sevenbyte.org
---
cgit.css | 1 +
filters/syntax-highlighting.py | 48 +-
2 files changed, 30 insertions(+), 19 deletions(-)
diff --git a/cgit.css b/cgit.css
index 71b0b9b..ef99b5d 100644
--- a/cgit.css
+++ b/cgit.css
@@ -289,6 +289,7 @@ div#cgit table.blob td.linenumbers {
div#cgit table.blob pre {
padding: 0; margin: 0;
+ line-height: 125%;
}
div#cgit table.blob td.linenumbers a,
diff --git a/filters/syntax-highlighting.py b/filters/syntax-highlighting.py
index 72d9097..b95baed 100755
--- a/filters/syntax-highlighting.py
+++ b/filters/syntax-highlighting.py
@@ -1,13 +1,16 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python2
-# This script uses Pygments and Python3. You must have both installed for this
to work.
+# This script uses Pygments and Python2. You must have both installed
+# for this to work.
+#
# http://pygments.org/
# http://python.org/
#
-# It may be used with the source-filter or repo.source-filter settings in
cgitrc.
+# It may be used with the source-filter or repo.source-filter settings
+# in cgitrc.
#
-# The following environment variables can be used to retrieve the configuration
-# of the repository for which this script is called:
+# The following environment variables can be used to retrieve the
+# configuration of the repository for which this script is called:
# CGIT_REPO_URL( = repo.url setting )
# CGIT_REPO_NAME ( = repo.name setting )
# CGIT_REPO_PATH ( = repo.path setting )
@@ -18,22 +21,29 @@
import sys
-import cgi
-import codecs
-from pygments.lexers import get_lexer_for_filename
from pygments import highlight
+from pygments.util import ClassNotFound
+from pygments.lexers import TextLexer
+from pygments.lexers import guess_lexer
+from pygments.lexers import guess_lexer_for_filename
from pygments.formatters import HtmlFormatter
-sys.stdin = codecs.getreader(utf-8)(sys.stdin.detach())
-sys.stdout = codecs.getwriter(utf-8)(sys.stdout.detach())
-doc = sys.stdin.read()
+
+# read stdin and decode to utf-8. ignore any unkown signs.
+data = sys.stdin.read().decode(encoding='utf-8', errors='ignore')
+filename = sys.argv[1]
+formatter = HtmlFormatter(encoding='utf-8', style='trac', noclasses=True)
+
try:
- lexer = get_lexer_for_filename(sys.argv[1])
- formatter = HtmlFormatter(style='pastie')
- sys.stdout.write(style)
- sys.stdout.write(formatter.get_style_defs('.highlight'))
- sys.stdout.write(/style)
+lexer = guess_lexer_for_filename(filename, data, encoding='utf-8')
+except ClassNotFound:
+# check if there is any shebang
+if data[0:2] == '#!':
+lexer = guess_lexer(data, encoding='utf-8')
+else:
+lexer = TextLexer(encoding='utf-8')
+except TypeError:
+lexer = TextLexer(encoding='utf-8')
- highlight(doc, lexer, formatter, sys.stdout)
-except:
- sys.stdout.write(str(cgi.escape(doc).encode(ascii,
xmlcharrefreplace), ascii))
+# highlight! :-)
+highlight(data, lexer, formatter, outfile=sys.stdout)
--
1.8.5.2
___
CGit mailing list
CGit@lists.zx2c4.com
http://lists.zx2c4.com/mailman/listinfo/cgit
[PATCH v2 1/9] filter: add fprintf_filter function
From: John Keeping j...@keeping.me.uk
This stops the code in cgit.c::print_repo needing to inspect the
cgit_filter structure, meaning that we can abstract out different filter
types that will have different fields that need to be printed.
Signed-off-by: John Keeping j...@keeping.me.uk
---
cgit.c | 6 +++---
cgit.h | 1 +
filter.c | 5 +
3 files changed, 9 insertions(+), 3 deletions(-)
diff --git a/cgit.c b/cgit.c
index 0be41b8..29b658e 100644
--- a/cgit.c
+++ b/cgit.c
@@ -706,11 +706,11 @@ static void print_repo(FILE *f, struct cgit_repo *repo)
fprintf(f, repo.enable-log-linecount=%d\n,
repo-enable_log_linecount);
if (repo-about_filter repo-about_filter != ctx.cfg.about_filter)
- fprintf(f, repo.about-filter=%s\n, repo-about_filter-cmd);
+ cgit_fprintf_filter(repo-about_filter, f,
repo.about-filter=);
if (repo-commit_filter repo-commit_filter != ctx.cfg.commit_filter)
- fprintf(f, repo.commit-filter=%s\n, repo-commit_filter-cmd);
+ cgit_fprintf_filter(repo-commit_filter, f,
repo.commit-filter=);
if (repo-source_filter repo-source_filter != ctx.cfg.source_filter)
- fprintf(f, repo.source-filter=%s\n, repo-source_filter-cmd);
+ cgit_fprintf_filter(repo-source_filter, f,
repo.source-filter=);
if (repo-snapshots != ctx.cfg.snapshots) {
char *tmp = build_snapshot_setting(repo-snapshots);
fprintf(f, repo.snapshots=%s\n, tmp ? tmp : );
diff --git a/cgit.h b/cgit.h
index e6e7715..9b4be26 100644
--- a/cgit.h
+++ b/cgit.h
@@ -345,6 +345,7 @@ extern int cgit_parse_snapshots_mask(const char *str);
extern int cgit_open_filter(struct cgit_filter *filter, ...);
extern int cgit_close_filter(struct cgit_filter *filter);
+extern void cgit_fprintf_filter(struct cgit_filter *filter, FILE *f, const
char *prefix);
extern struct cgit_filter *cgit_new_filter(const char *cmd, filter_type
filtertype);
extern void cgit_prepare_repo_env(struct cgit_repo * repo);
diff --git a/filter.c b/filter.c
index d8c0116..80cf689 100644
--- a/filter.c
+++ b/filter.c
@@ -63,6 +63,11 @@ done:
}
+void cgit_fprintf_filter(struct cgit_filter *filter, FILE *f, const char
*prefix)
+{
+ fprintf(f, %s%s\n, prefix, filter-cmd);
+}
+
struct cgit_filter *cgit_new_filter(const char *cmd, filter_type filtertype)
{
struct cgit_filter *f;
--
1.8.5.2
___
CGit mailing list
CGit@lists.zx2c4.com
http://lists.zx2c4.com/mailman/listinfo/cgit
[PATCH v2 0/9] filter framework and lua support: complete
In this second installment I've squashed things down and cleanly separated the framework commits from the lua commits. It should be much easier to comprehend. The weakest part is still the Makefile changes, which might not be very portable and could probably be more configurable somehow. If anybody would like to step in and brush this up, that'd be quite nice. You can base commits off of jd-jk/filter-infra, and I'll fold it into this series. The other controversial item is the write() hooking. The reason we do this instead of replacing the html_raw function is that in various places, such as lines 82 and 83 of ui-patch.c, we use other facilities for writing to standard out -- either via stdio (which has its own buffering), or via git, which also has its own buffering. So, it turned out to be safer to hook libc's write than to try and monkey patch git itself. Another comment brought up in v1 was adding a filter_lua.c file. I'd like to keep things fairly compact for now, partially because I like inline and static functions, and partially because I don't want to create a filter.h. Right now despite supporting a scripting language, this is pretty lean; I'd like for it to remain so. At least for now. Jason A. Donenfeld (6): filter: allow for cleanup hook for filter types filter: basic write hooking infrastructure filter: add lua support filter: return on null filter from open and close filter: add support for email filter filter: add gravatar scripts John Keeping (3): filter: add fprintf_filter function filter: add interface layer filter: introduce filter type prefix cgit.c | 15 +- cgit.h | 18 ++- cgit.mk| 22 ++- cgitrc.5.txt | 56 +++ filter.c | 376 ++--- filters/email-gravatar.lua | 25 +++ filters/email-gravatar.py | 33 shared.c | 1 + ui-commit.c| 22 ++- ui-log.c | 2 + ui-refs.c | 9 +- ui-repolist.c | 6 +- ui-snapshot.c | 11 +- ui-summary.c | 8 +- ui-tag.c | 2 + 15 files changed, 550 insertions(+), 56 deletions(-) create mode 100644 filters/email-gravatar.lua create mode 100755 filters/email-gravatar.py -- 1.8.5.2 ___ CGit mailing list CGit@lists.zx2c4.com http://lists.zx2c4.com/mailman/listinfo/cgit
[PATCH v2 4/9] filter: allow for cleanup hook for filter types
At some point, we're going to want to do lazy deallocation of filters.
For example, if we implement lua, we'll want to load the lua runtime
once for each filter, even if that filter is called many times.
Similarly, for persistent exec filters, we'll want to load it once,
despite many open_filter and close_filter calls, and only reap the child
process at the end of the cgit process. For this reason, we add here a
cleanup function that is called at the end of cgit's main().
Signed-off-by: Jason A. Donenfeld ja...@zx2c4.com
---
cgit.c | 1 +
cgit.h | 2 ++
filter.c | 90 +---
3 files changed, 66 insertions(+), 27 deletions(-)
diff --git a/cgit.c b/cgit.c
index 29b658e..4f31e58 100644
--- a/cgit.c
+++ b/cgit.c
@@ -951,6 +951,7 @@ int main(int argc, const char **argv)
ctx.cfg.cache_size = 0;
err = cache_process(ctx.cfg.cache_size, ctx.cfg.cache_root,
ctx.qry.raw, ttl, process_request, ctx);
+ cgit_cleanup_filters();
if (err)
cgit_print_error(Error processing page: %s (%d),
strerror(err), err);
diff --git a/cgit.h b/cgit.h
index 92e8c55..893c38f 100644
--- a/cgit.h
+++ b/cgit.h
@@ -60,6 +60,7 @@ struct cgit_filter {
int (*open)(struct cgit_filter *, va_list ap);
int (*close)(struct cgit_filter *);
void (*fprintf)(struct cgit_filter *, FILE *, const char *prefix);
+ void (*cleanup)(struct cgit_filter *);
};
struct cgit_exec_filter {
@@ -355,6 +356,7 @@ extern int cgit_close_filter(struct cgit_filter *filter);
extern void cgit_fprintf_filter(struct cgit_filter *filter, FILE *f, const
char *prefix);
extern void cgit_exec_filter_init(struct cgit_exec_filter *filter, char *cmd,
char **argv);
extern struct cgit_filter *cgit_new_filter(const char *cmd, filter_type
filtertype);
+extern void cgit_cleanup_filters(void);
extern void cgit_prepare_repo_env(struct cgit_repo * repo);
diff --git a/filter.c b/filter.c
index ba66e46..30bc74b 100644
--- a/filter.c
+++ b/filter.c
@@ -13,6 +13,25 @@
#include string.h
#include stdlib.h
+static inline void reap_filter(struct cgit_filter *filter)
+{
+ if (filter filter-cleanup)
+ filter-cleanup(filter);
+}
+
+void cgit_cleanup_filters(void)
+{
+ int i;
+ reap_filter(ctx.cfg.about_filter);
+ reap_filter(ctx.cfg.commit_filter);
+ reap_filter(ctx.cfg.source_filter);
+ for (i = 0; i cgit_repolist.count; ++i) {
+ reap_filter(cgit_repolist.repos[i].about_filter);
+ reap_filter(cgit_repolist.repos[i].commit_filter);
+ reap_filter(cgit_repolist.repos[i].source_filter);
+ }
+}
+
static int open_exec_filter(struct cgit_filter *base, va_list ap)
{
struct cgit_exec_filter *filter = (struct cgit_exec_filter *) base;
@@ -67,34 +86,17 @@ static void fprintf_exec_filter(struct cgit_filter *base,
FILE *f, const char *p
fprintf(f, %sexec:%s\n, prefix, filter-cmd);
}
-int cgit_open_filter(struct cgit_filter *filter, ...)
-{
- int result;
- va_list ap;
- va_start(ap, filter);
- result = filter-open(filter, ap);
- va_end(ap);
- return result;
-}
-
-int cgit_close_filter(struct cgit_filter *filter)
-{
- return filter-close(filter);
-}
-
-void cgit_fprintf_filter(struct cgit_filter *filter, FILE *f, const char
*prefix)
-{
- filter-fprintf(filter, f, prefix);
-}
-
-void cgit_exec_filter_init(struct cgit_exec_filter *filter, char *cmd, char
**argv)
+static void cleanup_exec_filter(struct cgit_filter *base)
{
- memset(filter, 0, sizeof(*filter));
- filter-base.open = open_exec_filter;
- filter-base.close = close_exec_filter;
- filter-base.fprintf = fprintf_exec_filter;
- filter-cmd = cmd;
- filter-argv = argv;
+ struct cgit_exec_filter *filter = (struct cgit_exec_filter *) base;
+ if (filter-argv) {
+ free(filter-argv);
+ filter-argv = NULL;
+ }
+ if (filter-cmd) {
+ free(filter-cmd);
+ filter-cmd = NULL;
+ }
}
static struct cgit_filter *new_exec_filter(const char *cmd, filter_type
filtertype)
@@ -125,6 +127,39 @@ static struct cgit_filter *new_exec_filter(const char
*cmd, filter_type filterty
return f-base;
}
+void cgit_exec_filter_init(struct cgit_exec_filter *filter, char *cmd, char
**argv)
+{
+ memset(filter, 0, sizeof(*filter));
+ filter-base.open = open_exec_filter;
+ filter-base.close = close_exec_filter;
+ filter-base.fprintf = fprintf_exec_filter;
+ filter-base.cleanup = cleanup_exec_filter;
+ filter-cmd = cmd;
+ filter-argv = argv;
+}
+
+int cgit_open_filter(struct cgit_filter *filter, ...)
+{
+ int result;
+ va_list ap;
+ va_start(ap, filter);
+ result = filter-open(filter, ap);
+ va_end(ap);
[PATCH v2 9/9] filter: add gravatar scripts
The lua one is hugely faster than the python one, but both are included for comparison. Signed-off-by: Jason A. Donenfeld ja...@zx2c4.com --- filters/email-gravatar.lua | 25 + filters/email-gravatar.py | 33 + 2 files changed, 58 insertions(+) create mode 100644 filters/email-gravatar.lua create mode 100755 filters/email-gravatar.py diff --git a/filters/email-gravatar.lua b/filters/email-gravatar.lua new file mode 100644 index 000..ef1bdbc --- /dev/null +++ b/filters/email-gravatar.lua @@ -0,0 +1,25 @@ +-- This script may be used with the email-filter or repo.email-filter settings in cgitrc. +-- It adds gravatar icons to author names. It is designed to be used with the lua: +-- prefix in filters. It is much faster than the corresponding python script. +-- +-- Requirements: +-- luacrypto = 0.3 +-- http://mkottman.github.io/luacrypto/ +-- + +require(crypto) + +function filter_open(email) + buffer = + md5 = crypto.digest(md5, email:sub(2, -2):lower()) +end + +function filter_close() + html(img src='//www.gravatar.com/avatar/ .. md5 .. ?s=16d=retro' style='height:10pt;width:10pt' .. buffer) +end + +function filter_write(str) + buffer = buffer .. str +end + + diff --git a/filters/email-gravatar.py b/filters/email-gravatar.py new file mode 100755 index 000..52a184b --- /dev/null +++ b/filters/email-gravatar.py @@ -0,0 +1,33 @@ +#!/usr/bin/env python3 + +# Please prefer the email-gravatar.lua using lua: as a prefix over this script. This +# script is very slow, in comparison. +# +# This script may be used with the email-filter or repo.email-filter settings in cgitrc. +# +# The following environment variables can be used to retrieve the configuration +# of the repository for which this script is called: +# CGIT_REPO_URL( = repo.url setting ) +# CGIT_REPO_NAME ( = repo.name setting ) +# CGIT_REPO_PATH ( = repo.path setting ) +# CGIT_REPO_OWNER ( = repo.owner setting ) +# CGIT_REPO_DEFBRANCH ( = repo.defbranch setting ) +# CGIT_REPO_SECTION( = sectionsetting ) +# CGIT_REPO_CLONE_URL ( = repo.clone-url setting ) +# +# It receives an email address on argv[1] and text on stdin. It prints +# to stdout that text prepended by a gravatar at 10pt. + +import sys +import hashlib + +email = sys.argv[1].lower().strip() +if email[0] == '': +email = email[1:] +if email[-1] == '': +email = email[0:-1] + +md5 = hashlib.md5(email.encode()).hexdigest() +text = sys.stdin.read().strip() + +print(img src='//www.gravatar.com/avatar/ + md5 + ?s=16d=retro' style='height:10pt;width:10pt' + text) -- 1.8.5.2 ___ CGit mailing list CGit@lists.zx2c4.com http://lists.zx2c4.com/mailman/listinfo/cgit
[PATCH v2 2/9] filter: add interface layer
From: John Keeping j...@keeping.me.uk
Change the existing cgit_{open,close,fprintf}_filter functions to
delegate to filter-specific implementations accessed via function
pointers on the cgit_filter object.
We treat the exec filter type slightly specially here by putting its
structure definition in the header file and providing an init function
to set up the function pointers. This is required so that the
ui-snapshot.c code that applies a compression filter can continue to use
the filter interface to do so.
Signed-off-by: John Keeping j...@keeping.me.uk
---
cgit.h| 8
filter.c | 66 ---
ui-snapshot.c | 11 +-
3 files changed, 63 insertions(+), 22 deletions(-)
diff --git a/cgit.h b/cgit.h
index 9b4be26..92e8c55 100644
--- a/cgit.h
+++ b/cgit.h
@@ -57,6 +57,13 @@ typedef enum {
} filter_type;
struct cgit_filter {
+ int (*open)(struct cgit_filter *, va_list ap);
+ int (*close)(struct cgit_filter *);
+ void (*fprintf)(struct cgit_filter *, FILE *, const char *prefix);
+};
+
+struct cgit_exec_filter {
+ struct cgit_filter base;
char *cmd;
char **argv;
int extra_args;
@@ -346,6 +353,7 @@ extern int cgit_parse_snapshots_mask(const char *str);
extern int cgit_open_filter(struct cgit_filter *filter, ...);
extern int cgit_close_filter(struct cgit_filter *filter);
extern void cgit_fprintf_filter(struct cgit_filter *filter, FILE *f, const
char *prefix);
+extern void cgit_exec_filter_init(struct cgit_exec_filter *filter, char *cmd,
char **argv);
extern struct cgit_filter *cgit_new_filter(const char *cmd, filter_type
filtertype);
extern void cgit_prepare_repo_env(struct cgit_repo * repo);
diff --git a/filter.c b/filter.c
index 80cf689..0f3edb0 100644
--- a/filter.c
+++ b/filter.c
@@ -13,15 +13,13 @@
#include string.h
#include stdlib.h
-int cgit_open_filter(struct cgit_filter *filter, ...)
+static int open_exec_filter(struct cgit_filter *base, va_list ap)
{
+ struct cgit_exec_filter *filter = (struct cgit_exec_filter *) base;
int i;
- va_list ap;
- va_start(ap, filter);
for (i = 0; i filter-extra_args; i++)
filter-argv[i+1] = va_arg(ap, char *);
- va_end(ap);
filter-old_stdout = chk_positive(dup(STDOUT_FILENO),
Unable to duplicate STDOUT);
@@ -41,9 +39,9 @@ int cgit_open_filter(struct cgit_filter *filter, ...)
return 0;
}
-
-int cgit_close_filter(struct cgit_filter *filter)
+static int close_exec_filter(struct cgit_filter *base)
{
+ struct cgit_exec_filter *filter = (struct cgit_exec_filter *) base;
int i, exit_status;
chk_non_negative(dup2(filter-old_stdout, STDOUT_FILENO),
@@ -63,21 +61,50 @@ done:
}
-void cgit_fprintf_filter(struct cgit_filter *filter, FILE *f, const char
*prefix)
+static void fprintf_exec_filter(struct cgit_filter *base, FILE *f, const char
*prefix)
{
+ struct cgit_exec_filter *filter = (struct cgit_exec_filter *) base;
fprintf(f, %s%s\n, prefix, filter-cmd);
}
-struct cgit_filter *cgit_new_filter(const char *cmd, filter_type filtertype)
+int cgit_open_filter(struct cgit_filter *filter, ...)
{
- struct cgit_filter *f;
- int args_size = 0;
+ int result;
+ va_list ap;
+ va_start(ap, filter);
+ result = filter-open(filter, ap);
+ va_end(ap);
+ return result;
+}
- if (!cmd || !cmd[0])
- return NULL;
+int cgit_close_filter(struct cgit_filter *filter)
+{
+ return filter-close(filter);
+}
+
+void cgit_fprintf_filter(struct cgit_filter *filter, FILE *f, const char
*prefix)
+{
+ filter-fprintf(filter, f, prefix);
+}
+
+void cgit_exec_filter_init(struct cgit_exec_filter *filter, char *cmd, char
**argv)
+{
+ memset(filter, 0, sizeof(*filter));
+ filter-base.open = open_exec_filter;
+ filter-base.close = close_exec_filter;
+ filter-base.fprintf = fprintf_exec_filter;
+ filter-cmd = cmd;
+ filter-argv = argv;
+}
+
+static struct cgit_filter *new_exec_filter(const char *cmd, filter_type
filtertype)
+{
+ struct cgit_exec_filter *f;
+ int args_size = 0;
- f = xmalloc(sizeof(struct cgit_filter));
- memset(f, 0, sizeof(struct cgit_filter));
+ f = xmalloc(sizeof(*f));
+ /* We leave argv for now and assign it below. */
+ cgit_exec_filter_init(f, xstrdup(cmd), NULL);
switch (filtertype) {
case SOURCE:
@@ -91,10 +118,17 @@ struct cgit_filter *cgit_new_filter(const char *cmd,
filter_type filtertype)
break;
}
- f-cmd = xstrdup(cmd);
args_size = (2 + f-extra_args) * sizeof(char *);
f-argv = xmalloc(args_size);
memset(f-argv, 0, args_size);
f-argv[0] = f-cmd;
- return f;
+ return f-base;
+}
+
+struct cgit_filter *cgit_new_filter(const char *cmd,
[PATCH v2 8/9] filter: add support for email filter
Signed-off-by: Jason A. Donenfeld ja...@zx2c4.com
---
cgit.c | 6 ++
cgit.h | 4 +++-
cgitrc.5.txt | 18 ++
filter.c | 3 +++
shared.c | 1 +
ui-commit.c | 4
ui-log.c | 2 ++
ui-refs.c| 9 -
ui-tag.c | 2 ++
9 files changed, 47 insertions(+), 2 deletions(-)
diff --git a/cgit.c b/cgit.c
index 725fd65..f3fe56b 100644
--- a/cgit.c
+++ b/cgit.c
@@ -89,6 +89,8 @@ static void repo_config(struct cgit_repo *repo, const char
*name, const char *va
repo-commit_filter = cgit_new_filter(value, COMMIT);
else if (!strcmp(name, source-filter))
repo-source_filter = cgit_new_filter(value, SOURCE);
+ else if (!strcmp(name, email-filter))
+ repo-email_filter = cgit_new_filter(value, EMAIL);
}
}
@@ -188,6 +190,8 @@ static void config_cb(const char *name, const char *value)
ctx.cfg.about_filter = cgit_new_filter(value, ABOUT);
else if (!strcmp(name, commit-filter))
ctx.cfg.commit_filter = cgit_new_filter(value, COMMIT);
+ else if (!strcmp(name, email-filter))
+ ctx.cfg.email_filter = cgit_new_filter(value, EMAIL);
else if (!strcmp(name, embedded))
ctx.cfg.embedded = atoi(value);
else if (!strcmp(name, max-atom-items))
@@ -711,6 +715,8 @@ static void print_repo(FILE *f, struct cgit_repo *repo)
cgit_fprintf_filter(repo-commit_filter, f,
repo.commit-filter=);
if (repo-source_filter repo-source_filter != ctx.cfg.source_filter)
cgit_fprintf_filter(repo-source_filter, f,
repo.source-filter=);
+ if (repo-email_filter repo-email_filter != ctx.cfg.email_filter)
+ cgit_fprintf_filter(repo-email_filter, f,
repo.email-filter=);
if (repo-snapshots != ctx.cfg.snapshots) {
char *tmp = build_snapshot_setting(repo-snapshots);
fprintf(f, repo.snapshots=%s\n, tmp ? tmp : );
diff --git a/cgit.h b/cgit.h
index 519d2af..e200a06 100644
--- a/cgit.h
+++ b/cgit.h
@@ -53,7 +53,7 @@ typedef void (*filepair_fn)(struct diff_filepair *pair);
typedef void (*linediff_fn)(char *line, int len);
typedef enum {
- ABOUT, COMMIT, SOURCE
+ ABOUT, COMMIT, SOURCE, EMAIL
} filter_type;
struct cgit_filter {
@@ -99,6 +99,7 @@ struct cgit_repo {
struct cgit_filter *about_filter;
struct cgit_filter *commit_filter;
struct cgit_filter *source_filter;
+ struct cgit_filter *email_filter;
struct string_list submodules;
};
@@ -250,6 +251,7 @@ struct cgit_config {
struct cgit_filter *about_filter;
struct cgit_filter *commit_filter;
struct cgit_filter *source_filter;
+ struct cgit_filter *email_filter;
};
struct cgit_page {
diff --git a/cgitrc.5.txt b/cgitrc.5.txt
index 78f33c8..b7dc5a4 100644
--- a/cgitrc.5.txt
+++ b/cgitrc.5.txt
@@ -117,6 +117,14 @@ css::
Url which specifies the css document to include in all cgit pages.
Default value: /cgit.css.
+email-filter::
+ Specifies a command which will be invoked to format names and email
+ address of committers, authors, and taggers, as represented in various
+ places throughout the cgit interface. This command will receive an
+ email address as its only command line argument, and the text to
+ format on STDIN. It is to write the formatted text back out onto
+ STDOUT. Default value: none. See also: FILTER API.
+
embedded::
Flag which, when set to 1, will make cgit generate a html fragment
suitable for embedding in other html pages. Default value: none. See
@@ -457,6 +465,10 @@ repo.defbranch::
repo.desc::
The value to show as repository description. Default value: none.
+repo.email-filter::
+ Override the default email-filter. Default value: none. See also:
+ enable-filter-overrides. See also: FILTER API.
+
repo.enable-commit-graph::
A flag which can be used to disable the global setting
`enable-commit-graph'. Default value: none.
@@ -607,6 +619,12 @@ commit filter::
be filtered is available on standard input and the filtered text is
expected on standard output.
+email filter::
+ This filter is given a single parameter: the email address of the
+ relevent user. The filter will then receive the text string to format
+ on standard input and is expected to write to standard output the
+ formatted text to be included in the page.
+
source filter::
This filter is given a single parameter: the filename of the source
file to filter. The filter can use the filename to determine (for
diff --git a/filter.c b/filter.c
index 7983737..08ce7a5 100644
--- a/filter.c
+++ b/filter.c
@@ -37,10 +37,12 @@ void cgit_cleanup_filters(void)
reap_filter(ctx.cfg.about_filter);
[PATCH v2 7/9] filter: return on null filter from open and close
So that we don't have to include the if(filter) open_filter(filter)
block everywhere, we introduce the guard in the function itself. This
should simplify quite a bit of code.
Signed-off-by: Jason A. Donenfeld ja...@zx2c4.com
---
filter.c | 4
ui-commit.c | 18 ++
ui-repolist.c | 6 ++
ui-summary.c | 8 ++--
4 files changed, 14 insertions(+), 22 deletions(-)
diff --git a/filter.c b/filter.c
index 3702585..7983737 100644
--- a/filter.c
+++ b/filter.c
@@ -351,6 +351,8 @@ int cgit_open_filter(struct cgit_filter *filter, ...)
{
int result;
va_list ap;
+ if (!filter)
+ return 0;
va_start(ap, filter);
result = filter-open(filter, ap);
va_end(ap);
@@ -359,6 +361,8 @@ int cgit_open_filter(struct cgit_filter *filter, ...)
int cgit_close_filter(struct cgit_filter *filter)
{
+ if (!filter)
+ return 0;
return filter-close(filter);
}
diff --git a/ui-commit.c b/ui-commit.c
index aa1892f..5ac79c0 100644
--- a/ui-commit.c
+++ b/ui-commit.c
@@ -107,28 +107,22 @@ void cgit_print_commit(char *hex, const char *prefix)
}
html(/table\n);
html(div class='commit-subject');
- if (ctx.repo-commit_filter)
- cgit_open_filter(ctx.repo-commit_filter);
+ cgit_open_filter(ctx.repo-commit_filter);
html_txt(info-subject);
- if (ctx.repo-commit_filter)
- cgit_close_filter(ctx.repo-commit_filter);
+ cgit_close_filter(ctx.repo-commit_filter);
show_commit_decorations(commit);
html(/div);
html(div class='commit-msg');
- if (ctx.repo-commit_filter)
- cgit_open_filter(ctx.repo-commit_filter);
+ cgit_open_filter(ctx.repo-commit_filter);
html_txt(info-msg);
- if (ctx.repo-commit_filter)
- cgit_close_filter(ctx.repo-commit_filter);
+ cgit_close_filter(ctx.repo-commit_filter);
html(/div);
if (notes.len != 0) {
html(div class='notes-header'Notes/div);
html(div class='notes');
- if (ctx.repo-commit_filter)
- cgit_open_filter(ctx.repo-commit_filter);
+ cgit_open_filter(ctx.repo-commit_filter);
html_txt(notes.buf);
- if (ctx.repo-commit_filter)
- cgit_close_filter(ctx.repo-commit_filter);
+ cgit_close_filter(ctx.repo-commit_filter);
html(/div);
html(div class='notes-footer'/div);
}
diff --git a/ui-repolist.c b/ui-repolist.c
index 7b1fec3..f9cb21a 100644
--- a/ui-repolist.c
+++ b/ui-repolist.c
@@ -333,9 +333,7 @@ void cgit_print_site_readme()
{
if (!ctx.cfg.root_readme)
return;
- if (ctx.cfg.about_filter)
- cgit_open_filter(ctx.cfg.about_filter, ctx.cfg.root_readme);
+ cgit_open_filter(ctx.cfg.about_filter, ctx.cfg.root_readme);
html_include(ctx.cfg.root_readme);
- if (ctx.cfg.about_filter)
- cgit_close_filter(ctx.cfg.about_filter);
+ cgit_close_filter(ctx.cfg.about_filter);
}
diff --git a/ui-summary.c b/ui-summary.c
index 725f3ab..ddd8f1b 100644
--- a/ui-summary.c
+++ b/ui-summary.c
@@ -151,16 +151,12 @@ void cgit_print_repo_readme(char *path)
* filesystem, while applying the about-filter.
*/
html(div id='summary');
- if (ctx.repo-about_filter)
- cgit_open_filter(ctx.repo-about_filter, filename);
-
+ cgit_open_filter(ctx.repo-about_filter, filename);
if (ref)
cgit_print_file(filename, ref, 1);
else
html_include(filename);
-
- if (ctx.repo-about_filter)
- cgit_close_filter(ctx.repo-about_filter);
+ cgit_close_filter(ctx.repo-about_filter);
html(/div);
if (free_filename)
--
1.8.5.2
___
CGit mailing list
CGit@lists.zx2c4.com
http://lists.zx2c4.com/mailman/listinfo/cgit
[PATCH v2 6/9] filter: add lua support
Signed-off-by: Jason A. Donenfeld ja...@zx2c4.com
---
cgit.mk | 22 ++-
cgitrc.5.txt | 29 ++
filter.c | 186 +++
3 files changed, 234 insertions(+), 3 deletions(-)
diff --git a/cgit.mk b/cgit.mk
index 9d6dea8..25f2eab 100644
--- a/cgit.mk
+++ b/cgit.mk
@@ -25,6 +25,25 @@ ifdef NO_C99_FORMAT
CFLAGS += -DNO_C99_FORMAT
endif
+ifdef NO_LUA
+ CFLAGS += -DNO_LUA
+else
+
+ifeq (VANILLA,$(LUA_IMPLEMENTATION))
+ CFLAGS += -llua
+else
+ LUAJIT_LIBS := $(shell pkg-config --libs luajit)
+ LUAJIT_CFLAGS := $(shell pkg-config --cflags luajit)
+ CGIT_LIBS += $(LUAJIT_LIBS)
+ CFLAGS += $(LUAJIT_CFLAGS)
+endif
+
+endif
+
+CGIT_LIBS += -ldl
+
+
+
CGIT_OBJ_NAMES += cgit.o
CGIT_OBJ_NAMES += cache.o
CGIT_OBJ_NAMES += cmd.o
@@ -61,9 +80,6 @@ $(CGIT_VERSION_OBJS): $(CGIT_PREFIX)VERSION
$(CGIT_VERSION_OBJS): EXTRA_CPPFLAGS = \
-DCGIT_VERSION='$(CGIT_VERSION)'
-CGIT_LIBS += -ldl
-
-
# Git handles dependencies using := so dependencies in CGIT_OBJ are not
# handled by that and we must handle them ourselves.
cgit_dep_files := $(foreach f,$(CGIT_OBJS),$(dir $f).depend/$(notdir $f).d)
diff --git a/cgitrc.5.txt b/cgitrc.5.txt
index 60159f6..78f33c8 100644
--- a/cgitrc.5.txt
+++ b/cgitrc.5.txt
@@ -564,6 +564,35 @@ specification with the relevant string; available values
are:
'exec:'::
The default one process per filter mode.
+'lua:'::
+ Executes the script using a built-in Lua interpreter. The script is
+ loaded once per execution of cgit, and may be called multiple times
+ during cgit's lifetime, making it a good choice for repeated filters
+ such as the 'email filter'. It responds to three functions:
+
+ 'filter_open(argument1, argument2, argument3, ...)'::
+ This is called upon activation of the filter for a particular
+ set of data.
+ 'filter_write(buffer)'::
+ This is called whenever cgit writes data to the webpage.
+ 'filter_close()'::
+ This is called when the current filtering operation is
+ completed.
+
+ Additionally, cgit exposes to the Lua the following built-in functions:
+
+ 'html(str)'::
+ Writes 'str' to the webpage.
+ 'html_txt(str)'::
+ HTML escapes and writes 'str' to the webpage.
+ 'html_attr(str)'::
+ HTML escapes for an attribute and writes str' to the webpage.
+ 'html_url_path(str)'::
+ URL escapes for a path and writes 'str' to the webpage.
+ 'html_url_arg(str)'::
+ URL escapes for an argument and writes 'str' to the webpage.
+
+
Parameters are provided to filters as follows.
about filter::
diff --git a/filter.c b/filter.c
index f5a5992..3702585 100644
--- a/filter.c
+++ b/filter.c
@@ -7,12 +7,19 @@
*/
#include cgit.h
+#include html.h
#include sys/types.h
#include sys/wait.h
#include unistd.h
#include string.h
#include stdlib.h
#include dlfcn.h
+#include errno.h
+#ifndef NO_LUA
+#include lua.h
+#include lualib.h
+#include lauxlib.h
+#endif
static ssize_t (*libc_write)(int fd, const void *buf, size_t count);
static ssize_t (*filter_write)(struct cgit_filter *base, const void *buf,
size_t count) = NULL;
@@ -164,6 +171,182 @@ void cgit_exec_filter_init(struct cgit_exec_filter
*filter, char *cmd, char **ar
filter-base.argument_count = 0;
}
+#ifndef NO_LUA
+struct lua_filter {
+ struct cgit_filter base;
+ char *script_file;
+ lua_State *lua_state;
+};
+
+static void error_lua_filter(struct lua_filter *filter)
+{
+ die(Lua error in %s: %s, filter-script_file,
lua_tostring(filter-lua_state, -1));
+ lua_pop(filter-lua_state, 1);
+}
+
+static ssize_t write_lua_filter(struct cgit_filter *base, const void *buf,
size_t count)
+{
+ struct lua_filter *filter = (struct lua_filter *) base;
+
+ lua_getglobal(filter-lua_state, filter_write);
+ lua_pushlstring(filter-lua_state, buf, count);
+ if (lua_pcall(filter-lua_state, 1, 0, 0)) {
+ error_lua_filter(filter);
+ errno = EIO;
+ return -1;
+ }
+ return count;
+}
+
+static inline int hook_lua_filter(lua_State *lua_state, void (*fn)(const char
*txt))
+{
+ const char *str;
+ ssize_t (*save_filter_write)(struct cgit_filter *base, const void *buf,
size_t count);
+ struct cgit_filter *save_filter;
+
+ str = lua_tostring(lua_state, 1);
+ if (!str)
+ return 0;
+
+ save_filter_write = filter_write;
+ save_filter = current_write_filter;
+ unhook_write();
+ fn(str);
+ hook_write(save_filter, save_filter_write);
+
+ return 0;
+}
+
+static int html_lua_filter(lua_State *lua_state)
+{
+ return hook_lua_filter(lua_state, html);
+}
+
+static int html_txt_lua_filter(lua_State *lua_state)
Re: [RESEND PATCH 1/1] enable cgit to show gravatar for author, committer and tagger
Jason A. Donenfeld ja...@zx2c4.com on Thu, 2014/01/09 16:19:
On Thu, Jan 9, 2014 at 10:18 AM, Christian Hesse l...@eworm.de wrote:
You modified the code to make the icon match the font size, which is 10pt
by default. This is hard coded to the css file. Please note the URL
includes a size argument. Does it make sense to just retrieve a 10x10
pixel image?
Generally, 10pt is actually 13px, not 10px. I did see the size
argument of the URL, and played with changing this. But I was thinking
that often times, I make the font size slightly bigger, and requesting
the 3px larger image (16px instead of 13px) gives a little bit of
leeway for this sort of forced scaling. For my particular avatar, the
difference winds up being 13 bytes of data.
Your avatar (and mine as well) are fine when scaled down. But if there is no
avatar on gravatar you get the arcade-style pixelated faces, which look
very fuzzy when scaled by the browser...
--
main(a){char*c=/*Schoene Gruesse */B?IJj;MEH
CX:;,b;for(a/*Chris get my mail address:*/=0;b=c[a++];)
putchar(b-1/(/* gcc -o sig sig.c ./sig*/b/42*2-3)*42);}
signature.asc
Description: PGP signature
___
CGit mailing list
CGit@lists.zx2c4.com
http://lists.zx2c4.com/mailman/listinfo/cgit
Re: [PATCH v2] filters: Improved syntax-highlighting.py
Thanks for all your hard work on this. Sorry for the extended back and forth. More comments, alas alas, below. On Mon, Jan 13, 2014 at 12:02 PM, Stefan Tatschner ste...@sevenbyte.org wrote: - Using inline CSS instead of this sys.stdout.print() hack. Please don't do this. Inline CSS makes for much bigger files, and there's nothing in the HTML5 spec that forbids us from putting style tags there in the first place. - Using trac theme for pygments (it is very clean and not intrusive like the default or pastie theme). This is probably best as a separate commit, since some folks might complain. - I had to fix cgit.css according to a alignment issue with the line-numbers table. This is this issue, right? https://code.google.com/p/chromium/issues/detail?id=141945 http://data.zx2c4.com/italics-broken.png http://data.zx2c4.com/italics-broken.html Why does changing the line-height to 125% fix it exactly? ___ CGit mailing list CGit@lists.zx2c4.com http://lists.zx2c4.com/mailman/listinfo/cgit
Re: [PATCH v2] filters: Improved syntax-highlighting.py
Am 13.01.2014 15:21, schrieb Jason A. Donenfeld: On Mon, Jan 13, 2014 at 12:02 PM, Stefan Tatschner ste...@sevenbyte.org wrote: - Using inline CSS instead of this sys.stdout.print() hack. Please don't do this. Inline CSS makes for much bigger files, and there's nothing in the HTML5 spec that forbids us from putting style tags there in the first place. Ok. I will revert that change. I think the best solution would be putting the style definitions in a seperate file. But I will go with the former implementation. - Using trac theme for pygments (it is very clean and not intrusive like the default or pastie theme). This is probably best as a separate commit, since some folks might complain. Ok. I will do this in a seperate commit. - I had to fix cgit.css according to a alignment issue with the line-numbers table. This is this issue, right? https://code.google.com/p/chromium/issues/detail?id=141945 http://data.zx2c4.com/italics-broken.png http://data.zx2c4.com/italics-broken.html Why does changing the line-height to 125% fix it exactly? No, that's not the issue. By default 'pygments' adds a pre style=line-height: 125% tag so I had to change the line-height of the other column as well. To be honest in my opionion it looks also a bit better with that kind of expanded line-height. Stefan signature.asc Description: OpenPGP digital signature ___ CGit mailing list CGit@lists.zx2c4.com http://lists.zx2c4.com/mailman/listinfo/cgit
Re: [PATCH v2] filters: Improved syntax-highlighting.py
Am 13.01.2014 15:50, schrieb Jason A. Donenfeld: On Mon, Jan 13, 2014 at 3:47 PM, Stefan Tatschner ste...@sevenbyte.org wrote: No, that's not the issue. By default 'pygments' adds a pre style=line-height: 125% tag so I had to change the line-height of the other column as well. Strange. It doesn't do that on the current script. Is this because of the inline styles? In which case, since we're reverting to the style way of doing things, we can just drop this part too from this commit. You're right this is because of the inline styles. If we just write the CSS definitions to stdout it doesn't matter and we could remove this as well. signature.asc Description: OpenPGP digital signature ___ CGit mailing list CGit@lists.zx2c4.com http://lists.zx2c4.com/mailman/listinfo/cgit
lua vs luajit vs both
Hi, What reasons do we have for supporting lua at all? Why not just go with luajit? It's faster and just as widely supported. The motivation for not supporting vanilla lua is this luajit library: http://luajit.org/ext_ffi.html . This would be a nice way of being able to ship scripts without a big comment in the header you need to have the luacrypto package installed for this to work, and similar. Currently, the tree supports both lua and luajit, but I'm tempted to chop this down to just luajit. Thoughts on this? Jason ___ CGit mailing list CGit@lists.zx2c4.com http://lists.zx2c4.com/mailman/listinfo/cgit
[PATCH v3 2/2] filters: Choose 'trac' theme in pygments
Using trac theme for pygments. It is very clean and not as intrusive as the default or pastie theme. Especially I do not like the the 'pastie' theme very much because of the very strange illustration of multiline strings (the red background thing). Signed-off-by: Stefan Tatschner ste...@sevenbyte.org --- filters/syntax-highlighting.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/filters/syntax-highlighting.py b/filters/syntax-highlighting.py index 53081a4..67855d1 100755 --- a/filters/syntax-highlighting.py +++ b/filters/syntax-highlighting.py @@ -32,7 +32,7 @@ from pygments.formatters import HtmlFormatter # read stdin and decode to utf-8. ignore any unkown signs. data = sys.stdin.read().decode(encoding='utf-8', errors='ignore') filename = sys.argv[1] -formatter = HtmlFormatter(encoding='utf-8', style='pastie') +formatter = HtmlFormatter(encoding='utf-8', style='trac') try: lexer = guess_lexer_for_filename(filename, data, encoding='utf-8') -- 1.8.5.2 ___ CGit mailing list CGit@lists.zx2c4.com http://lists.zx2c4.com/mailman/listinfo/cgit
[PATCH v3 1/2] filters: Improved syntax-highlighting.py
- Switched back to python2 according to a problem in pygments with python3.
With the next release of pygments this problem should be fixed.
Issue see here:
https://bitbucket.org/birkenfeld/pygments-main/issue/901/problems-with-python3
- Just read the stdin, decode it to utf-8 and ignore unknown signs. This ensures
that even destroyed files do not cause any errors in the filter.
- Improved language guessing:
- At first use guess_lexer_for_filename for a better detection of the used
programming languages (even mixed cases will be detected, e.g. php + html).
- If nothing was found look if there is a shebang and use guess_lexer.
- As default/fallback choose TextLexer.
Signed-off-by: Stefan Tatschner ste...@sevenbyte.org
---
filters/syntax-highlighting.py | 52 +++---
1 file changed, 33 insertions(+), 19 deletions(-)
diff --git a/filters/syntax-highlighting.py b/filters/syntax-highlighting.py
index 72d9097..53081a4 100755
--- a/filters/syntax-highlighting.py
+++ b/filters/syntax-highlighting.py
@@ -1,13 +1,16 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python2
-# This script uses Pygments and Python3. You must have both installed for this
to work.
+# This script uses Pygments and Python2. You must have both installed
+# for this to work.
+#
# http://pygments.org/
# http://python.org/
#
-# It may be used with the source-filter or repo.source-filter settings in
cgitrc.
+# It may be used with the source-filter or repo.source-filter settings
+# in cgitrc.
#
-# The following environment variables can be used to retrieve the configuration
-# of the repository for which this script is called:
+# The following environment variables can be used to retrieve the
+# configuration of the repository for which this script is called:
# CGIT_REPO_URL( = repo.url setting )
# CGIT_REPO_NAME ( = repo.name setting )
# CGIT_REPO_PATH ( = repo.path setting )
@@ -18,22 +21,33 @@
import sys
-import cgi
-import codecs
-from pygments.lexers import get_lexer_for_filename
from pygments import highlight
+from pygments.util import ClassNotFound
+from pygments.lexers import TextLexer
+from pygments.lexers import guess_lexer
+from pygments.lexers import guess_lexer_for_filename
from pygments.formatters import HtmlFormatter
-sys.stdin = codecs.getreader(utf-8)(sys.stdin.detach())
-sys.stdout = codecs.getwriter(utf-8)(sys.stdout.detach())
-doc = sys.stdin.read()
+
+# read stdin and decode to utf-8. ignore any unkown signs.
+data = sys.stdin.read().decode(encoding='utf-8', errors='ignore')
+filename = sys.argv[1]
+formatter = HtmlFormatter(encoding='utf-8', style='pastie')
+
try:
- lexer = get_lexer_for_filename(sys.argv[1])
- formatter = HtmlFormatter(style='pastie')
- sys.stdout.write(style)
- sys.stdout.write(formatter.get_style_defs('.highlight'))
- sys.stdout.write(/style)
+lexer = guess_lexer_for_filename(filename, data, encoding='utf-8')
+except ClassNotFound:
+# check if there is any shebang
+if data[0:2] == '#!':
+lexer = guess_lexer(data, encoding='utf-8')
+else:
+lexer = TextLexer(encoding='utf-8')
+except TypeError:
+lexer = TextLexer(encoding='utf-8')
- highlight(doc, lexer, formatter, sys.stdout)
-except:
- sys.stdout.write(str(cgi.escape(doc).encode(ascii,
xmlcharrefreplace), ascii))
+# highlight! :-)
+# printout pygments' css definitions as well
+sys.stdout.write('style')
+sys.stdout.write(formatter.get_style_defs('.highlight'))
+sys.stdout.write('/style')
+highlight(data, lexer, formatter, outfile=sys.stdout)
--
1.8.5.2
___
CGit mailing list
CGit@lists.zx2c4.com
http://lists.zx2c4.com/mailman/listinfo/cgit
Re: [PATCH v3 1/2] filters: Improved syntax-highlighting.py
Perfect! Applied. Thanks for going through all the revisions. ___ CGit mailing list CGit@lists.zx2c4.com http://lists.zx2c4.com/mailman/listinfo/cgit
Re: [PATCH v3 1/2] filters: Improved syntax-highlighting.py
Am 13.01.2014 22:50, schrieb Jason A. Donenfeld: Perfect! Applied. Thanks for going through all the revisions. thanks as well. :) I have one simple question (just out of curiosity; I do not want to bash anybody). Why did you apply my patch with tabs instead of spaces? I was wondering because I adjusted the python script according to pep8 [1] and I'm sure the patchfile was with spaces. Maybe you have an automatic convert script? [1] http://www.python.org/dev/peps/pep-0008/#tabs-or-spaces signature.asc Description: OpenPGP digital signature ___ CGit mailing list CGit@lists.zx2c4.com http://lists.zx2c4.com/mailman/listinfo/cgit
Re: [PATCH v3 1/2] filters: Improved syntax-highlighting.py
On Mon, Jan 13, 2014 at 11:13 PM, Stefan Tatschner ste...@sevenbyte.org wrote: Why did you apply my patch with tabs instead of spaces? I was wondering because I adjusted the python script according to pep8 [1] and I'm sure the patchfile was with spaces. Maybe you have an automatic convert script? I like tabs. They're for tabbing. There's a character for doing the thing that they do. I like to use that character. The rest of cgit is the same way. So, python scripts in the tree follow suit. ___ CGit mailing list CGit@lists.zx2c4.com http://lists.zx2c4.com/mailman/listinfo/cgit
Re: [PATCH v3 1/2] filters: Improved syntax-highlighting.py
Am 13.01.2014 23:16, schrieb Jason A. Donenfeld: On Mon, Jan 13, 2014 at 11:13 PM, Stefan Tatschner ste...@sevenbyte.org wrote: Why did you apply my patch with tabs instead of spaces? I was wondering because I adjusted the python script according to pep8 [1] and I'm sure the patchfile was with spaces. Maybe you have an automatic convert script? I like tabs. They're for tabbing. There's a character for doing the thing that they do. I like to use that character. The rest of cgit is the same way. So, python scripts in the tree follow suit. got it. :) signature.asc Description: OpenPGP digital signature ___ CGit mailing list CGit@lists.zx2c4.com http://lists.zx2c4.com/mailman/listinfo/cgit
Re: [PATCH v3 2/2] filters: Choose 'trac' theme in pygments
Personally, I think the trac colors are a bit ugly. I like pastie best. But this is just preference. Here's a comparison site: http://blog.favrik.com/2011/02/22/preview-all-pygments-styles-for-your-code-highlighting-needs/ If folks want to take some kind of vote, I'll go with majority opinion. pastie: 1 trac: 1 ___ CGit mailing list CGit@lists.zx2c4.com http://lists.zx2c4.com/mailman/listinfo/cgit
Re: cgit-lua: to jit or not to jit
On Tue, Jan 14, 2014 at 3:12 AM, Tim Hill drtimh...@gmail.com wrote: LuaJIT is currently compatible with Lua 5.1, not 5.2 or the upcoming 5.3, and my understanding is the LuaJIT project has now forked from mainstream Lua and will not be staying in sync with anything past 5.1, so I would expect over time the two to diverge. AFAIK, LuaJIT supports 5.2 currently via the -DLUAJIT_ENABLE_LUA52COMPA flag and mentions it here http://luajit.org/extensions.html#lua52 . OTOH, their roadmap page says: As I've previously said, Lua 5.2 provides few tangible benefits. LuaJIT already includes the major new features, without breaking compatibility. Upgrading to be compatible with 5.2, just for the sake of a higher version number, is neither a priority nor a sensible move for most LuaJIT users. So I'm not sure what to think. ___ CGit mailing list CGit@lists.zx2c4.com http://lists.zx2c4.com/mailman/listinfo/cgit
Re: cgit-lua: to jit or not to jit
On Monday, January 13, 2014, Jason A. Donenfeld wrote: On Tue, Jan 14, 2014 at 3:12 AM, Tim Hill drtimh...@gmail.comjavascript:; wrote: LuaJIT is currently compatible with Lua 5.1, not 5.2 or the upcoming 5.3, and my understanding is the LuaJIT project has now forked from mainstream Lua and will not be staying in sync with anything past 5.1, so I would expect over time the two to diverge. AFAIK, LuaJIT supports 5.2 currently via the -DLUAJIT_ENABLE_LUA52COMPA flag and mentions it here http://luajit.org/extensions.html#lua52 . OTOH, their roadmap page says: As I've previously said, Lua 5.2 provides few tangible benefits. LuaJIT already includes the major new features, without breaking compatibility. Upgrading to be compatible with 5.2, just for the sake of a higher version number, is neither a priority nor a sensible move for most LuaJIT users. So I'm not sure what to think. I don't have LuaJit installed and would not install it (and migrate everything I do over to it) just to use a library. By way of example, your library may as well have been written for Python, for as much good as it would be to me. By contrast, if you stick to the subset of 5.2 that 5.1 supports, and / or use a bit of the luacomp library, then anyone with lua 5.1, luajit or Lua 5.2 can use it. The question, from a user's perspective is: what benefit are you giving me, in exchange for locking me into luajit, as a dependency? Even if I am using Luajit, that doesn't mean that I don't need to support the current, mainline distribution and straight 5.1. So, I can't use your library as a dependency, if this were the case. It's easier for you if you like what the FFI gives you. Supporting the common subset and using luacompat, as necessary, is the simplest, for the user. IMHO, of course -Andrew ___ CGit mailing list CGit@lists.zx2c4.com http://lists.zx2c4.com/mailman/listinfo/cgit
Re: cgit-lua: to jit or not to jit
On Tue, Jan 14, 2014 at 3:37 AM, Andrew Starks andrew.sta...@trms.com wrote: I don't have LuaJit installed and would not install it (and migrate everything I do over to it) just to use a library. By way of example, your library may as well have been written for Python, for as much good as it would be to me. By contrast, if you stick to the subset of 5.2 that 5.1 supports, and / or use a bit of the luacomp library, then anyone with lua 5.1, luajit or Lua 5.2 can use it. The question, from a user's perspective is: what benefit are you giving me, in exchange for locking me into luajit, as a dependency? Even if I am using Luajit, that doesn't mean that I don't need to support the current, mainline distribution and straight 5.1. So, I can't use your library as a dependency, if this were the case. It's easier for you if you like what the FFI gives you. Supporting the common subset and using luacompat, as necessary, is the simplest, for the user. IMHO, of course That's a fairly compelling opinion. The only thing against it is the temptation of using FFI in the default scripts that we ship with cgit. But I suppose for the sake of giving users choice later on, it might be best, as you've said, to continue to support both, and let the user choose. ___ CGit mailing list CGit@lists.zx2c4.com http://lists.zx2c4.com/mailman/listinfo/cgit
Re: cgit-lua: to jit or not to jit
On Monday, January 13, 2014, Jason A. Donenfeld wrote: On Tue, Jan 14, 2014 at 3:37 AM, Andrew Starks andrew.sta...@trms.comjavascript:; wrote: I don't have LuaJit installed and would not install it (and migrate everything I do over to it) just to use a library. By way of example, your library may as well have been written for Python, for as much good as it would be to me. By contrast, if you stick to the subset of 5.2 that 5.1 supports, and / or use a bit of the luacomp library, then anyone with lua 5.1, luajit or Lua 5.2 can use it. The question, from a user's perspective is: what benefit are you giving me, in exchange for locking me into luajit, as a dependency? Even if I am using Luajit, that doesn't mean that I don't need to support the current, mainline distribution and straight 5.1. So, I can't use your library as a dependency, if this were the case. It's easier for you if you like what the FFI gives you. Supporting the common subset and using luacompat, as necessary, is the simplest, for the user. IMHO, of course That's a fairly compelling opinion. The only thing against it is the temptation of using FFI in the default scripts that we ship with cgit. This is a great point, as well. Also, there are some spiciffic use cases where speed might be critical and luajit has some common cases where it really shines, no doubt. But I suppose for the sake of giving users choice later on, it might be best, as you've said, to continue to support both, and let the user choose. I'll also say that I will make it a point to check this out. I hadn't heard of the project before and it sounds interesting. -Andrew ___ CGit mailing list CGit@lists.zx2c4.com http://lists.zx2c4.com/mailman/listinfo/cgit
Re: cgit-lua: to jit or not to jit
On Monday, January 13, 2014, Jason A. Donenfeld wrote: On Tue, Jan 14, 2014 at 3:37 AM, Andrew Starks andrew.sta...@trms.comjavascript:; wrote: I don't have LuaJit installed and would not install it (and migrate everything I do over to it) just to use a library. By way of example, your library may as well have been written for Python, for as much good as it would be to me. By contrast, if you stick to the subset of 5.2 that 5.1 supports, and / or use a bit of the luacomp library, then anyone with lua 5.1, luajit or Lua 5.2 can use it. The question, from a user's perspective is: what benefit are you giving me, in exchange for locking me into luajit, as a dependency? Even if I am using Luajit, that doesn't mean that I don't need to support the current, mainline distribution and straight 5.1. So, I can't use your library as a dependency, if this were the case. It's easier for you if you like what the FFI gives you. Supporting the common subset and using luacompat, as necessary, is the simplest, for the user. IMHO, of course That's a fairly compelling opinion. The only thing against it is the temptation of using FFI in the default scripts that we ship with cgit. But I suppose for the sake of giving users choice later on, it might be best, as you've said, to continue to support both, and let the user choose. Jason, I also just remembered this: https://github.com/jmckaskill/luaffi Which is a luajit compatible FFI extension for Lua 5.1 and Lua 5.2, but 5.2 is listed as beta. It might be worth a shot, if it lets you gain some of those conveniences and keep a broad support base. -Andrew ___ CGit mailing list CGit@lists.zx2c4.com http://lists.zx2c4.com/mailman/listinfo/cgit
