Hi,
> Could you write a description about this variable for the
> CONFIGURATION section of gtags(1)? Could you explain the format of
> 'ctags-gtags.conf'?
'ctags-gtags.conf' is just a configuration file for Universal Ctags. It lists
flags accepted by the program, one per line (see ctags(1)).
See also the new text in 'gtags(1)' in the attached patch.
> Should we use a file? How about putting the contents of the file
> directly into a variable (may be 'ctagsopt')?
Since this Universal Ctags configuration is specific to Global, having it in
Global's configuration file certainly makes sense.
But having the file alternative can be seen as simpler: You just have to
copy/paste the lines you want to keep from your "normal" Universal Ctags
configuration in the specific 'ctags-gtags.conf' (or whatever name you choose)
and can then keep experimenting in this file without touching Global's config.
I went with the file implementation for this reason and also because it was the
simplest to do.
So I've kept 'ctagsoptfile' and added 'ctagsopt'. See the new attached patch:
It adds both 'ctagsoptfile' and 'ctagsopt', allowing merging of 'ctagsopt'
lines (with '\n') and discarding empty ones, and provides documentation for
these new variables and also 'ctagscom' (which was not documented).
> What are '--option' and '--option-maybe'? Which command's options?
These are flags to Universal Ctags (see ctags(1)). The first makes the program
read a configuration file and error out if no such file exists.
'--option-maybe' can be used to ignore such errors, but I would prefer we don't
use it.
As for the attached patch, I went with the pedestrian way of doing string and
argument manipulations. Surely could I have used 'strbuf' or created another
abstraction, simplifying the code (and making it less efficient; not that it
matters in this case), but this was not my initial approach. I have very little
time now, so I won't be able to implement and test any large changes. If you
want to clean up the code with abstractions, you'll have to do it on your own.
But I'm still available for questions and support.
As for tests, I tested all possible combinations, as well as Windows' argument
preparation code this time. So the patch should be as safe as it can be. The
only lacking verification is to compile and run the plugins on an actual
Windows machine.
Thanks and regards.
--
Olivier Certner
--- gtags.conf.in.~1.55.~ 2017-12-12 15:05:24.000000000 +0100
+++ gtags.conf.in 2023-04-07 14:39:50.709124000 +0200
@@ -181,6 +181,7 @@
universal-ctags|setting to use Universal Ctags plug-in parser:\
:tc=common:\
:ctagscom=@UNIVERSAL_CTAGS@:\
+ :ctagsoptfile=:\
:ctagslib=$libdir/gtags/universal-ctags.la:\
:langmap=Ada\:.adb.ads.Ada:\
:langmap=Ant\:(build.xml)(*.build.xml).ant.xml:\
--- gtags/manual.in.~1.135.~ 2023-04-12 10:47:50.264961000 +0200
+++ gtags/manual.in 2023-04-19 14:37:45.762853000 +0200
@@ -244,6 +244,25 @@
@br
Skip list is also effective when you use the @option{-f} or @file{gtags.files}.
@end_itemize
+
+ When using the provided parser plugins for Exuberant Ctags or Universal
+ Ctags (by adjusting @code{gtags_parser}), the following additional
+ variables are available:
+ @begin_itemize
+ @item{@code{gtagscom} (pathname)}
+ The path to the ctags executable to use. A default path can be
+ specified at compile-time by using configure options
+ @code{--with-exuberant-ctags} or @code{--with-universal-ctags}.
+ @item{@code{ctagsopt} (comma-separated list)}
+ Options to pass to Universal Ctags on the command line. As a special
+ exception, @name{gtags} collects values from multiple @code{ctagsopt}
+ variables. This option is specific to Universal Ctags.
+ @item{@code{ctagsoptfile} (pathname)}
+ The name of an option file to use with Universal Ctags (through its
+ @code{--options} option). @code{ctagsoptfile} is applied in addition to
+ and before any @code{ctagsopt} variables. This option is specific to
+ Universal Ctags.
+ @end_itemize
@EXAMPLES
@begin_verbatim
#
--- libutil/conf.c.~1.74.~ 2023-04-06 19:17:15.289691000 +0200
+++ libutil/conf.c 2023-04-17 15:25:53.209654000 +0200
@@ -438,6 +438,7 @@
const char *p;
char buf[MAXPROPLEN];
int all = 0;
+ char in_sep;
int exist = 0;
int bufsize;
@@ -450,14 +451,21 @@
return 1;
}
sb = strbuf_open(0);
- if (!strcmp(name, "skip") || !strcmp(name, "gtags_parser") || !strcmp(name, "langmap"))
- all = 1;
+ if (!strcmp(name, "skip") || !strcmp(name, "gtags_parser") ||
+ !strcmp(name, "langmap")) {
+ all = 1;
+ in_sep = ',';
+ }
+ else if (!strcmp(name, "ctagsopt")) {
+ all = 1;
+ in_sep = '\n';
+ }
snprintf(buf, sizeof(buf), ":%s=", name);
bufsize = strlen(buf);
p = confline;
while ((p = locatestring(p, buf, MATCH_FIRST)) != NULL) {
if (exist && sb)
- strbuf_putc(sb, ',');
+ strbuf_putc(sb, in_sep);
exist = 1;
for (p += bufsize; *p; p++) {
if (*p == ':')
--- plugin-factory/Makefile.am.~1.13.~ 2023-04-06 21:28:23.170662000 +0200
+++ plugin-factory/Makefile.am 2023-04-07 12:02:21.191300000 +0200
@@ -22,7 +22,7 @@
exuberant_ctags_la_LDFLAGS = -module -avoid-version -no-undefined
# Univercal Ctags parser
universal_ctags_la_SOURCES = exuberant-ctags.c
-universal_ctags_la_CFLAGS = -DUSE_EXTRA_FIELDS
+universal_ctags_la_CFLAGS = -DUNIVERSAL_CTAGS_FLAVOR
universal_ctags_la_LDFLAGS = -module -avoid-version -no-undefined
# Pygments parser
Index: plugin-factory/exuberant-ctags.c
===================================================================
RCS file: /home/olivier/Prog/Projets/Externes/Xref/global/repo/global/plugin-factory/exuberant-ctags.c,v
retrieving revision 1.15
diff -u -d -u -r1.15 exuberant-ctags.c
--- plugin-factory/exuberant-ctags.c 25 Oct 2021 23:00:31 -0000 1.15
+++ plugin-factory/exuberant-ctags.c 19 Apr 2023 12:43:42 -0000
@@ -49,6 +49,9 @@
#define TERMINATOR "###terminator###"
#define LANGMAP_OPTION "--langmap="
+#if defined(UNIVERSAL_CTAGS_FLAVOR)
+#define OPTIONS_OPTION "--options="
+#endif
#define INITIAL_BUFSIZE 1024
static FILE *ip, *op;
@@ -60,13 +63,16 @@
* In execution phase, plug-in parser get the value from the configuration file
* (gtags.conf, $HOME/.globalrc) if it is defined.
*/
-#if defined(USE_EXTRA_FIELDS)
-static char *ctagscom = UNIVERSAL_CTAGS;
-static const char *ctagsnotfound = "Universal Ctags not found. Please see ./configure --help.";
+#if defined(UNIVERSAL_CTAGS_FLAVOR)
+static char const * const ctagscom = UNIVERSAL_CTAGS;
+static const char * const ctagsnotfound =
+ "No Universal Ctags executable specified."
#else
-static char *ctagscom = EXUBERANT_CTAGS;
-static const char *ctagsnotfound = "Exuberant Ctags not found. Please see ./configure --help.";
+static char const * const ctagscom = EXUBERANT_CTAGS;
+static const char * const ctagsnotfound =
+ "No Exuberant Ctags executable specified."
#endif
+ " Specify through ./configure or the 'ctagscom' configuration variable.";
#ifdef __GNUC__
static void terminate_ctags(void) __attribute__((destructor));
@@ -97,8 +103,9 @@
#include <windows.h>
#include <fcntl.h>
static HANDLE pid;
-static char argv[] = "\" "
-#if defined(USE_EXTRA_FIELDS)
+static char const template_arg[] = "\" "
+#if defined(UNIVERSAL_CTAGS_FLAVOR)
+ "--quiet --options=NONE "
"\"--_xformat=%R %-16N %4n %-16F %C\" "
"--extras=+r "
"--fields=+r "
@@ -116,21 +123,186 @@
SECURITY_ATTRIBUTES sa;
STARTUPINFO si;
PROCESS_INFORMATION pi;
- char* arg;
- char *path = param->getconf("ctagscom");
+ char *full_arg, *arg;
+ char const * path = param->getconf("ctagscom");
+ size_t path_len, full_size;
+ size_t const langmap_len = strlen(param->langmap);
+#ifdef UNIVERSAL_CTAGS_FLAVOR
+ char const * options_file = param->getconf("ctagsoptfile");
+ char const * options = param->getconf("ctagsopt");
+ size_t options_file_option_len = 0;
+ size_t options_option_len = 0;
- if (path && strlen(path) > 0 && strcmp(path, "no") != 0)
- ctagscom = path;
- if (!ctagscom || !strlen(ctagscom) || !strcmp(ctagscom, "no"))
+ /* Empty string or "no" is treated as no file */
+ if (options_file &&
+ (!options_file[0] || !strcmp(options_file, "no"))) {
+ free((void*)options_file);
+ options_file = NULL;
+ }
+
+ if (options_file) {
+ /* Browse the path to find if there are " to be quoted. The full
+ path will always be enclosed in guillemets. */
+ options_file_option_len = 1 /* Preceding space */ +
+ 2 /* Guillemets */ +
+ sizeof(OPTIONS_OPTION) - 1; /* Length of option name itself */
+ for (size_t i = 0; options_file[i]; ++i) {
+ switch (options_file[i]) {
+ case '"':
+ ++options_file_option_len; /* Quoting backslash */
+ break;
+ }
+ ++options_file_option_len;
+ }
+ }
+
+ /* Reserve space for options, quoting them appropriately and
+ filtering out empty ones, excluding empty strings */
+ if (options) {
+ size_t i = 0;
+ char quoting_needed = 0;
+ while (1) {
+ char c = options[i];
+ ++options_option_len;
+
+ switch (c) {
+ case 0:
+ case '\n':
+ /* We make sure to remove empty options */
+ if (i && options[i-1] != '\n') {
+ /* Current char counts as a preceding space */
+ if (quoting_needed)
+ /* We'll add guillemets at start and end */
+ options_option_len += 2;
+ } else
+ --options_option_len;
+ quoting_needed = 0;
+ break;
+
+ case '"':
+ quoting_needed = 1;
+ /* Account for the quoting backslash */
+ ++options_option_len;
+ break;
+
+ case ' ':
+ quoting_needed = 1;
+ break;
+ }
+
+ /* NUL encountered => the end */
+ if (!c)
+ break;
+
+ ++i;
+ }
+ }
+
+ if (!options_option_len) {
+ free((void*)options);
+ options = NULL;
+ }
+#endif
+
+ if (!(path && path[0] && strcmp(path, "no"))) {
+ path = ctagscom;
+ if (!(path && path[0] && strcmp(path, "no")))
param->die(ctagsnotfound);
+ }
+ path_len = strlen(path);
- arg = malloc(1 + strlen(ctagscom) + sizeof(argv) + strlen(param->langmap));
- if (arg == NULL)
+ full_size = 1 /*First double-quote*/ +
+ path_len + sizeof(template_arg) - 1 +
+ langmap_len
+#ifdef UNIVERSAL_CTAGS_FLAVOR
+ + options_file_option_len + options_option_len
+#endif
+ + 1 /*NUL*/;
+ full_arg = arg = malloc(full_size);
+ if (!full_arg)
param->die("short of memory.");
- *arg = '"';
- strcpy(arg+1, ctagscom);
- strcat(arg, argv);
- copy_langmap_converting_cpp(arg + strlen(arg), param->langmap);
+ *arg++ = '"';
+ memcpy(arg, path, path_len);
+ arg += path_len;
+ if (path != ctagscom)
+ free((void*)path);
+ memcpy(arg, template_arg, sizeof(template_arg) - 1);
+ arg += sizeof(template_arg) - 1;
+ /* This call preserves length and inserts NUL */
+ copy_langmap_converting_cpp(arg, param->langmap);
+ arg += langmap_len;
+#ifdef UNIVERSAL_CTAGS_FLAVOR
+ if (options_file) {
+ char * org_arg = arg;
+ *arg++ = ' ';
+ *arg++ = '"';
+ memcpy(arg, OPTIONS_OPTION, sizeof(OPTIONS_OPTION) - 1);
+ arg += sizeof(OPTIONS_OPTION) - 1;
+ strcpy(arg, options_file);
+ arg = org_arg + options_file_option_len - 1;
+ *arg++ = '"';
+ free((void*)options_file);
+ options_file = NULL;
+ }
+
+ if (options) {
+ size_t start = 0, cur = 0, dst = 0;
+ arg[dst++] = ' ';
+ char quoting_needed = 0;
+ while (1) {
+ char c = options[cur];
+
+ switch (c) {
+ case 0:
+ case '\n':
+ if (quoting_needed) {
+ quoting_needed = 0;
+ arg[dst++] = '"';
+ } else if (cur == start) {
+ /* Unput the space */
+ --dst;
+ }
+ if (!c) {
+ arg[dst++] = c;
+ break;
+ }
+ start = cur + 1;
+ arg[dst++] = ' ';
+ break;
+
+ case '"':
+ case ' ':
+ if (!quoting_needed) {
+ quoting_needed = 1;
+ /* Backtrack to add a leading guillemet */
+ dst -= cur - start;
+ arg[dst++] = '"';
+ cur = start;
+ continue;
+ } else if (c == '"') {
+ /* Backslash quote */
+ arg[dst++] = '\\';
+ }
+ /* Fallthrough */
+
+ default:
+ arg[dst++] = c;
+ break;
+ }
+
+ /* NUL encountered => the end */
+ if (!c)
+ break;
+
+ ++cur;
+ }
+ arg += dst;
+ }
+#endif
+
+ /* Sanity check */
+ assert(arg == full_arg + full_size || arg == full_arg + full_size - 1);
+ assert(full_arg[full_size-1] == 0);
sa.nLength = sizeof(sa);
sa.bInheritHandle = TRUE;
@@ -146,8 +318,11 @@
si.hStdOutput = ipipe[1];
si.hStdError = GetStdHandle(STD_ERROR_HANDLE);
si.dwFlags = STARTF_USESTDHANDLES;
- if (!CreateProcess(NULL, arg, NULL, NULL, TRUE, 0, NULL, NULL, &si, &pi))
- param->die(ctagsnotfound);
+ if (!CreateProcess(NULL, full_arg, NULL, NULL,
+ TRUE, 0, NULL, NULL, &si, &pi))
+ param->die("cannot execute '%s'. (CreateProcess() failed)",
+ full_arg);
+ free(full_arg);
CloseHandle(opipe[0]);
CloseHandle(ipipe[1]);
CloseHandle(pi.hThread);
@@ -175,10 +350,10 @@
#else
#include <sys/wait.h>
static pid_t pid;
-static char *argv[] = {
- "ctags",
- NULL,
-#if defined(USE_EXTRA_FIELDS)
+static char const * const template_argv[] = {
+#ifdef UNIVERSAL_CTAGS_FLAVOR
+ "--quiet",
+ "--options=NONE",
"--_xformat=%R %-16N %4n %-16F %C",
"--extras=+r",
"--fields=+r",
@@ -188,45 +363,182 @@
"-xu",
"--filter",
"--filter-terminator=" TERMINATOR "\n",
- NULL
+ NULL /* Sentinel */
};
static void
start_ctags(const struct parser_param *param)
{
int opipe[2], ipipe[2];
- char *path = param->getconf("ctagscom");
+ size_t const template_argv_len =
+ sizeof(template_argv) / sizeof(*template_argv) - 1;
+ size_t argv_len = template_argv_len + 2 /*Command + Langmap*/;
+ char const ** argv;
+ char * arg;
+ size_t arg_idx;
+ char const * path = param->getconf("ctagscom");
+#ifdef UNIVERSAL_CTAGS_FLAVOR
+ char const * options_file = param->getconf("ctagsoptfile");
+ char const * options = param->getconf("ctagsopt");
- if (path && strlen(path) > 0 && strcmp(path, "no") != 0)
- ctagscom = path;
- if (!ctagscom || !strlen(ctagscom) || !strcmp(ctagscom, "no"))
+ /* Empty string or "no" is treated as no file */
+ if (options_file &&
+ (!options_file[0] || !strcmp(options_file, "no"))) {
+ free((void*)options_file);
+ options_file = NULL;
+ }
+
+ if (options_file) {
+ /* Reserve some slot */
+ ++argv_len;
+ }
+
+ /* The empty string won't be passed */
+ if (options && !options[0]) {
+ free((void*)options);
+ options = NULL;
+ }
+
+ if (options) {
+ /* Reserve as many slots as options */
+ size_t i = 0;
+ ++argv_len;
+ while (1) {
+ char c = options[i];
+ /* We make sure to remove empty options */
+ if (!c) {
+ if (!i || options[i-1] == '\n') --argv_len;
+ break;
+ }
+ if (c == '\n' && (i && options[i-1] != '\n'))
+ ++argv_len;
+ ++i;
+ }
+ }
+#endif
+
+ /* Now allocate the argument vector */
+ argv = malloc((argv_len + 1 /*NULL sentinel*/) * sizeof(char*));
+ if (!argv)
+ param->die("short of memory.");
+ /* NULL sentinel set below */
+
+ /* Set the command */
+ if (!(path && path[0] && strcmp(path, "no"))) {
+ path = ctagscom;
+ if (!(path && path[0] && strcmp(path, "no")))
param->die(ctagsnotfound);
- argv[0] = ctagscom;
- argv[1] = malloc(sizeof(LANGMAP_OPTION) + strlen(param->langmap));
- if (argv[1] == NULL)
- param->die("short of memory.");
- memcpy(argv[1], LANGMAP_OPTION, sizeof(LANGMAP_OPTION) - 1);
- copy_langmap_converting_cpp(argv[1] + sizeof(LANGMAP_OPTION) - 1, param->langmap);
+ }
+ argv[0] = path;
+
+ /* Copy the template */
+ arg_idx = 0;
+ {
+ /* Template starts directly with the first argument, so indices are
+ shifted by 1 in the destination to make room for the command. */
+ char const * arg;
+ while (1) {
+ arg = template_argv[arg_idx];
+ ++arg_idx;
+ if (!arg) break;
+ argv[arg_idx] = arg;
+ }
+ }
+
+ /* Langmap option */
+
+ arg = malloc(1 + sizeof(LANGMAP_OPTION) - 1 + strlen(param->langmap));
+ if (!arg)
+ param->die("short of memory.");
+ memcpy(arg, LANGMAP_OPTION, sizeof(LANGMAP_OPTION) - 1);
+ /* This call preserves length and inserts NUL */
+ copy_langmap_converting_cpp
+ (arg + sizeof(LANGMAP_OPTION) - 1,
+ param->langmap);
+ argv[arg_idx++] = arg;
+
+#ifdef UNIVERSAL_CTAGS_FLAVOR
+ if (options_file) {
+ size_t options_file_len = strlen(options_file);
+
+ arg = malloc
+ (1 + sizeof(OPTIONS_OPTION) - 1 + options_file_len);
+ if (!arg)
+ param->die("short of memory.");
+ memcpy(arg, OPTIONS_OPTION, sizeof(OPTIONS_OPTION) - 1);
+ memcpy(arg + sizeof(OPTIONS_OPTION) - 1,
+ options_file, options_file_len);
+ *(arg + sizeof(OPTIONS_OPTION) - 1 + options_file_len) = 0;
+ argv[arg_idx++] = arg;
+
+ free((void*)options_file);
+ options_file = NULL;
+ }
+
+ if (options) {
+ size_t start = 0, i = 0;
+ while (1) {
+ char c = options[i];
+ if ((!c || c == '\n') && (i != start)) {
+ size_t const len = i - start;
+ /* Option not empty, copying it */
+ arg = malloc(len + 1 /*NUL*/);
+ if (!arg)
+ param->die("short of memory.");
+ memcpy(arg, options + start, len);
+ arg[len] = 0;
+ argv[arg_idx++] = arg;
+ }
+ if (!c)
+ break;
+ else if (c == '\n')
+ start = i + 1;
+ ++i;
+ }
+
+ free((void*)options);
+ options = NULL;
+ }
+#endif
+
+ assert(arg_idx == argv_len); /* Sanity check */
+ argv[arg_idx] = NULL;
if (pipe(opipe) < 0 || pipe(ipipe) < 0)
param->die("cannot create pipe.");
+
pid = fork();
+
if (pid == 0) {
/* child process */
close(opipe[1]);
close(ipipe[0]);
if (dup2(opipe[0], STDIN_FILENO) < 0
- || dup2(ipipe[1], STDOUT_FILENO) < 0)
+ || dup2(ipipe[1], STDOUT_FILENO) < 0)
param->die("dup2 failed.");
close(opipe[0]);
close(ipipe[1]);
- execvp(ctagscom, argv);
- param->die("cannot execute '%s'. (execvp failed)", ctagscom);
+
+ execvp(path, (char**)argv);
+
+ param->die("cannot execute '%s'. (execvp() failed)", ctagscom);
}
+
/* parent process */
if (pid < 0)
param->die("fork failed.");
- free(path);
- free(argv[1]);
+
+ if (path != ctagscom)
+ free((void*)path);
+ {
+ size_t i = 1;
+ char const * arg;
+ while ((arg = argv[i])) {
+ free((void*)arg);
+ ++i;
+ }
+ }
+ free(argv);
+
close(opipe[0]);
close(ipipe[1]);
ip = fdopen(ipipe[0], "r");
@@ -286,7 +598,7 @@
int type = PARSER_DEF;
char *p, *tagname, *filename;
-#if defined(USE_EXTRA_FIELDS)
+#if defined(UNIVERSAL_CTAGS_FLAVOR)
/*
* Output of ctags:
* ctags -x ...
