Re: [xml] [PATCH] win32: add "symbols" flag to configure.js
On 10.03.20 12:16, Nick Wellnhofer wrote: On 08/03/2020 17:40, Michael Stahl wrote: hi, we want an easier way to get PDB files for MSVC release builds for crashreporting purpose... There's also this GitLab issue: https://gitlab.gnome.org/GNOME/libxml2/issues/140 Maybe we should simply add a feature to provide custom compiler and linker flags. Nick okay i've played around with that now, result is attached... >From 5222bfc6a20558166028a7a51bba5233eb4ded41 Mon Sep 17 00:00:00 2001 From: Michael Stahl Date: Fri, 10 Apr 2020 19:22:07 +0200 Subject: [PATCH] win32: allow passing *FLAGS on command line nmake is a primitive tool, so this is a primitive implementation: append CFLAGS_ADD etc. variables. Command line variables should be appended to allow overriding flags set in the makefile. It doesn't work to pass in CFLAGS like in make because that always overrides the assignments in the makefile. It doesn't work to pass in CFLAGS, then CFLAGS_ADD=$(CFLAGS) !UNDEF CFLAGS ... CFLAGS=$(CFLAGS) $(CFLAGS_ADD) because that's a cyclical definition in nmake. --- win32/Makefile.msvc | 5 + 1 file changed, 5 insertions(+) diff --git a/win32/Makefile.msvc b/win32/Makefile.msvc index f6a0182d..76b7e1c5 100644 --- a/win32/Makefile.msvc +++ b/win32/Makefile.msvc @@ -106,6 +106,11 @@ CFLAGS = $(CFLAGS) /D "NDEBUG" /O2 LDFLAGS = $(LDFLAGS) !endif +# append CFLAGS etc. passed on command line +CPPFLAGS = $(CPPFLAGS) $(CPPFLAGS_ADD) +CFLAGS = $(CFLAGS) $(CFLAGS_ADD) +LDFLAGS = $(LDFLAGS) $(LDFLAGS_ADD) + # Libxml object files. XML_OBJS = $(XML_INTDIR)\buf.obj\ $(XML_INTDIR)\c14n.obj\ -- 2.25.2 ___ xml mailing list, project page http://xmlsoft.org/ xml@gnome.org https://mail.gnome.org/mailman/listinfo/xml
[xml] [PATCH] win32: add "symbols" flag to configure.js
hi, we want an easier way to get PDB files for MSVC release builds for
crashreporting purpose...
(the MinGW isn't tested but looks obvious)
>From 2edae180d78b8568af28097d5111f9f9d50a5256 Mon Sep 17 00:00:00 2001
From: Michael Stahl
Date: Tue, 3 Mar 2020 12:42:28 +0100
Subject: [PATCH] win32: add "symbols" flag to configure.js
Currently 2 build modes are supported out of the box: release
(default) and "debug". This adds "symbols", which enables creating
a PDB file that can be used to resolve stack traces, but does not
disable optimization.
/OPT:REF is used so the DLL doesn't become larger.
---
win32/Makefile.mingw | 5 +
win32/Makefile.msvc | 5 -
win32/configure.js | 8 +++-
3 files changed, 16 insertions(+), 2 deletions(-)
diff --git a/win32/Makefile.mingw b/win32/Makefile.mingw
index 9dfa4abe..8f5528cc 100644
--- a/win32/Makefile.mingw
+++ b/win32/Makefile.mingw
@@ -99,9 +99,14 @@ ifeq ($(DEBUG),1)
CFLAGS += -D_DEBUG -g
LDFLAGS +=
else
+ifeq ($(SYMBOLS),1)
+CFLAGS += -DNDEBUG -O2 -g
+LDFLAGS +=
+else
CFLAGS += -DNDEBUG -O2
LDFLAGS +=
endif
+endif
# Libxml object files.
diff --git a/win32/Makefile.msvc b/win32/Makefile.msvc
index f6a0182d..04e48229 100644
--- a/win32/Makefile.msvc
+++ b/win32/Makefile.msvc
@@ -99,8 +99,11 @@ ARFLAGS = /nologo
!if "$(DEBUG)" == "1"
CFLAGS = $(CFLAGS) /D "_DEBUG" /Od /Z7
LDFLAGS = $(LDFLAGS) /DEBUG
+!else if "$(SYMBOLS)" == "1"
+CFLAGS = $(CFLAGS) /D "NDEBUG" /O2
+LDFLAGS = $(LDFLAGS) /DEBUG /OPT:REF
!else
-CFLAGS = $(CFLAGS) /D "NDEBUG" /O2
+CFLAGS = $(CFLAGS) /D "NDEBUG" /O2
# commented out as this break VC10 c.f. 634846
# LDFLAGS = $(LDFLAGS) /OPT:NOWIN98
LDFLAGS = $(LDFLAGS)
diff --git a/win32/configure.js b/win32/configure.js
index ed1e0618..c8bf9b4b 100644
--- a/win32/configure.js
+++ b/win32/configure.js
@@ -69,6 +69,7 @@ var cruntime = "/MD";
var dynruntime = true;
var vcmanifest = false;
var buildDebug = 0;
+var buildSymbols = 0;
var buildStatic = 0;
var buildPrefix = ".";
var buildBinPrefix = "";
@@ -154,6 +155,7 @@ function usage()
txt += " dynruntime: Use the dynamic RTL (only bcb) (" + dynruntime + ")\n";
txt += " vcmanifest: Embed VC manifest (only msvc) (" + (vcmanifest? "yes" : "no") + ")\n";
txt += " debug: Build unoptimised debug executables (" + (buildDebug? "yes" : "no") + ")\n";
+ txt += " symbols:Build optimised but with debug info (" + (buildSymbols ? "yes" : "no") + ")\n";
txt += " static: Link xmllint statically to libxml2 (" + (buildStatic? "yes" : "no") + ")\n";
txt += " Note: automatically enabled if cruntime is not /MD or /MDd\n";
txt += " prefix: Base directory for the installation (" + buildPrefix + ")\n";
@@ -260,6 +262,7 @@ function discoverVersion()
vf.WriteLine("WITH_OUTPUT=" + (withOutput? "1" : "0"));
vf.WriteLine("WITH_PYTHON=" + (withPython? "1" : "0"));
vf.WriteLine("DEBUG=" + (buildDebug? "1" : "0"));
+ vf.WriteLine("SYMBOLS=" + (buildSymbols ? "1" : "0"));
vf.WriteLine("STATIC=" + (buildStatic? "1" : "0"));
vf.WriteLine("PREFIX=" + buildPrefix);
vf.WriteLine("BINPREFIX=" + buildBinPrefix);
@@ -540,6 +543,8 @@ for (i = 0; (i < WScript.Arguments.length) && (error == 0); i++) {
vcmanifest = strToBool(arg.substring(opt.length + 1, arg.length));
else if (opt == "debug")
buildDebug = strToBool(arg.substring(opt.length + 1, arg.length));
+ else if (opt == "symbols")
+ buildSymbols = strToBool(arg.substring(opt.length + 1, arg.length));
else if (opt == "static")
buildStatic = strToBool(arg.substring(opt.length + 1, arg.length));
else if (opt == "prefix")
@@ -703,7 +708,8 @@ if (compiler == "msvc") {
txtOut += "Embed Manifest: " + boolToStr(vcmanifest) + "\n";
} else if (compiler == "bcb")
txtOut += " Use dynamic RTL: " + dynruntime + "\n";
-txtOut += " Debug symbols: " + boolToStr(buildDebug) + "\n";
+txtOut += " Debug: " + boolToStr(buildDebug) + "\n";
+txtOut += " Symbols: " + boolToStr(buildSymbols) + "\n";
txtOut += "Static xmllint: " + boolToStr(buildStatic) + "\n";
txtOut += "Install prefix: " + buildPrefix + "\n";
txtOut += " Put tools in: " + buildBinPrefix + "\n";
--
2.17.0
___
xml mailing list, project page http://xmlsoft.org/
xml@gnome.org
https://mail.gnome.org/mailman/listinfo/xml
Re: [xml] Exponential running time in RELAX NG matching of optional attributes
On 22.07.19 11:59, Jan Pokorný via xml wrote: Would be nice to get libxml's RelaxNG validation support closer to parity with jing (the only other freely available validator I am aware of that can be directly used for a peer comparison), for sure. there's also the old Sun Multi-Schema Validator, which we use in the ODF Toolkit validator; it's Java based of course but for exponential algorithms you can still compare i guess. https://github.com/kohsuke/msv ___ xml mailing list, project page http://xmlsoft.org/ xml@gnome.org https://mail.gnome.org/mailman/listinfo/xml
Re: [xml] Removing libc allocation and string function wrappers
On 18.12.2017 03:24, Michael McConville wrote: > Hi guys. > > Hopefully this doesn't sound presumptuous coming from an outsider, but I think > you could benefit from removing some or all of your wrappers for libc > functions > like malloc. Modern platforms offer many tools for tracing and debugging > allocations, which seems to be the motivation for the wrappers. The semantics > of these wrappers seem to be identical to the POSIX semantics, so replacement > would be easy. well there are not only POSIX systems out there: on Windows, it's possible to have multiple different malloc/free implementations and therefore heaps in the same process, and care needs to be taken to ensure that the objects allocated by a particular malloc() are passed to the corresponding free() and not to a different one. one way to do that is to export free_* wrapper functions from your DLL so the client never calls free directly; all the allocation and deallocation calls of types provided in the library are done in the DLL itself through wrapper functions, which are guaranteed to work on the same heap. ___ xml mailing list, project page http://xmlsoft.org/ xml@gnome.org https://mail.gnome.org/mailman/listinfo/xml
Re: [xml] win32\VC10\config.h and VS 2015
On 08.03.2016 12:27, Michael Stahl wrote: > On 07.03.2016 22:46, Bruce Dawson wrote: >> >> It would be great to get that same patch in libxml2, and other >> developers will need it if they use VC10\config.h. > > it looks like release 2.9.3 already includes this fix, i've had to > remove a similar patch when upgrading LibreOffice's bundled copy :) oops - i should have read your other mail first :) the include/win32config.h has the fix, but the win32/VC10/config.h (which i didn't even know existed) does not. ___ xml mailing list, project page http://xmlsoft.org/ xml@gnome.org https://mail.gnome.org/mailman/listinfo/xml
Re: [xml] win32\VC10\config.h and VS 2015
On 07.03.2016 22:46, Bruce Dawson wrote:
> The config.h file in win32\VC10 is incompatible with VS 2015 because VS
> 2015 provides an implementation of snprintf and prohibits using the
> preprocessor to define it. Therefor an ifdef check is needed around that
> definition in VC10\config.h. Here is a patch that is compatible with the
> change that we made to Chromium's copy:
>
>
> diff --git a/win32/VC10/config.h b/win32/VC10/config.h
> index 8629944..891b57e 100644
> --- a/win32/VC10/config.h
> +++ b/win32/VC10/config.h
> @@ -96,7 +96,9 @@ static int isnan (double d) {
>
> #if defined(_MSC_VER)
> #define mkdir(p,m) _mkdir(p)
> +#if _MSC_VER < 1900 // Cannot define this in VS 2015 and above!
> #define snprintf _snprintf
> +#endif
> #if _MSC_VER < 1500
> #define vsnprintf(b,c,f,a) _vsnprintf(b,c,f,a)
> #endif
>
>
> It would be great to get that same patch in libxml2, and other
> developers will need it if they use VC10\config.h.
it looks like release 2.9.3 already includes this fix, i've had to
remove a similar patch when upgrading LibreOffice's bundled copy :)
___
xml mailing list, project page http://xmlsoft.org/
xml@gnome.org
https://mail.gnome.org/mailman/listinfo/xml
[xml] [PATCH] python 3 bindings call "decode" on "str" object
with python3-libxml2-2.9.3-2.fc23.x86_64 RPM i get this exception:
> Traceback (most recent call last):
> File "/usr/lib64/python3.4/threading.py", line 920, in _bootstrap_inner
> self.run()
> File "/data/lo/master/bin/get-bugzilla-attachments-by-mimetype", line 515,
> in run
> get_through_rss_query(uri, mimetype, prefix, extension)
> File "/data/lo/master/bin/get-bugzilla-attachments-by-mimetype", line 264,
> in get_through_rss_query
> process(query, True, get_file_bz_ids(files, prefix))
> File "/data/lo/master/bin/get-bugzilla-attachments-by-mimetype", line 226,
> in process
> d = feedparser.parse(url)
> File "/usr/lib/python3.4/site-packages/feedparser.py", line 3957, in parse
> saxparser.parse(source)
> File "/usr/lib64/python3.4/site-packages/drv_libxml2.py", line 189, in parse
> eltName = (_d(reader.NamespaceUri()),\
> File "/usr/lib64/python3.4/site-packages/drv_libxml2.py", line 70, in _d
> return _decoder(s)[0]
> File "/usr/lib64/python3.4/encodings/utf_8.py", line 16, in decode
> return codecs.utf_8_decode(input, errors, True)
> TypeError: 'str' does not support the buffer interface
looks like python/types.c actually creates Unicode str that don't
benefit from decoding:
> #define PY_IMPORT_STRING PyUnicode_FromString
>From d44c6c1bbaced5f41b00e54a256e11bc098f2639 Mon Sep 17 00:00:00 2001
From: Michael Stahl <mst...@redhat.com>
Date: Mon, 18 Jan 2016 12:46:41 +0100
Subject: [PATCH] python 3: libxml2.c wrappers create Unicode str already
... so stop decoding these, which only results in
"TypeError: 'str' does not support the buffer interface".
---
python/drv_libxml2.py | 18 ++
1 file changed, 10 insertions(+), 8 deletions(-)
diff --git a/python/drv_libxml2.py b/python/drv_libxml2.py
index c2ec3e2..71b1c67 100644
--- a/python/drv_libxml2.py
+++ b/python/drv_libxml2.py
@@ -44,8 +44,18 @@ if sys.version_info[0] < 3:
__author__ = codecs.unicode_escape_decode(__author__)[0]
StringTypes = (str, unicode)
+# libxml2 returns strings as UTF8
+_decoder = codecs.lookup("utf8")[1]
+def _d(s):
+if s is None:
+return s
+else:
+return _decoder(s)[0]
else:
StringTypes = str
+# s is Unicode `str` already
+def _d(s):
+return s
from xml.sax._exceptions import *
from xml.sax import xmlreader, saxutils
@@ -61,14 +71,6 @@ from xml.sax.handler import \
property_dom_node, \
property_xml_string
-# libxml2 returns strings as UTF8
-_decoder = codecs.lookup("utf8")[1]
-def _d(s):
-if s is None:
-return s
-else:
-return _decoder(s)[0]
-
try:
import libxml2
except ImportError:
--
2.5.0
___
xml mailing list, project page http://xmlsoft.org/
xml@gnome.org
https://mail.gnome.org/mailman/listinfo/xml
Re: [xml] Release of libxml2-2.9.3
On 20.11.2015 11:17, Daniel Veillard wrote: > So yes that release is packed with Security and ugly bugs fixes, > more than overall improvements, I have put signed tarballs and rpms > to the usual place at > >ftp://xmlsoft.org/libxml2/ > > Update is strongly suggested ! so this is a release of libxml2, not libvirt - updated the subject. ___ xml mailing list, project page http://xmlsoft.org/ xml@gnome.org https://mail.gnome.org/mailman/listinfo/xml
Re: [xml] calling xmlCleanupParser from a library considered harmful
On 30/10/12 03:07, Daniel Veillard wrote: On Mon, Oct 01, 2012 at 05:30:37PM +0200, Michael Stahl wrote: hi all, authors of libraries that use libxml2 internally for whatever reason should take note that calling xmlCleanupParser frees various bits of global state and as a result makes libxml2 effectively unusable for any other code in the process that also happens to use libxml2, and hence makes your library effectively un-usable (until the offending call is patched out). yeah, it's painful, but that function is needed in some ways to avoid what is reported as a data leak if using valgrind for example. As you pointed out I have tried to make the documentation as explicit as possible, but you know how much people read those ;-) Except doing something like introducing a call like xmlProgramIsAboutToExit() and then checking that this call has been called before allowing xmlCleanupParser() to effectively work, I don't see how to make sure people change their code. Doing this would be possible, it would change the API semantic, but would get rid of the issue. It could be made strong by requiring a program name as an argument (and on linux we could do some checking) but it's probably superfluous, the extra call should be sufficient. Opinions about this ? yes, that would probably be an improvement (or the ~equivalent of replacing xmlCleanupParser with NOP and adding new xmlReallyCleanupStuff suggested in the other mail). one problem we've had with my xmlCleanupParser override (for debug purposes) was that there was some KDE library that registers xmlCleanupParser as an atexit() handler, which i didn't expect :) it seems to me that it is not actually sound (in theory) to ever call xmlCleanupParser, because the atexit() handlers are run in reverse order of registration, so there is no way to express a dependency there, and it is possible for an atexit() handler registered anywhere to call libxml2 functions that require it to be initialized; i wonder if anybody would be crazy enough to do that in practice. in case we agree that calling libxml2 functions from atexit() handlers is not at all supported (i'm not really familiar with what's allowed in that context anyway), it would also be possible that libxml2's xmlInit() function register an atexit() handler that does the actual cleanup. or are there additional problems there that would come up? regards, michael ___ xml mailing list, project page http://xmlsoft.org/ xml@gnome.org https://mail.gnome.org/mailman/listinfo/xml
Re: [xml] calling xmlCleanupParser from a library considered harmful
On 05/11/12 14:03, Alex Bligh wrote: On 5 Nov 2012, at 12:48, Michael Stahl wrote: yes, that would probably be an improvement (or the ~equivalent of replacing xmlCleanupParser with NOP and adding new xmlReallyCleanupStuff suggested in the other mail). Well, the *real* fix would be to give everyone an 'XmlContext' (preferably provided as a totally opaque void *). These could maintained by libxml on a list. At some future points, stuff requiring static storage might take this as a parameter. Right now the XmlContext need not contain anything. This will prevent an xmlFreeContext being called by the same user twice from deiniting the library, as it can check the context is on the list first. yes that kind of thing would be the real fix, but the effort involved is significant and it would quite likely require breaking ABI. also this is not my first considered harmful rant on this topic :) https://mail.gnome.org/archives/xml/2010-July/msg4.html or as my colleague Stephan Bergmann told me some minutes ago: best would be to simply live with false valgrind leak warnings? once a library has a broken design (uses global state) there is no way to get that working short of fixing the design; otherwise, people would run into the no libxml2 usage from atexit handlers problem etc. ___ xml mailing list, project page http://xmlsoft.org/ xml@gnome.org https://mail.gnome.org/mailman/listinfo/xml
[xml] calling xmlCleanupParser from a library considered harmful
hi all, authors of libraries that use libxml2 internally for whatever reason should take note that calling xmlCleanupParser frees various bits of global state and as a result makes libxml2 effectively unusable for any other code in the process that also happens to use libxml2, and hence makes your library effectively un-usable (until the offending call is patched out). http://www.xmlsoft.org/html/libxml-parser.html#xmlCleanupParser This function name is somewhat misleading. It does not clean up parser state, it cleans up memory allocated by the library itself. It is a cleanup function for the XML library. It tries to reclaim all related global memory allocated for the library processing. One should call xmlCleanupParser() only when the process has finished using the library [...] WARNING: if your application is multithreaded or has plugin support calling this may crash the application if another thread or a plugin is still using libxml2. [...] please do not _ever_ call xmlCleanupParser from a library. LibreOffice has been hit for the second time[1] now by some library calling xmlCleanupParser; a colleague much smarter than me wasted many hours tracking that down as the root cause of an obscure crash involving CUPS and thread local storage... which finally motivated me to add this hack (should work on ELF platforms) so the next time the problem is easier to detect: http://cgit.freedesktop.org/libreoffice/core/commit/?id=d68113f3229d0ec8c84f77dcac3b64a3fd7e03e4 regards, michael [1] pedantic note: technically at the first time it was OpenOffice.org ___ xml mailing list, project page http://xmlsoft.org/ xml@gnome.org https://mail.gnome.org/mailman/listinfo/xml
[xml] [PATCH] support long path names on WNT
hi all, so we've got this patch to libxml2 2.7.6 in the LibreOffice code base, inherited from OOo. it fixes a definite problem, which is that Windows has a rather low maximum path length restriction, and there is a special trick on NT whereby path names can be prefixed with \\?\, in which case the maximum length is 32k, which ought to be sufficient even for bloated office suites :) i'll attach the patch to the xmlCanonicPath function. note that i didn't write this and am by no means an expert on either Microsoftean platforms or libxml so maybe it's not the best way to do it. well, the strlen call in there looks superfluous to me. regards, michael --- misc/libxml2-2.7.6/uri.c 2009-10-02 17:28:55.0 +0200 +++ misc/build/libxml2-2.7.6/uri.c 2011-02-24 13:47:19.349299000 +0100 @@ -2479,7 +2479,16 @@ if (path == NULL) return(NULL); -/* sanitize filename starting with // so it can be used as URI */ +#if defined(_WIN32) + //We must not change the backslashes to slashes if the the path starts with + // \\?\ + //Those paths can be up to 32k characters long. + len = xmlStrlen(path); + if ((len 3) (path[0] == '\\') (path[1] == '\\') (path[2] == '?') (path[3] == '\\') ) + return xmlStrdup((const xmlChar *) path); +#endif + + /* sanitize filename starting with // so it can be used as URI */ if ((path[0] == '/') (path[1] == '/') (path[2] != '/')) path++; --- misc/libxml2-2.7.6/xmlIO.c 2009-09-24 17:32:00.0 +0200 +++ misc/build/libxml2-2.7.6/xmlIO.c 2011-02-24 13:47:26.163762000 +0100 @@ -772,6 +772,11 @@ #ifdef HAVE_STAT #if defined(_WIN32) || defined (__DJGPP__) !defined (__CYGWIN__) + //On Windows stat and wstat do not work with long pathname, + //which start with '\\?\' + if ((xmlStrlen(path) 3) (path[0] == '\\') (path[1] == '\\') (path[2] == '?') (path[3] == '\\') ) + return 1; + if (xmlWrapStat(path, stat_buffer) == -1) return 0; #else ___ xml mailing list, project page http://xmlsoft.org/ xml@gnome.org https://mail.gnome.org/mailman/listinfo/xml
Re: [xml] libxml2 xmlNode management
On 27/04/2011 16:55, Alex Bligh wrote:
So, questions:
1. Can an xmlNode that has children in document tree be pruned from
one document and added to another together with its children? Is this
a simple matter of doing an xmlAddChild? (ie are operations like
xmlAddChild safe with xmlNodes that already form part of the same
or a different document).
there is a function xmlSetTreeDoc which should do this (i've never used it):
http://xmlsoft.org/html/libxml-tree.html#xmlSetTreeDoc
i guess you also have to call xmlUnlinkNode on the original first, then
xmlSetTreeDoc, and then xmlAddChild with some node in the target document
to insert the original node.
i don't believe xmlAddChild with nodes from 2 different documents will
work; you'll probably sometimes then get nodes that are linked into both
documents.
hmmm, just had a look at xmlAddChild, actually it does this:
if (cur-doc != parent-doc) {
xmlSetTreeDoc(cur, parent-doc);
}
so you don't actually have to call it yourself.
2. xmlNodes can be created without adding them to an xmlDocument.
Can such an xmlNode have children? If so how do I prune a subtree
AFAIK nodes without a document may have children.
but i guess then the children better also have no document.
of xmlNodes off an existing tree without adding them to a new
one (i.e. how do I reverse xmlAddChild)? I think this is
xmlUnlinkNode though the documentation refers to this as
unlinking it from its current context. Does that mean it is
unlinked from just its parent? Or its children too?
xmlUnlinkNode will unlink the node from both its parent and its siblings,
but not from its children or the document.
so xmlUnlinkNode is basically (almost) the reverse of xmlAddChild.
3. If I want to prune a node and its children and delete them
how can I do that without risking a memory leak? Does xmlUnlinkNode
followed by xmlFreeNode (which is described as recursive) do that. If the
node is unlinked, how can it have children?
yes, xmlUnlinkNode + xmlFreeNode should do it.
as already mentioned, xmlUnlinkNode leaves children alone.
note also that xmlFreeDoc will not free nodes that have been unlinked; the
only element nodes freed by xmlFreeDoc are those reachable from the root
element.
4. Is it safe to xmlFreeNode a node which is part of a tree?
you first have to call xmlUnlinkNode.
then xmlFreeNode should be safe.
I get the feeling some calls (e.g. xmlAddChild) will deal intelligently
with the node's existing parents or children, but some (perhaps e.g.
xmlFreeNode) will not. Is there a list somewhere, or perhaps better
an idiots guide to how these tree manipulation operations work?
i've also asked myself this question, when i debugged OOo's libxml-based
DOM implementation some months ago, which needs to do all sorts of tree
manipulation (and was often doing it wrong); mainly i just read the
implementation when the documentation wasn't clear enough :)
regards,
michael
--
If you think good architecture is expensive, try bad architecture.
-- Brian Foote and Joseph Yoder, Big Ball Of Mud
___
xml mailing list, project page http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml
[xml] libxml2/libxslt linker version scripts vs. Solaris ld
hi all, in order to solve this bug: http://www.openoffice.org/issues/show_bug.cgi?id=112480 i've had to build libxml2 and libxslt with versioned symbols on Solaris. it turns out that the version scripts that are shipped with libxml2/libxslt only work with GNU ld, but not with Solaris ld. my changes to make the version scripts work with Solaris ld are here: http://hg.services.openoffice.org/hg/cws/solaris10u8/rev/4aca101677ed http://hg.services.openoffice.org/hg/cws/solaris10u8/rev/0e826b9ac4c7 when Solaris ld sees a symbol marked as global: in a version script, it always puts a reference into the symbol table, while GNU ld only seems to do this for symbols that are actually defined in the input object files. as a result, linking programs against libxml2/libxslt will fail, because some of the symbols in libxml2/libxslt are not actually defined. another difference is that Solaris ld only exports the symbols that are explicitly marked as global: in the version script. specific failures: - libxml2: xmlDllMain seems only defined on win32 = linking xmllint fails - libxml2: globals.h wraps all its global variable in an #ifdef LIBXML_THREAD_ENABLED, and if this condition is true, actually defines a macro that expands to a function call. all of these functions are missing from the mapfile = linking xmllint fails [ with GNU ld, these function symbols are actually exported un-versioned, which i guess is probably not intended. ] - libxslt: there are some symbols that are not implemented anywhere i could find, e.g., xsltMatchPattern, xsltTransStorageAdd/Remove = linking xsltproc fails - libxslt: there are some symbols where the implementation is guarded by XSLT_REFACTORED macro, but this does not seem to be defined anywhere, so the functions do not exist = linking xsltproc fails also, i've found it necessary to include an auto-reduction statement in the version script to hide all non-public symbols: local: *; it seems to me that including symbols in a generic version like LIBXML2_1.1.10 that may or may not exist, depending on what preprocessor macros are defined, is a questionable practice: a dependent library or executable that depends on libxslt.so.1 with version LIBXML2_1.1.10 may or may not run against an actual libxslt.so.1 that contains the LIBXML2_1.1.10 version. according to the Solaris documentation, the version script does not need to define a linear list of dependent versions; probably it would be better to define guarded symbols with a distinct version (so the versions form a tree). but to please Solaris ld it would still be necessary to include exactly the symbols that are actually defined; i'm not sure how to do this, especially given that the linker version scripts seem to be generated from some XML file. maybe generate several version scripts with different options, or generate fragments and concatenate them at build-time... opinions? regards, michael -- ENOSIG ___ xml mailing list, project page http://xmlsoft.org/ xml@gnome.org http://mail.gnome.org/mailman/listinfo/xml
[xml] libxml2/libxslt: global variables considered harmful
hi all, i've just remembered that i've long wanted to write a rant on how evil the global variables in libxml2/libxslt are. i've come to this conclusion with this bug: http://qa.openoffice.org/issues/show_bug.cgi?id=110523 what happens here is that there is now some new code that uses the redland and raptor libraries to parse RDF files. raptor uses libxml2 and libxslt to parse various different formats. by simply initializing the raptor library, a completely different part of our enormous, complex, bloated system was broken in a catastrophic way: the online help browser would no longer display any help, just a white screen. the online help has nothing at all to do with RDF, but it also uses the libxslt library. initializing raptor sets some global variable in libxslt that prevents opening file:// urls, and that breaks the online help. in a large and complex system it's generally impossible to answer questions like where is library $foo used?; thus hidden communication channels that can be accidentally invoked (like global variables) are a huge PITA. actually, read-only global data is not a problem; i think it is only a problem if the global variables can be changed by client code. now if you look at the interface of the redland library you can see one approach to solve this problem: redland defines a redland_world struct, and this contains all its global state. http://librdf.org/docs/api/redland-world.html all redland public methods either take a librdf_world parameter, or some parameter that contains a pointer to a librdf_world. then there are functions to create, init and destroy librdf_worlds. so different clients of the redland library are (hopefully) completely independent. the only problem is that redland depends on other libraries, and likes to call their de-initialization functions when freeing a librdf_world :( IMHO such a design would also be possible for libxml2/libxslt, but of course this would be an incompatible interface change. usually there isn't much enthusiasm for that kind of thing :) but if there ever is an opportunity for a libxml3, _please_ get rid of global variables. regards, michael -- UTF-16 is the devil's work. -- Robert O'Callahan ___ xml mailing list, project page http://xmlsoft.org/ xml@gnome.org http://mail.gnome.org/mailman/listinfo/xml
Re: [xml] win32 getaddrinfo
Roumen Petrov wrote:
Michael Stahl wrote:
Rob Richards wrote:
Michael Stahl wrote:
Rob Richards wrote:
/snip
I think I finally understand what you are trying to do. You are building
on Win 2003 with VS 2005 and then trying to use the lib on on Win2K or
lower and it is there that it is failing to link?
well, it fails to link with vs 2005 already because libxml (using 2.6.31
here) is not linked against ws2_32.dll, so ld (or whatever msft calls its
equivalent) complained that getaddrinfo and freeaddrinfo are undefined
symbols.
I get it now and thought that was only an issue with VS 2008. Can you
try building from SVN or even a snapshot. The lib was added just after
the .31 release.
ok, i have built a snapshot (downloaded friday iirc). building works, and
the dll is linked against ws2_32.dll.
Can you try the following patch?
Index: wsockcompat.h
===
--- wsockcompat.h(revision 3721)
+++ wsockcompat.h(working copy)
@@ -21,6 +21,7 @@
/* Check if ws2tcpip.h is a recent version which provides getaddrinfo()
*/
#if defined(GetAddrInfo)
+#include wspiapi.h
#define HAVE_GETADDRINFO
#endif
#endif
ah yes, that seems to work fine here, and is obviously the cleaner way to
do it.
If the build from SVN fixes the initial problem, can you try using it on
w Win2K machine? It's possible that the patch might not be needed though
after more research I have doubts. I'm looking around here for an old OS
to test on as well. I'm pretty sure I have an old VM image lying around
here somewhere.
so today i finally got access to a win2k machine. the result:
with 2.6.31+my patch, the application runs.
with the snapshot, the application fails to run, and displays a message
box complaining that ws2_32.dll could not be found.
so, i would suggest you revert the patch that introduced linking to
ws2_32.dll, and use wspapi.h instead.
regards
michael
Michael,
If my patch is you patch first posted to the list I think that you
understand that patch break libxml2 build on other platforms. Is you
suggestion to broke libxml2 intentional ?
errm, no, by my patch i meant the general approach of including the
header wspiapi.h _instead_of_ linking against ws2_32.dll, however that may
be implemented so that it works on all supported platforms. i have merely
told how i could get it to work on my setup (building on win2k3 and
deploying on win2k), and that the status quo does _not_ work in this
setup. i will leave the details to those who actually know something about
win32.
Rob,
may be the problem can be resolved with this proof of concept code:
do {
/* resolve getaddrinfo at run time instead at compile time */
h = LoadLibraryA(ws2_32);
if (h != NULL) {
f = GetProcAddress(h, getaddrinfo);
if (f == NULL) {
FreeLibrary(h);
h = NULL;
} else {
/* */
}
}
h = LoadLibraryA(wship6);
if (h != NULL) {
f = GetProcAddress(h, getaddrinfo);
if (f == NULL) {
FreeLibrary(h);
h = NULL;
} else {
call f
}
}
/* fail-back to gethostbyname */
} while (0);
As I understand wship6 is not default on platforms before nt 5.1.
Roumen
well, the msdn page for getaddrinfo claims that this is basically what the
macro in wspiapi.h does.
michael
--
Experience is that marvelous thing that enables you recognize a
mistake when you make it again. -- F. P. Jones
___
xml mailing list, project page http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml
Re: [xml] win32 getaddrinfo
Rob Richards wrote: Michael Stahl wrote: Rob Richards wrote: /snip I think I finally understand what you are trying to do. You are building on Win 2003 with VS 2005 and then trying to use the lib on on Win2K or lower and it is there that it is failing to link? well, it fails to link with vs 2005 already because libxml (using 2.6.31 here) is not linked against ws2_32.dll, so ld (or whatever msft calls its equivalent) complained that getaddrinfo and freeaddrinfo are undefined symbols. I get it now and thought that was only an issue with VS 2008. Can you try building from SVN or even a snapshot. The lib was added just after the .31 release. ok, i have built a snapshot (downloaded friday iirc). building works, and the dll is linked against ws2_32.dll. Can you try the following patch? Index: wsockcompat.h === --- wsockcompat.h(revision 3721) +++ wsockcompat.h(working copy) @@ -21,6 +21,7 @@ /* Check if ws2tcpip.h is a recent version which provides getaddrinfo() */ #if defined(GetAddrInfo) +#include wspiapi.h #define HAVE_GETADDRINFO #endif #endif ah yes, that seems to work fine here, and is obviously the cleaner way to do it. If the build from SVN fixes the initial problem, can you try using it on w Win2K machine? It's possible that the patch might not be needed though after more research I have doubts. I'm looking around here for an old OS to test on as well. I'm pretty sure I have an old VM image lying around here somewhere. so today i finally got access to a win2k machine. the result: with 2.6.31+my patch, the application runs. with the snapshot, the application fails to run, and displays a message box complaining that ws2_32.dll could not be found. so, i would suggest you revert the patch that introduced linking to ws2_32.dll, and use wspapi.h instead. regards michael -- Fifty percent of all marriages end in divorce. But look at the bright side: the other fifty percent end in death. -- Richard Jeni ___ xml mailing list, project page http://xmlsoft.org/ xml@gnome.org http://mail.gnome.org/mailman/listinfo/xml
Re: [xml] win32 getaddrinfo
Rob Richards wrote: Michael Stahl wrote: Daniel Veillard wrote: On Tue, Apr 01, 2008 at 10:23:56AM +0200, Michael Stahl wrote: Roumen Petrov wrote: to quote the msdn documentation: The freeaddrinfo function was added to the Ws2_32.dll on Windows XP and later. [...] When the Wspiapi.h include file is added, the freeaddrinfo function is defined to the WspiapiFreeAddrInfo inline function in the Wspiapi.h file. the question is how do you detect that + #include Wspiapi.h /* for getaddrinfo, freeaddrinfo */ is actually available ? What about older version, WinCE ... I guess this need some expert review (I'm not one for any Win stuff !) me neither, i just quote msdn :) Header Declared in Ws2tcpip.h on Windows Server 2008, Windows Vista, Windows Server 2003, and Windows XP. Declared in Ws2tcpip.h on Windows 2000, Windows NT, and Windows Me/98/95; include Wspiapi.h. so, it says nothing about wince, but everything else from msft seems to have it. What are you compiling with? The use of getaddrinfo actually came about due to an issue with win98. http://bugzilla.gnome.org/show_bug.cgi?id=317431 I'm curious to know how HAVE_GETADDRINFO ended up getting defined. from a quick grep, this seems to be defined in include/wsockcompat.h that should be no surprise, as getaddrinfo is (afaik) actually defined in Ws2_32.dll on my build box; the error i had originally occured when _linking_. however, that dll is only available on NT 5.1 (XP) or newer. cl.exe says: Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 not that i have any clue about the setup, i just type build and fix what breaks :) actually i'm compiling on a newer version of NT, let me see, it says: Windows Server 2003 R2 but the result should run on win 2000 as well, which does not have the Ws2_32.dll; according to msdn Wspiapi.h should work in that case as well. hmm, maybe i should test if it actually runs, if i can find a win 2000 box around here... michael -- Good judgement comes from experience; experience comes from bad judgement. -- Fred Brooks ___ xml mailing list, project page http://xmlsoft.org/ xml@gnome.org http://mail.gnome.org/mailman/listinfo/xml
Re: [xml] win32 getaddrinfo
Roumen Petrov wrote: Michael Stahl wrote: libxml2 2.6.31 would not build for me on win32. when linking, it complained about missing getadddrinfo and freeaddrinfo functions. because i have no clue when it comes to win32, i googled around, which yielded a msdn page that claimed that there are inline implementations of these functions in a certain header, resulting in the following patch. regards, michael stahl Why Wspiapi.h ? Why with upper case letter ? Why not ws2tcpip.h ? Roumen to quote the msdn documentation: The freeaddrinfo function was added to the Ws2_32.dll on Windows XP and later. [...] When the Wspiapi.h include file is added, the freeaddrinfo function is defined to the WspiapiFreeAddrInfo inline function in the Wspiapi.h file. so, this header should work also on NT 5.0 (win 2000) and earlier; of course i have not tested that :) re: uppercase: are you telling me that msft's documentation does not even spell their headers correctly? note that i have no clue when it comes to win32, so i assumed they would at least get _that_ right... michael -- You probably know that arrogance in computer science is measured in nano-Dijkstras. -- Alan Kay ___ xml mailing list, project page http://xmlsoft.org/ xml@gnome.org http://mail.gnome.org/mailman/listinfo/xml
Re: [xml] win32 getaddrinfo
Daniel Veillard wrote: On Tue, Apr 01, 2008 at 10:23:56AM +0200, Michael Stahl wrote: Roumen Petrov wrote: to quote the msdn documentation: The freeaddrinfo function was added to the Ws2_32.dll on Windows XP and later. [...] When the Wspiapi.h include file is added, the freeaddrinfo function is defined to the WspiapiFreeAddrInfo inline function in the Wspiapi.h file. the question is how do you detect that + #include Wspiapi.h /* for getaddrinfo, freeaddrinfo */ is actually available ? What about older version, WinCE ... I guess this need some expert review (I'm not one for any Win stuff !) me neither, i just quote msdn :) Header Declared in Ws2tcpip.h on Windows Server 2008, Windows Vista, Windows Server 2003, and Windows XP. Declared in Ws2tcpip.h on Windows 2000, Windows NT, and Windows Me/98/95; include Wspiapi.h. so, it says nothing about wince, but everything else from msft seems to have it. michael -- Q: If I write the code int i, j; can I assume that (i + 1) == j? A: Only sometimes. It's not portable, because in EBCDIC, i and j are not adjacent. ___ xml mailing list, project page http://xmlsoft.org/ xml@gnome.org http://mail.gnome.org/mailman/listinfo/xml
[xml] win32 getaddrinfo
libxml2 2.6.31 would not build for me on win32. when linking, it complained about missing getadddrinfo and freeaddrinfo functions. because i have no clue when it comes to win32, i googled around, which yielded a msdn page that claimed that there are inline implementations of these functions in a certain header, resulting in the following patch. regards, michael stahl -- In the beginning the Universe was created. This has made a lot of people very angry and been widely regarded as a bad move. -- Douglas Adams *** misc/libxml2-2.6.31/nanohttp.c Tue Aug 28 19:34:00 2007 --- misc/build/libxml2-2.6.31/nanohttp.cMon Mar 31 18:36:16 2008 *** *** 68,73 --- 68,77 #endif + #ifdef _MSC_VER + #include Wspiapi.h /* for getaddrinfo, freeaddrinfo */ + #endif + #ifdef VMS #include stropts #define XML_SOCKLEN_T unsigned int ___ xml mailing list, project page http://xmlsoft.org/ xml@gnome.org http://mail.gnome.org/mailman/listinfo/xml
[xml] libxml2 under HP-UX successfully compiled with HP-UX Ansi C Compiler
The INSTALL file delivered with libxml2 sources (libxml2-sources-2.6.23.tar.gz) states that libxml2 cannot be translated with the HP-UX C-Compiler: 3.libxml does not compile with HP-UX's optional ANSI-C compiler Contrary to this statement, I successfully translated libxml2 with a very old (Dec 2000) version of the HP-Unix C-Compiler under HP-UX B.11.00. Ind I did not modify any of the supplied files not did I specify and additional options (just ./configure; make; make install). The compiler supplies the following version information: cpp.ansi: HP92453-01 B.11.11.02 HP C Preprocessor (ANSI) ccom: HP92453-01 B.11.11.02 HP C Compiler Seen the fact that the regression tests (make tests) were executed without problems and also my application runs fine, I think that the statement in INSTALL should be removed (or at least worded differently). -- With kind regards | Mit freundlichen Gruessen Hans-Michael Stahl Condat AG Alt-Moabit 91d | 10559 Berlin | Germany tel: +49.30.3949-1155 | fax: +49.30.3949-2221155 http://www.condat.de/ ___ xml mailing list, project page http://xmlsoft.org/ xml@gnome.org http://mail.gnome.org/mailman/listinfo/xml
