Re: [xml] Research about vulnerabilities

[Aleksey Sanin]

People do attack XML parsing (as well as any other input),
the encryption on the wire doesn't stop a malicious client
from crafting special input and sending it to the server.

I did a cursory look at the code and I believe Nick is correct
that the function in question is never called with a user-input
for the format string. Thus, I don't believe this particular
issue is really a vulnerability.

LibXML2 was tested by Coverity (last test from 2018 build):

https://scan.coverity.com/projects/libxml2

It might make sense to upload a fresh build and see what Coverity
finds. Looks like in 2018 it found a few issues (though hard to
say how real are those without deeper look).

Best,

Aleksey

On 10/29/19 3:05 PM, Eric Eberhard wrote:
I agree.  I also don’t think people attack XML parsing.  The 
sending/receiving can be done encrypted.  This seems a lot like a 
theoretical problem, not a real-world problem.


My feelings are that protecting against all possible attacks is not 
possible.  Or stupid programming.


Take the physical world.  Maybe you just lock your house.  Or maybe you 
have an alarm system.  Or maybe locked gates.  Maybe patrolled by 
Dobermans.  Maybe patrolled by armed guards with machine guns.


My point is – people can pick locks.  Defeat alarms.  Scale gates.  Kill 
dogs.  Kill guards.  It can get silly.  Maybe one should make an 
underground compound with fingerprint and retina scans as well.


You have to stop it someplace.  If this was CIA or FBI or something I’d 
go along with the complaint.  But most of us use this in a business 
setting and we apply *reasonable *security measures, not insanely 
overkill measures.


Eric

*From:*xml [mailto:xml-boun...@gnome.org] *On Behalf Of *Webb Scales
*Sent:* Tuesday, October 29, 2019 7:18 AM
*To:* Raphael de Carvalho Muniz 
*Cc:* xml@gnome.org
*Subject:* Re: [xml] Research about vulnerabilities

Raphael,

First, the disclaimers:  I'm not an XML maintainer or even a 
contributor; and, I've only given this a cursory glance.


Here are my reactions.

First, the routine in question is declared to be of module static 
scope.  I believe that this means that any exploitation of it would have 
to be indirect, coming from within LibXML2.


Second, the code appears to properly protect against buffer overflow, so 
the only exploitation that I see is the one that you illustrated, 
leaking memory position information.  However, the function in question 
does nothing that its caller could not do -- it calls malloc() and (in 
the example you provided) returns the address of what might be a stack 
location (although, really, it could be any random junk, I think).  By 
definition, the caller already knows the stack location, and the caller 
certainly has its own access to malloc().


So, I for one, don't see any weakness exposed by the code which you 
provided.  This doesn't mean that there is no weakness in LibXML2, but, 
to find it if it's there, you'll have to interrogate the code which 
calls this routine.



                 Webb


On 10/29/19 9:30 AM, Raphael de Carvalho Muniz wrote:

  Dear libxml2 owners,

I am performing research about weaknesses in C open source programs.
As part of my research, I am studying weaknesses that may be
vulnerabilities in the Libxml2 project.

I found in the commit history of Libxml2 (commit 9acef28) the
presence of the following code snippet in the libxml.c file (Lines
1,597 - 1,612). I believe that this commit presents a weakness that,
If format strings can be influenced by an attacker, they can be
exploited. This weakness is characterized by CWE Project as CWE-134:
Use of Externally-Controlled Format String. When an attacker can
modify an externally-controlled format string, this can lead to
buffer overflows, denial of service, or data representation problems.

Moreover, I performed a software testing strategy with respect to
confirm the vulnerability. We provide as input to the char *msg the
value "%xtest" and the function libxml_buildMessage return the
value"fc0c748ex", exposing a memory position.

This is the GitHub link to the commit:


https://github.com/GNOME/libxml2/commit/9acef289285008f81b4b66b4880baf600773cf67

Code snippet:

static char *
libxml_buildMessage(const char *msg, va_list ap){
   int chars;
   char *str;

   str = (char *) xmlMalloc(1000);
   if (str == NULL)
     return NULL;

   chars = vsnprintf(str, 999, msg, ap);
   if (chars >= 998)
     str[999] = 0;

   return str;
}


Looking at this code snippet, I am wondering if you could answer the
following brief questions:
We understand that this code has a weakness. Do you agree?
How do you detect weaknesses? Do you use any tool to detect them?
We would be very grateful if you say to us if you agree, and if you
are going to fix it.

-- 


Raphael de Carvalho Muniz, M.Sc.

Lattes: 

Re: [xml] Need help on normalization/canonicalization with namespace prefix rewrite

[Aleksey Sanin]

I don't understand what are you trying to do so I really don't have
a good answer for you.

Aleksey

On 1/29/18 12:33 AM, Mikhail Goloborodko wrote:
> Alexey,
> 
> Thank you!
> I'am new to libxml2. Would appretiate if you advice how to achieve this.
> May be I need a callback for in Reader or Writer?
> 
> Mikhail
> 
> On Sun, Jan 28, 2018 at 10:02 PM, Aleksey Sanin <alek...@aleksey.com
> <mailto:alek...@aleksey.com>> wrote:
> 
> I am not sure what is the suggest algorithm to "rewrite namespace
> prefixes". Regardless, this is not part of C14N spec and something
> you will have to do yourself.
> 
> Aleksey
> 
> On 1/28/18 3:19 AM, Mikhail Goloborodko wrote:
> > Hi All,
> >
> > I will appreciate if somebody could help on how to normalize and
> > canonicalize XML.
> >
> > For example
> > 
> > 
> > 
> >
> > I need to get
> >
> > 
> >
> > And for
> >
> > 
> > 
> >   
> > 
> >
> > I need
> >
> > <n1:N1 xmlns:n1="urn:ru:ed:v2.0"> attr="4583001999" > attr="value">
> >
> > In other words I need to remove whitespaces and rewrite namespace
> prefixes
> > I use
> > string src;
> > xmlChar * canon;
> > xmlDocPtr xDoc = xmlReadMemory(src.data(), src.size(), nullptr,
> nullptr,
> > XML_PARSE_NOBLANKS);
> > int bytes = xmlC14NDocDumpMemory(xDoc, nullptr, 0, nullptr, 0, &
> canon);
> >
> > It removes whitespaces, need help with namespace prefix rewrite.
> >
> > Thank you in advance.
> >
> > On Sun, Jan 28, 2018 at 12:41 AM, Mikhail Goloborodko
> > <mgoloboro...@gmail.com <mailto:mgoloboro...@gmail.com>
> <mailto:mgoloboro...@gmail.com <mailto:mgoloboro...@gmail.com>>> wrote:
> >
> >     Hi,
> >
> >     I need help on how to normalize and canonicalize XML.
> >     For example
> >     
> >      xmlns:ed="urn:ru:ed:v2.0">
> >     
> >
> >     I need to get
> >
> >     
> >
> >     And for
> >
> >     
> >     
> >       
> >     
> >
> >     I need
> >
> >      >     attr="value">
> >
> >     In other words I need to remove whitespaces and rewrite namespace
> >     prefixes
> >     I use
> >     string src;
> >     xmlChar * canon;
> >     xmlDocPtr xDoc = xmlReadMemory(src.data(), src.size(), nullptr,
> >     nullptr, XML_PARSE_NOBLANKS);
> >     int bytes = xmlC14NDocDumpMemory(xDoc, nullptr, 0, nullptr, 0,
> & canon);
> >
> >     It clearly removes whitespace, need help with namespace prefix
> rewrite.
> >
> >     Thank you in advance.
> >
> >     Mikhail 
> >
> >
> >
> >
> > ___
> > xml mailing list, project page  http://xmlsoft.org/
> > xml@gnome.org <mailto:xml@gnome.org>
> > https://mail.gnome.org/mailman/listinfo/xml
> <https://mail.gnome.org/mailman/listinfo/xml>
> >
> 
> 
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
https://mail.gnome.org/mailman/listinfo/xml

Re: [xml] Need help on normalization/canonicalization with namespace prefix rewrite

[Aleksey Sanin]

I am not sure what is the suggest algorithm to "rewrite namespace
prefixes". Regardless, this is not part of C14N spec and something
you will have to do yourself.

Aleksey

On 1/28/18 3:19 AM, Mikhail Goloborodko wrote:
> Hi All,
> 
> I will appreciate if somebody could help on how to normalize and
> canonicalize XML.
> 
> For example
> 
> 
> 
> 
> I need to get
> 
> 
> 
> And for
> 
> 
> 
>   
> 
> 
> I need
> 
>  attr="4583001999" attr="value">
> 
> In other words I need to remove whitespaces and rewrite namespace prefixes
> I use
> string src;
> xmlChar * canon;
> xmlDocPtr xDoc = xmlReadMemory(src.data(), src.size(), nullptr, nullptr,
> XML_PARSE_NOBLANKS);
> int bytes = xmlC14NDocDumpMemory(xDoc, nullptr, 0, nullptr, 0, & canon);
> 
> It removes whitespaces, need help with namespace prefix rewrite.
> 
> Thank you in advance.
> 
> On Sun, Jan 28, 2018 at 12:41 AM, Mikhail Goloborodko
> > wrote:
> 
> Hi,
> 
> I need help on how to normalize and canonicalize XML.
> For example
> 
> 
> 
> 
> I need to get
> 
> 
> 
> And for
> 
> 
> 
>   
> 
> 
> I need
> 
>  attr="value">
> 
> In other words I need to remove whitespaces and rewrite namespace
> prefixes
> I use
> string src;
> xmlChar * canon;
> xmlDocPtr xDoc = xmlReadMemory(src.data(), src.size(), nullptr,
> nullptr, XML_PARSE_NOBLANKS);
> int bytes = xmlC14NDocDumpMemory(xDoc, nullptr, 0, nullptr, 0, & canon);
> 
> It clearly removes whitespace, need help with namespace prefix rewrite.
> 
> Thank you in advance.
> 
> Mikhail 
> 
> 
> 
> 
> ___
> xml mailing list, project page  http://xmlsoft.org/
> xml@gnome.org
> https://mail.gnome.org/mailman/listinfo/xml
> 
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
https://mail.gnome.org/mailman/listinfo/xml

Re: [xml] C14N issue with digital signature due to pointer comparison

[Aleksey Sanin]

I'll re-iterate, as long as you use official LibXML2 API, it should
work just fine. Or there is a bug in the LibXML2 code :)

Aleksey

On 3/21/14, 1:06 AM, Frank Gross wrote:
 Ok, I understand the need of speed, but there is still something that
 worries me. This means that the Canonicalization process depends then on
 the way the XML document is in memory. For instance, if I load a XML
 document from disk and canonicalize it, or build the same document in
 memory from scratch with node built maybe from copy or clone of another
 document in memory, the Canonicalization result can be different, even
 if document saved on disk would be the same.
 
 But ok, I will fix it in my code
 
 Thanks a lot,
 Frank
 
 
 Le 20/03/2014 15:04, Aleksey Sanin a écrit :
 The tradeoff here is speed. Again, if you are using standard LibXML2
 API then everything will work as expected. I have no idea why do you
 need to create new xmlNsPtr objects - this sounds like a waste of
 CPU/memory to me but you might have your reasons. But this also breaks
 a few assumptions on how LibXML2 represents the DOM tree in memory
 and you are on your own in this case.

 Aleksey

 On 3/20/14, 1:26 AM, Frank Gross wrote:
 Hi,

   Actually, I have an additional layer to emulate DOM compliant API over
 libxml. In that layer, I have a global list of xmlNsPtr for all
 namespaces encountered by the nodes (and referenced in node-ns) of a
 single XML document. And for each namespace definition (xmlns:xxx) set
 on a node, a new xmlNsPtr is created and referenced in node-nsDef.  So
 during canonicalization process, when a node looks for a parent
 namespace definition via xmlSearchNs, it can happen that the xmlNsPtr
 reference from the node is different from the parent node-nsDef , even
 if the href and prefix value are the same. Then of course simple pointer
 comparison doesn't work. So I was asking myself if such situation cannot
 happen to other people, and maybe change the pointer comparison because
 depending on how people build and manipulate the document, the test is
 not always accurate.


 Frank


 Le 20/03/2014 03:16, Aleksey Sanin a écrit :
 How do you manipulate the XML tree? If you are using official LibXML2
 function then I believe this code should work just fine unless there is
 a bug in the strings dictionary.

 Aleksey

 On 3/12/14, 10:28 AM, Frank Gross wrote:
 Hi,

 I'm getting some trouble to verify a XML signature because the
 xmlC14NProcessNamespacesAxis() function does a xmlNsPtr pointer
 comparison to decide whether a sub node belongs to the same default
 namespace as an ancestor. But if the sub node has been manipulated by
 program (in my case) to point to another xmlNsPtr with same values,
 the
 canonicalization process breaks. Wouldn't it be better to check the
 namespace href values instead as in following patch ?

 diff --git a/lib-xmlsoft-libxml2/src/c14n.c
 b/lib-xmlsoft-libxml2/src/c14n.c
 index 9c3cad2..f73e709 100644
 --- a/lib-xmlsoft-libxml2/src/c14n.c
 +++ b/lib-xmlsoft-libxml2/src/c14n.c
 @@ -623,7 +623,7 @@ xmlC14NProcessNamespacesAxis(xmlC14NCtxPtr ctx,
 xmlNodePtr cur, int visible)
for(ns = n-nsDef; ns != NULL; ns = ns-next) {
tmp = xmlSearchNs(cur-doc, cur, ns-prefix);

 -if((tmp == ns)  !xmlC14NIsXmlNs(ns) 
 xmlC14NIsVisible(ctx,
 ns, cur)) {
 +if((xmlStrEqual(tmp-href,ns-href)) 
 (xmlStrEqual(tmp-prefix,ns-prefix))  !xmlC14NIsXmlNs(ns) 
 xmlC14NIsVisible(ctx, ns, cur)) {
already_rendered =
 xmlC14NVisibleNsStackFind(ctx-ns_rendered,
 ns);
if(visible) {
xmlC14NVisibleNsStackAdd(ctx-ns_rendered, ns,
 cur);

 Regards,
 Frank

 
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
https://mail.gnome.org/mailman/listinfo/xml

Re: [xml] C14N issue with digital signature due to pointer comparison

[Aleksey Sanin]

The tradeoff here is speed. Again, if you are using standard LibXML2
API then everything will work as expected. I have no idea why do you
need to create new xmlNsPtr objects - this sounds like a waste of
CPU/memory to me but you might have your reasons. But this also breaks
a few assumptions on how LibXML2 represents the DOM tree in memory
and you are on your own in this case.

Aleksey

On 3/20/14, 1:26 AM, Frank Gross wrote:
 Hi,
 
  Actually, I have an additional layer to emulate DOM compliant API over
 libxml. In that layer, I have a global list of xmlNsPtr for all
 namespaces encountered by the nodes (and referenced in node-ns) of a
 single XML document. And for each namespace definition (xmlns:xxx) set
 on a node, a new xmlNsPtr is created and referenced in node-nsDef.  So
 during canonicalization process, when a node looks for a parent
 namespace definition via xmlSearchNs, it can happen that the xmlNsPtr
 reference from the node is different from the parent node-nsDef , even
 if the href and prefix value are the same. Then of course simple pointer
 comparison doesn't work. So I was asking myself if such situation cannot
 happen to other people, and maybe change the pointer comparison because
 depending on how people build and manipulate the document, the test is
 not always accurate.
 
 
 Frank
 
 
 Le 20/03/2014 03:16, Aleksey Sanin a écrit :
 How do you manipulate the XML tree? If you are using official LibXML2
 function then I believe this code should work just fine unless there is
 a bug in the strings dictionary.

 Aleksey

 On 3/12/14, 10:28 AM, Frank Gross wrote:
 Hi,

I'm getting some trouble to verify a XML signature because the
 xmlC14NProcessNamespacesAxis() function does a xmlNsPtr pointer
 comparison to decide whether a sub node belongs to the same default
 namespace as an ancestor. But if the sub node has been manipulated by
 program (in my case) to point to another xmlNsPtr with same values, the
 canonicalization process breaks. Wouldn't it be better to check the
 namespace href values instead as in following patch ?

 diff --git a/lib-xmlsoft-libxml2/src/c14n.c
 b/lib-xmlsoft-libxml2/src/c14n.c
 index 9c3cad2..f73e709 100644
 --- a/lib-xmlsoft-libxml2/src/c14n.c
 +++ b/lib-xmlsoft-libxml2/src/c14n.c
 @@ -623,7 +623,7 @@ xmlC14NProcessNamespacesAxis(xmlC14NCtxPtr ctx,
 xmlNodePtr cur, int visible)
   for(ns = n-nsDef; ns != NULL; ns = ns-next) {
   tmp = xmlSearchNs(cur-doc, cur, ns-prefix);

 -if((tmp == ns)  !xmlC14NIsXmlNs(ns)  xmlC14NIsVisible(ctx,
 ns, cur)) {
 +if((xmlStrEqual(tmp-href,ns-href)) 
 (xmlStrEqual(tmp-prefix,ns-prefix))  !xmlC14NIsXmlNs(ns) 
 xmlC14NIsVisible(ctx, ns, cur)) {
   already_rendered = xmlC14NVisibleNsStackFind(ctx-ns_rendered,
 ns);
   if(visible) {
   xmlC14NVisibleNsStackAdd(ctx-ns_rendered, ns, cur);

 Regards,
 Frank

 
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
https://mail.gnome.org/mailman/listinfo/xml

Re: [xml] C14N issue with digital signature due to pointer comparison

[Aleksey Sanin]

How do you manipulate the XML tree? If you are using official LibXML2
function then I believe this code should work just fine unless there is
a bug in the strings dictionary.

Aleksey

On 3/12/14, 10:28 AM, Frank Gross wrote:
 Hi,
 
   I'm getting some trouble to verify a XML signature because the
 xmlC14NProcessNamespacesAxis() function does a xmlNsPtr pointer
 comparison to decide whether a sub node belongs to the same default
 namespace as an ancestor. But if the sub node has been manipulated by
 program (in my case) to point to another xmlNsPtr with same values, the
 canonicalization process breaks. Wouldn't it be better to check the
 namespace href values instead as in following patch ?
 
 diff --git a/lib-xmlsoft-libxml2/src/c14n.c
 b/lib-xmlsoft-libxml2/src/c14n.c
 index 9c3cad2..f73e709 100644
 --- a/lib-xmlsoft-libxml2/src/c14n.c
 +++ b/lib-xmlsoft-libxml2/src/c14n.c
 @@ -623,7 +623,7 @@ xmlC14NProcessNamespacesAxis(xmlC14NCtxPtr ctx,
 xmlNodePtr cur, int visible)
  for(ns = n-nsDef; ns != NULL; ns = ns-next) {
  tmp = xmlSearchNs(cur-doc, cur, ns-prefix);
 
 -if((tmp == ns)  !xmlC14NIsXmlNs(ns)  xmlC14NIsVisible(ctx,
 ns, cur)) {
 +if((xmlStrEqual(tmp-href,ns-href)) 
 (xmlStrEqual(tmp-prefix,ns-prefix))  !xmlC14NIsXmlNs(ns) 
 xmlC14NIsVisible(ctx, ns, cur)) {
  already_rendered = xmlC14NVisibleNsStackFind(ctx-ns_rendered,
 ns);
  if(visible) {
  xmlC14NVisibleNsStackAdd(ctx-ns_rendered, ns, cur);
 
 Regards,
 Frank
 
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
https://mail.gnome.org/mailman/listinfo/xml

Re: [xml] libxml2 [Sub-document C14N / Custom Malloc]

[Aleksey Sanin]

Take a look at xmlC14NIsVisibleCallback

Aleksey

On 3/11/2010 9:21 AM, Tejus A G wrote:

Hi All,
I have 2 queries w.r.t libxml2 API/usage.

1. I am trying to C14N transform a sub-tree of a XML document. Is there
any simple way to do this ? [I tried XPath but I couldn't find a query
to get the
full sub-tree using xpath].

I am right now using C14NExecute and have my callback check each node

2. Is there any way to give the libXML2 library a custom malloc function ?
So, that all memory allocations the library does goes through it instead of
whatever libXML2 is using.

-TIA



___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] xmlCleanupParser() question / patch

[Aleksey Sanin]

I am sure you know about __attribute__ ((deprecated)) :)

Aleksey


On 1/19/2010 7:47 AM, Daniel Veillard wrote:

On Tue, Jan 19, 2010 at 04:38:24PM +0100, Martin B. wrote:

Daniel Veillard wrote:

Best thing would be if xmlCleanupParser() would be removed completely, no?


  That is impossible, for ABI compatibility,



Ah OK. In that case the best thing would be to have the function in
the ABI but get a compiler error if it's used with the newer
headers. Don't have a clue how that could magically be achieved
though :-)


   Yeah, no idea how to do that ! Unfortunately :-\

Daniel


___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] c14n 1.1 support (patch)

[Aleksey Sanin]

thanks for the patch, sorry for the delay, I was really focused
on work stuff :-\


No worries, I know how it works :)


  I don't know if this is nasty or not, but git decided to remove some
white spaces, which I'm afraid may change the output of the test, so
please double check... I also got a few similar warnings with patch 0003
and 004 in the second tarball. On the other hand, the regression tests
look fine, but double checking still a good idea IMHO :-)


Will take a look. I believe this is a known problem with git patches
produced with cygwin (this is what I was using).


  that one is more serious, that's an ABI breakage, unfortunately we
can't do that, the C standard doesn't define the size of an enum :-(


Yeah, I wasn't sure about that one myself. Thanks for fixing it!


  I commited the result, I guess it should be very close to what you had
in your tree, but like you I'm learning git so please double check the
code in git is actually what you expected :-) (module the small API
change).


Thanks again, I'll take a look later today and let you know.


Best,
Aleksey

___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] c14n 1.1 support (patch)

[Aleksey Sanin]

  I commited the result, I guess it should be very close to what you had
in your tree, but like you I'm learning git so please double check the
code in git is actually what you expected :-) (module the small API
change).


Thanks again, I'll take a look later today and let you know.


There were few more places where my original patch changed
the ABI (int - enum). Please find attached a patch that fixes
these places and also adds a check that int is actually enum :)

Thanks again for your help!

Aleksey
From 5bd0ec467b4bf6f5ea33ec445feb7db95dca3ef6 Mon Sep 17 00:00:00 2001
From: Aleksey Sanin alek...@aleksey.com
Date: Thu, 9 Jul 2009 08:38:17 -0700
Subject: [PATCH] c14n11: fix other ABI changes

---
 c14n.c|   24 +++-
 include/libxml/c14n.h |6 +++---
 2 files changed, 22 insertions(+), 8 deletions(-)

diff --git a/c14n.c b/c14n.c
index 53a19c1..5c6c456 100644
--- a/c14n.c
+++ b/c14n.c
@@ -1863,10 +1863,11 @@ xmlC14NNewCtx(xmlDocPtr doc,
  */
 int
 xmlC14NExecute(xmlDocPtr doc, xmlC14NIsVisibleCallback is_visible_callback,
-void* user_data, xmlC14NMode mode, xmlChar **inclusive_ns_prefixes,
+void* user_data, int mode, xmlChar **inclusive_ns_prefixes,
 int with_comments, xmlOutputBufferPtr buf) {
 
 xmlC14NCtxPtr ctx;
+xmlC14NMode c14n_mode = XML_C14N_1_0;
 int ret;
 
 if ((buf == NULL) || (doc == NULL)) {
@@ -1874,6 +1875,19 @@ xmlC14NExecute(xmlDocPtr doc, xmlC14NIsVisibleCallback 
is_visible_callback,
 return (-1);
 }
 
+/* for backward compatibility, we have to have mode as int 
+   and here we check that user gives valid value */
+switch(mode) {
+case XML_C14N_1_0:
+case XML_C14N_EXCLUSIVE_1_0:
+case XML_C14N_1_1: 
+ c14n_mode = (xmlC14NMode)mode;
+ break;
+default:   
+xmlC14NErrParam(invalid mode for executing c14n);
+return (-1);
+}
+
 /*
  *  Validate the encoding output buffer encoding
  */
@@ -1884,8 +1898,8 @@ xmlC14NExecute(xmlDocPtr doc, xmlC14NIsVisibleCallback 
is_visible_callback,
 }
 
 ctx = xmlC14NNewCtx(doc, is_visible_callback, user_data, 
-   mode, inclusive_ns_prefixes,
-with_comments, buf);
+   c14n_mode, inclusive_ns_prefixes,
+with_comments, buf);
 if (ctx == NULL) {
 xmlC14NErr(NULL, (xmlNodePtr) doc, XML_C14N_CREATE_CTXT,
   xmlC14NExecute: unable to create C14N context\n);
@@ -1986,7 +2000,7 @@ xmlC14NDocSaveTo(xmlDocPtr doc, xmlNodeSetPtr nodes,
  */
 int
 xmlC14NDocDumpMemory(xmlDocPtr doc, xmlNodeSetPtr nodes,
- xmlC14NMode mode, xmlChar ** inclusive_ns_prefixes,
+ int mode, xmlChar ** inclusive_ns_prefixes,
  int with_comments, xmlChar ** doc_txt_ptr)
 {
 int ret;
@@ -2057,7 +2071,7 @@ xmlC14NDocDumpMemory(xmlDocPtr doc, xmlNodeSetPtr nodes,
  */
 int
 xmlC14NDocSave(xmlDocPtr doc, xmlNodeSetPtr nodes,
-   xmlC14NMode mode, xmlChar ** inclusive_ns_prefixes,
+   int mode, xmlChar ** inclusive_ns_prefixes,
int with_comments, const char *filename, int compression)
 {
 xmlOutputBufferPtr buf;
diff --git a/include/libxml/c14n.h b/include/libxml/c14n.h
index e99b144..3011af7 100644
--- a/include/libxml/c14n.h
+++ b/include/libxml/c14n.h
@@ -75,7 +75,7 @@ XMLPUBFUN int XMLCALL
 XMLPUBFUN int XMLCALL
xmlC14NDocDumpMemory(xmlDocPtr doc,
 xmlNodeSetPtr nodes,
-xmlC14NMode mode,
+int mode, /* a xmlC14NMode */
 xmlChar **inclusive_ns_prefixes,
 int with_comments,
 xmlChar **doc_txt_ptr);
@@ -83,7 +83,7 @@ XMLPUBFUN int XMLCALL
 XMLPUBFUN int XMLCALL
xmlC14NDocSave  (xmlDocPtr doc,
 xmlNodeSetPtr nodes,
-xmlC14NMode mode,
+int mode, /* a xmlC14NMode */
 xmlChar **inclusive_ns_prefixes,
 int with_comments,
 const char* filename,
@@ -111,7 +111,7 @@ XMLPUBFUN int XMLCALL
xmlC14NExecute  (xmlDocPtr doc,
 xmlC14NIsVisibleCallback 
is_visible_callback,
 void* user_data,
-xmlC14NMode mode,
+int mode, /* a xmlC14NMode */
 xmlChar **inclusive_ns_prefixes,
 int with_comments

Re: [xml] c14n 1.1 support (patch)

[Aleksey Sanin]

Thanks for committing it!

Aleksey

Daniel Veillard wrote:

On Thu, Jul 09, 2009 at 08:42:12AM -0700, Aleksey Sanin wrote:

  I commited the result, I guess it should be very close to what you had
in your tree, but like you I'm learning git so please double check the
code in git is actually what you expected :-) (module the small API
change).

Thanks again, I'll take a look later today and let you know.

There were few more places where my original patch changed
the ABI (int - enum). Please find attached a patch that fixes
these places and also adds a check that int is actually enum :)

Thanks again for your help!


  Oops right I didn't see those ! Applied and commited,

thanks !

Daniel


___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

[xml] c14n 1.1 support (patch)

[Aleksey Sanin]

Daniel,

Please find attached a patch that adds support for the new
version of c14n (http://www.w3.org/TR/xml-c14n11/). I am
getting questions about it in the xmlsec mailing list and
I finally decided to implement it. I would greatly appreciate
if you can accept this patch and push it into the gnome git
repository (note, that there are some new files/folders added
for the new test cases).


Thank you in advance,
Aleksey Sanin






c14n11.diff.gz
Description: GNU Zip compressed data
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] c14n 1.1 support (patch)

[Aleksey Sanin]

Please ignore the previous patch, it is incomplete. This is the
correct one.

Sorry, still learning git :)

Aleksey

Aleksey Sanin wrote:

Daniel,

Please find attached a patch that adds support for the new
version of c14n (http://www.w3.org/TR/xml-c14n11/). I am
getting questions about it in the xmlsec mailing list and
I finally decided to implement it. I would greatly appreciate
if you can accept this patch and push it into the gnome git
repository (note, that there are some new files/folders added
for the new test cases).


Thank you in advance,
Aleksey Sanin







___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml


c14n-11.tar.gz
Description: GNU Zip compressed data
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] c14n 1.1 support (patch)

[Aleksey Sanin]

And one more patch that adds extra test vectors and speeds up
code a little bit.

Aleksey

Aleksey Sanin wrote:

Please ignore the previous patch, it is incomplete. This is the
correct one.

Sorry, still learning git :)

Aleksey

Aleksey Sanin wrote:

Daniel,

Please find attached a patch that adds support for the new
version of c14n (http://www.w3.org/TR/xml-c14n11/). I am
getting questions about it in the xmlsec mailing list and
I finally decided to implement it. I would greatly appreciate
if you can accept this patch and push it into the gnome git
repository (note, that there are some new files/folders added
for the new test cases).


Thank you in advance,
Aleksey Sanin







___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml




___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml


c14n11-2nd.tar.gz
Description: GNU Zip compressed data
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] xml output streaming and c14n

[Aleksey Sanin]

Look at testC14N.c for a c14n example.

Aleksey

Prashant R wrote:
I have a requirement, where I would not want to have the entire xml data 
in memory and preferably writing out canonicalized XML to IO . 
I had a few questions to understand this better 

I was planning on doing the following 


* creating an OutputBuffer using the API - xmlOutputBufferCreateIO
  http://xmlsoft.org/html/libxml-xmlIO.html#xmlOutputBufferCreateIO
  (xmlOutputWriteCallback
  http://xmlsoft.org/html/libxml-xmlIO.html#xmlOutputWriteCallback
  iowrite, xmlOutputCloseCallback
  http://xmlsoft.org/html/libxml-xmlIO.html#xmlOutputCloseCallback
  ioclose, void * ioctx, xmlCharEncodingHandlerPtr
  http://xmlsoft.org/html/libxml-encoding.html#xmlCharEncodingHandlerPtr
  encoder) // as I need to stream to I/O
* then  using the API  xmlTextWriterPtr
  http://xmlsoft.org/html/libxml-xmlwriter.html#xmlTextWriterPtr
  xmlNewTextWriter (xmlOutputBufferPtr
  http://xmlsoft.org/html/libxml-tree.html#xmlOutputBufferPtr out)
  , to create an xmlWriterObject

* write out the xml nodes based on my application logic


Quesions
1. When will the actual write ( to I/O , in my case )occur ? Could it 
occur before the entire document is fully completed?


2. Is there a way where I could stream xml data directly to IO without 
building the tree in memory ( using DOM API's)?


3 If there is a way to do (2) , then is there a way to specify that the 
output be canonicalized ?


Thanks in advance





___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] C14n for subelements

[Aleksey Sanin]

How Do I add the pNode and all its sub-nodes to the NodeSet ( without 
using xpath ! )
xmlXPathNodeSetCreate(pNode) seems to add only 1 Node to the node set 
and not the child nodes 


By manually walking the subtree?

Aleksey

___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] C14N does not handle empty-namespace declarations as the implementation in Java

[Aleksey Sanin]

Yes.

Aleksey

Arne Müller wrote:

just that I get this right:

The namespace nodes of a node are not just the namespaces declared at 
that node, but all namespaces that are known at that node? Is that correct?


thanks for your help,
Arne

2008/8/27 Aleksey Sanin [EMAIL PROTECTED] mailto:[EMAIL PROTECTED]

Yes, namespace nodes are *nodes* and all the usual rules apply.

Aleksey

Arne Müller wrote:

Hi Aleksey,

thanks for pointing me to those test-files. With them I also do
not get xmlns=.
So then I checked, where there were differences between my code
and the one in testC14N.c.
I realized, that the xmlns= appears, because I use a nodeset.
It seems, if I use a default namespace, I have to add for each
descandant a copy of it into the NodeSet, else it will not find
it and will not set, that it has found a default-namespace.
(see line 623 in c14n.c)
Is this the intended behavior? I thought it would be sufficient
to add the namespace for the namespace-declaring node into the
nodeset and not for all descendant nodes.

Somehow the example code does this correctly (when given the
XPath /descendant::* | /descendant::*/attribute::* |
/descendant::*/namespace::*).

So XPath seems to add namespace declarations everywhere and thus
avoid the problem, while I have build the nodeset myself and not
added as many namespace declarations (which I actually think
create very much overhead).

I think I could now avoid the problem by modifying my
NodeSet-creation code, to make it add namespaces for all
descendants into the nodeset.
But maybe it would be better non the less to simply check if the
nodeset contains a namespace-declaration for a parent node? Or
am I missing something?

Arne


2008/8/26 Aleksey Sanin [EMAIL PROTECTED]
mailto:[EMAIL PROTECTED] mailto:[EMAIL PROTECTED]
mailto:[EMAIL PROTECTED]


   Arne,

   Could you please give an exact example that shows the problem
   using the testC14N command? For example, for the xml snippet
   from your previous email I generated C14N and I see no xmlns=
   definitions:

   $ testC14N --with-comments test.xml  test-c14n.xml

   Thanks
   Aleksey


   Arne Mueller wrote:

   Am Sonntag, den 24.08.2008, 21:21 -0700 schrieb Aleksey
Sanin:

   Hi, Arne!

   Sorry, but as I have replied in xmlsec mailing list,
I believe
   that libxml2 is doing the right thing here.

   Aleksey


   Hi Aleksey,
   you might be perfectly right with adding one empty namespace.

   But I think one thing is buggy non the less, in the
   c14n-specification
   it says:

   quote
   4.6 Superfluous Namespace Declarations
   Unnecessary namespace declarations are not made in the
canonical
   form.
   Whether for an empty default namespace, a non-empty default
   namespace,
   or a namespace prefix binding, the XML canonicalization
method
   omits a
   declaration if it determines that the immediate parent
element
   in the
   canonical form has an equivalent declaration in scope.
The root
   document
   element is handled specially since it has no parent
element. All
   namespace declarations in it are retained, except the
   declaration of an
   empty default namespace is automatically omitted.
   /quote

   Thus, there should be at most one empty namespace
declaration,
   because
   an parent element has already defined the empty namespace
   declaration.
   The same holds for the redefinition of dsig in the
XPath element.

   Arne






___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] C14N does not handle empty-namespace declarations as the implementation in Java

[Aleksey Sanin]

Arne,

Could you please give an exact example that shows the problem
using the testC14N command? For example, for the xml snippet
from your previous email I generated C14N and I see no xmlns=
definitions:

$ testC14N --with-comments test.xml  test-c14n.xml

Thanks
Aleksey

Arne Mueller wrote:

Am Sonntag, den 24.08.2008, 21:21 -0700 schrieb Aleksey Sanin:

Hi, Arne!

Sorry, but as I have replied in xmlsec mailing list, I believe
that libxml2 is doing the right thing here.

Aleksey


Hi Aleksey,
you might be perfectly right with adding one empty namespace.

But I think one thing is buggy non the less, in the c14n-specification
it says:

quote
4.6 Superfluous Namespace Declarations
Unnecessary namespace declarations are not made in the canonical form.
Whether for an empty default namespace, a non-empty default namespace,
or a namespace prefix binding, the XML canonicalization method omits a
declaration if it determines that the immediate parent element in the
canonical form has an equivalent declaration in scope. The root document
element is handled specially since it has no parent element. All
namespace declarations in it are retained, except the declaration of an
empty default namespace is automatically omitted.
/quote

Thus, there should be at most one empty namespace declaration, because
an parent element has already defined the empty namespace declaration.
The same holds for the redefinition of dsig in the XPath element.

Arne

dsig:SignedInfo xmlns=http://www.w3.org/2000/09/xmldsig#; xmlns:dsig=http://www.w3.org/2000/09/xmldsig#;
dsig:CanonicalizationMethod Algorithm=http://www.w3.org/TR/2001/REC-xml-c14n-20010315;/dsig:CanonicalizationMethod
dsig:SignatureMethod Algorithm=http://www.w3.org/2000/09/xmldsig#rsa-sha1;/dsig:SignatureMethoddsig:Reference URI=
dsig:Transforms
dsig:Transform Algorithm=http://www.w3.org/TR/1999/REC-xpath-19991116;dsig:XPath
count(ancestor-or-self::dsig:Signature  | here()/ancestor::dsig:Signature[1])
gt;  count(ancestor-or-self::dsig:Signature)
/dsig:XPath
/dsig:Transform
/dsig:Transforms
dsig:DigestMethod Algorithm=http://www.w3.org/2000/09/xmldsig#sha1;/dsig:DigestMethod
dsig:DigestValuenDF2V/bzRd0VE3EwShWtsBzTEDc=/dsig:DigestValue/dsig:Reference
/dsig:SignedInfodsig:SignedInfo 
	xmlns=http://www.w3.org/2000/09/xmldsig#;
	xmlns:dsig=http://www.w3.org/2000/09/xmldsig#;
dsig:CanonicalizationMethod
	Algorithm=http://www.w3.org/TR/2001/REC-xml-c14n-20010315/
dsig:SignatureMethod
	Algorithm=http://www.w3.org/2000/09/xmldsig#rsa-sha1/dsig:Reference URI=
dsig:Transforms
dsig:Transform
Algorithm=http://www.w3.org/TR/1999/REC-xpath-19991116;dsig:XPath
xmlns:dsig=http://www.w3.org/2000/09/xmldsig#;
count(ancestor-or-self::dsig:Signature  | here()/ancestor::dsig:Signature[1])
gt;  count(ancestor-or-self::dsig:Signature)
/dsig:XPath
/dsig:Transform
/dsig:Transforms
dsig:DigestMethod Algorithm=http://www.w3.org/2000/09/xmldsig#sha1/
dsig:DigestValuenDF2V/bzRd0VE3EwShWtsBzTEDc=/dsig:DigestValue/dsig:Reference
/dsig:SignedInfo
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] C14N does not handle empty-namespace declarations as the implementation in Java

[Aleksey Sanin]

Hi, Arne!

Sorry, but as I have replied in xmlsec mailing list, I believe
that libxml2 is doing the right thing here.

Aleksey

Arne Mueller wrote:

Hello,

I've filed the following bug-report, but Daniel Veillard told me, that
it'll be best to post on the mailing list and check with Aleksey. So
this is what I'll do now ;)

So here's my bugreport:

Please describe the problem:
The Problem is the following section of the C14N Recommondation:

4.7 Propagation of Default Namespace Declaration in Document Subsets
The XPath data model represents an empty default namespace with the absence of
a node, not with the presence of a default namespace node having an empty
value. Thus, with respect to the fact that element e3 in the following examples
is not namespace qualified, we cannot tell the difference between e1
xmlns=a:be2 xmlns=e3//e2/e1 versus e1 xmlns=a:be2e3
xmlns=//e2/e1. All we know is that e3 was not namespace qualified on
input, so we preserve this information on output if e2 is omitted so that e3
does not take on the default namespace qualification of e1. 


The implementation of C14n in libxml2 does not seem to do have the same
behavior, as the implementation of Sun (for Java) or IBM (see
http://www.aleksey.com/pipermail/xmlsec/2003/000976.html ), thus I do not
think, this will change in the future.

I'm actually not quite sure, what is meant by that part of the specification,
so it might be quite possible, that Sun and IBM are correct, but libxml2 is
not.

Steps to reproduce:
Try to canocalize the following XML-Document (I have reduced it to the
essential part):

dsig:SignedInfo xmlns=http://www.w3.org/2000/09/xmldsig#;
xmlns:dsig=http://www.w3.org/2000/09/xmldsig#;
dsig:CanonicalizationMethod
Algorithm=http://www.w3.org/TR/2001/REC-xml-c14n-20010315/
dsig:SignatureMethod
Algorithm=http://www.w3.org/2000/09/xmldsig#rsa-sha1/dsig:Reference URI=
dsig:Transforms
dsig:Transform
Algorithm=http://www.w3.org/TR/1999/REC-xpath-19991116;dsig:XPath
xmlns:dsig=http://www.w3.org/2000/09/xmldsig#;
count(ancestor-or-self::dsig:Signature  | here()/ancestor::dsig:Signature[1])
gt;  count(ancestor-or-self::dsig:Signature)
/dsig:XPath
/dsig:Transform
/dsig:Transforms
dsig:DigestMethod Algorithm=http://www.w3.org/2000/09/xmldsig#sha1/
dsig:DigestValuenDF2V/bzRd0VE3EwShWtsBzTEDc=/dsig:DigestValue/dsig:Reference
/dsig:SignedInfo

Actual results:
libxml2 canocalisation adds, because in SignedInfo
xmlns=http://www.w3.org/2000/09/xmldsig#; is defined, loads of xmlns=
declarations to all child nodes of SignedInfo and it does not remove the extra
namespace declaration in XPath (as it must accoriding to
http://www.w3.org/TR/2001/REC-xml-c14n-20010315#SuperfluousNSDecl )

Expected results:
Java does not add the extra xmlns= and removes the extra namespace
declaration in XPath. Because C14N is used for signing documents, all
implementations must behave the same, else the signatures cannot be verified.
So I expect libxml2 to behave as Java (and as it seems the rest of the world
does)

Does this happen every time?
I only have tested it with this example (or slight modifications), but I
checked the code and it should not behave very differently in other cases.

cheers,
Arne

PS: I already wrote a fix that seems to work for me, but I'm not quite sure, if 
it's really correct.
I think I actually only had one example case with that special problem to test 
it against.




___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] FW: LibXml2-2.6-32 : Canonical XML implementation test program

[Aleksey Sanin]

Sorry for delay with the reply. I just got back from vacation...

Your xpath is incorrect, try to use ancestor-or-self::ns1:Operation
instead.

Best,
Aleksey


Andrew Hartley wrote:

Hi,

 

Really sorry to have bothered you with this.  I should have spotted that 
the XML file from which I was extracting the data was at fault.  After I 
sent the emails earlier, I spotted the console window reporting xmlns 
errors.


 


Sorry.

 


Best regards,

Andy

 


*From:* Andrew Hartley [mailto:[EMAIL PROTECTED]
*Sent:* 05 July 2008 17:47
*To:* '[EMAIL PROTECTED]'
*Subject:* FW: LibXml2-2.6-32 : Canonical XML implementation test program

 


Sorry, I forgot to attaché the files.

 


Andy

 


*From:* Andrew Hartley [mailto:[EMAIL PROTECTED]
*Sent:* 05 July 2008 17:46
*To:* '[EMAIL PROTECTED]'
*Subject:* LibXml2-2.6-32 : Canonical XML implementation test program

 


Hi,

 

I hope you don’t mind me contacting you.   I would appreciate if you 
could let me have a reply to the question below.


 


 I have been using your LibXml2 library, thanks for writing this.

 

I am running your Canonical XML implementation test program and with the 
above version of the library.  I am using the following argument list 
(and enclose the files herewith):


 


--exc-without-comments d:\temp\ros.xml d:\temp\MyXpathDoc.xml soapenv

 


All is working fine until I reach:

 


ret = xmlC14NDocDumpMemory(doc,

(xpath) ? xpath-nodesetval : NULL,

exclusive, inclusive_namespaces,

with_comments, result);

 

The xmlC14NDocDumpMemory return -1 and the result is still NULL.   I am 
running Visual Studio 2005, and Vista Home Premium with UAC switched off.


 

Can you think of any reason as to why this is failing.  I don’t have the 
source for LibXml2, otherwise I could run the source and check.


 

 


___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] export xmlTextReaderSetup() function

[Aleksey Sanin]

   Can you write an example showing the use then, I'm a bit lost I admit :-)

Sure :) This is pretty long one but 

xmlTextReaderPtr XmlReaderContext::CreateTextReader(
xmlParserInputBufferPtr input_buffer,
const string uri)
{
 assert(input_buffer);

 xmlParserInputBufferPtr input_buffer2 = input_buffer;
 if(HasEncryptKey()) {
 assert(!m_base64_input_buffer.get());
 assert(!m_encrypt_input_buffer.get());

 // insert base64 input buffer
 m_base64_input_buffer.reset(new 
Base64DecodeInputBuffer(input_buffer2));
 assert(m_base64_input_buffer.get());

 input_buffer2 = xmlParserInputBufferCreateIO(
 XmlInputReadFromBase64EncodeCallback,
 XmlInputCloseBase64EncodeCallback,
 m_base64_input_buffer.get(),
 XML_CHAR_ENCODING_NONE);

 if (input_buffer2 == NULL) {
 C8LOG_THROW(this, LogError, Failed to create base64 
decoder input buffer.) {
 XmlErrorReport(c8log, xmlGetLastError());
 }
 }
 m_input_buffers.push_back(input_buffer2);

 // insert encrypt input buffer
 m_encrypt_input_buffer.reset(new 
EncryptInputBuffer(input_buffer2, GetEncryptKey()));
 assert(m_encrypt_input_buffer.get());

 input_buffer2 = xmlParserInputBufferCreateIO(
 XmlInputReadFromEncryptCallback,
 XmlInputCloseEncryptCallback,
 m_encrypt_input_buffer.get(),
 XML_CHAR_ENCODING_NONE);

 if (input_buffer2 == NULL) {
 C8LOG_THROW(this, LogError, Failed to create encrypt 
decoder input buffer.) {
 XmlErrorReport(c8log, xmlGetLastError());
 }
 }
 m_input_buffers.push_back(input_buffer2);
 }

 xmlTextReaderPtr reader = xmlNewTextReader(input_buffer2, uri.c_str());
 if (reader == NULL) {
 return NULL;
 }

 int options = (m_report_warnings ? 0:XML_PARSE_NOWARNING) |
   XML_PARSE_NOENT |
   XML_PARSE_PEDANTIC |
   XML_PARSE_NOCDATA;

 xmlTextReaderSetup(reader, NULL, uri.c_str(), NULL, options);

 return reader;
}

   Yes except that CVs was replaced with SVN (Subversion) and I didn't
 yet tried to look how to hook in the new infrastructure. I think there are
 pages on gnome.org about it to babystep the process 
 

Oh...


Aleksey

___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] export xmlTextReaderSetup() function

[Aleksey Sanin]

  But if really you think xmlReaderForIO() ain't proper, then go ahead.
  My experience is that xmlParserInputBufferPtr is
 complex and will confuse users  :-)

I though about that :) The reason I decided to use
xmlParserInputBufferPtr is that I can get free
objects for parsing from memory, files, ...

Then I can add my specific layers (if needed). I agree
that xmlParserInputBufferPtr is not trivial to use
but I think I figured it out :) I have the source
code :) :) :)

   See http://developer.gnome.org/tools/svn.html
 just use

Thanks! :)

Aleksey
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

[xml] export xmlTextReaderSetup() function

[Aleksey Sanin]

I have a tiny patch that exports xmlTextReaderSetup()
function. I need it for chaining xmlParserInputBuffers
(in-place compression + encryption + ...). W/o this
function, xmlNewTextReader() function is pretty much
useless ;)

Is it OK to check it in CVS?

Thanks,
Aleksey


diff -ur include/libxml/xmlreader.h include/libxml/xmlreader.h
--- include/libxml/xmlreader.h  2006-01-04 02:36:37.0 -0800
+++ include/libxml/xmlreader.h  2007-01-02 18:36:22.115928000 -0800
@@ -113,9 +113,15 @@
 const char *URI);
 XMLPUBFUN xmlTextReaderPtr XMLCALL 
xmlNewTextReaderFilename(const char *URI);
+
 XMLPUBFUN void XMLCALL 
xmlFreeTextReader   (xmlTextReaderPtr reader);
 
+XMLPUBFUN int XMLCALL  
+xmlTextReaderSetup(xmlTextReaderPtr reader,
+   xmlParserInputBufferPtr input, const char *URL,
+   const char *encoding, int options);
+
 /*
  * Iterators
  */
diff -ur win32/libxml2.def.src win32/libxml2.def.src
--- win32/libxml2.def.src   2006-02-23 14:04:52.0 -0800
+++ win32/libxml2.def.src   2007-01-02 18:33:33.463417600 -0800
@@ -1949,6 +1949,7 @@
 xmlTextReaderSetParserProp
 xmlTextReaderSetSchema
 xmlTextReaderSetStructuredErrorHandler
+xmlTextReaderSetup
 xmlTextReaderStandalone
 xmlTextReaderValue
 xmlTextReaderXmlLang
diff -ur xmlreader.c xmlreader.c
--- xmlreader.c 2007-01-02 14:41:43.190924800 -0800
+++ xmlreader.c 2007-01-02 18:31:01.995617600 -0800
@@ -4855,7 +4855,7 @@
  * 
  * Returns 0 in case of success and -1 in case of error.
  */
-static int
+int
 xmlTextReaderSetup(xmlTextReaderPtr reader,
xmlParserInputBufferPtr input, const char *URL,
const char *encoding, int options)
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] XPath child axis traversal is broken when docb is disabled (libxml2 2.6.27)

[Aleksey Sanin]

 I think this one was already reported:
http://mail.gnome.org/archives/xml/2006-October/msg00123.html
 and fixed in CVS the day after 2.6.27 was released :-).

Oh, thanks! Sorry, I did not have time to look at the cvs today.

Aleksey


___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] c14n document subsets

[Aleksey Sanin]

This callback can be used to do canonicalization in more complex
case when constructing the nodeset can be very expensive. For example,
you can take a look at how it is used inside xmlsec library:

http://www.aleksey.com/xmlsec

Aleksey

pmd wrote:
 Thank you for the quick response.
 
 I'm trying to understand the c14n code, so I'd be great if somebody can 
 tell me what the is_visible_callback is for.
 

___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] c14n - attributes

[Aleksey Sanin]

Yes, it is correct behavior. I'll take a look but I don;t
know much about schemas code so no promises

Aleksey

Enrico Silterra wrote:
 Aleksey,
 we are switching from using a dtd - to using a schema -
 and are looking for the equivalent behavior - where, when
 using a dtd, the default attributes were supplied.
 
 I am trying to understand why - what seems to be the
 correct behavior (in my opinion)  -- maybe I don't understand the c14n 
 spec --
 does not happen, and if, using schemas, there is a way to
 get it to happen.
 
 Thanks,
 Rick
 
 Aleksey Sanin wrote:
 Is there some other way to get this behavior?
 I am not sure that this would work with --schema option.
 Can you try to put default attributes into embeded DTD
 directly in the XML file, please?

 Aleksey



 
 
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] c14n - attributes

[Aleksey Sanin]

I looked at it and there is no easy way to fix this:
1) Default attributes for c14n are inserted into the
tree by xml parser. c14n code itself does not do
anything  about this.
2) As Daniel wrote, schemas support code in libxml2
is separate from the main xml parser. There is no easy
way to get access to schema data from xml parser
(at least, for my limited knowledge of this code).

I can suggest a workaround though... You can parse
xml file, then *manually* add default attributes from
the schema and then call C14N. Not the best option
but it should work

Aleksey


Aleksey Sanin wrote:
 Yes, it is correct behavior. I'll take a look but I don;t
 know much about schemas code so no promises
 
 Aleksey
 

___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] How to use xmlC14NExecute

[Aleksey Sanin]

... but I still couldn't figure out what data should I assign 

 for void * user_data actually.

void * user_data is ... user data. You can use this parameter
to pass around the context to the callback function. Libxml2
does nothing with this data, just passes it to your callback.

Aleksey

___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] Is it a BUG for exclusive XML canonicalization in LibXML2?

[Aleksey Sanin]

Anyway, the 'extra' empty default namespace generated on 
CanonicalizationMethod by LibXML2
mentioned in my first mail violates the 
http://www.w3.org/TR/xml-exc-c14n/ if my understanding of that spec is 
correct.


Yes, you are correct. There is a bug in libxml2 c14n module.
I appologize for missing it first time in the bug you've
reported:

http://bugzilla.gnome.org/show_bug.cgi?id=319367

I am working on a fix (eta 30 minutes).


Done. See bug #319367 for checkin log.

Thanks again for noticing and reporting this problem!

Aleksey
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] Error of Canonical while replacing External Entities

[Aleksey Sanin]

According to C14N specification, it is responsible for doing
these replacements

 The XML processor performs the following tasks in order:
1. normalize line feeds
2. normalize attribute values
3. replace CDATA sections with their character content
4. resolve character and parsed entity references

and you have to have an xmlDocPtr which was loaded with all
the flags you describe. If you are not sure then you can
serialize the document to a buffer and parse it again. Note
that instead of setting global parser flags you can set them
in the parser context and then these flags will be used for
the parsed document only.

Aleksey


___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

[xml] Re: patch to make gcc 2.96 happy

[Aleksey Sanin]

Ops, sorry. Let me take it back. It is not about gcc but about
me not able to distinguish HTTP and HTML words :(

Sorry,
Aleksey

Aleksey Sanin wrote:

I have a small patch (attached) to make gcc 2.96 happy (don't ask
me why it did not like the original code). I hope nobody would mind
if I check it in.

Thanks,
Aleksey


___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

[xml] patch to testapi.c

[Aleksey Sanin]

Fixes testapi.c compilation with --enable-html --disable-http options.

Aleksey
--- testapi.c.orig  Sat Apr  2 00:44:00 2005
+++ testapi.c   Tue Jun 28 06:14:28 2005
@@ -675,6 +675,9 @@
xmlNanoHTTPClose(val);
 }
 }
+static void desret_xmlNanoHTTPCtxtPtr(void *val) {
+xmlNanoHTTPClose(val);
+}
 #endif
 
 #define gen_nb_xmlCharEncoding 4
@@ -814,9 +817,6 @@
 #ifdef LIBXML_HTML_ENABLED
 static void desret_const_htmlEntityDesc_ptr(const htmlEntityDesc * val 
ATTRIBUTE_UNUSED) {
 }
-static void desret_xmlNanoHTTPCtxtPtr(void *val) {
-xmlNanoHTTPClose(val);
-}
 #endif
 #ifdef LIBXML_FTP_ENABLED
 static void desret_xmlNanoFTPCtxtPtr(void *val) {
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] c14n

[Aleksey Sanin]

The attached patch special cases DAV: namespace in c14n's
relative namespaces check and replaces #ifdef DEBUG_C14N/#endif
with structured error messages as you described bellow. Please
take a look and let me know if I can check it in or need to
do something.
Aleksey
Aleksey Sanin wrote:
Sounds good. I'll do it this week (I hope).
Aleksey
Daniel Veillard wrote:
On Tue, Mar 15, 2005 at 10:13:32AM -0800, Aleksey Sanin wrote:
Currently, all c14n errors are ifdef'd with DEBUG_C14N. Should
we just turn them on by default?

Depends:
   1/ do they use the existing structured error reporting framework ?
  I think so but double checking quickly would be good

Yes. Errors are printed out using
   xmlGenericError(xmlGenericErrorContext, ...)
calls.

  Hum, it's not the structured API. The structured one provides more
informations, more generically, see include/libxml/xmlerror.h,
there is a XML_FROM_C14N error domain, and an error range for C14N
(starting at XML_C14N_CREATE_CTXT 1950 to 2000) for error numbers.
Best is to define in c14n.c a couple of intermediate calls which sets
the context up and call __xmlRaiseError or __xmlSimpleError, see for
example xmlTreeErrMemory() and xmlTreeErr() at the beginning of the 
tree.c module. Not really fun to do, defining new enums for errors
and calling the intermediate functions but it makes a huge difference
when the library is embedded in a more complex framework.


   2/ is there any drawback for example speed wise ?

I don't think so. It is only error case which should not happen during
normal run. I'll make sure that only errors will be undefed.

  The main drawback can be the code size inflation, but using 
intermediate
simpler calls at the beginning of the modules avoid most of the code 
size increase (don't get me started on ELF symbols lookup at runtime ...)


   3/ would that be a problem for upper layer like xmldsig ?

Not to xmldsig. It should be no difference for normal run.

  okay, though the structured error handling might be useful for your
users too ;-)

In general people appreciate receiving error messages when stuff fails.

Yes, I know :)

 especially if they can automate the processing of said errors :-)
Daniel
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml
Index: c14n.c
===
RCS file: /cvs/gnome/gnome-xml/c14n.c,v
retrieving revision 1.22
diff -u -w -r1.22 c14n.c
--- c14n.c  25 May 2004 17:39:48 -  1.22
+++ c14n.c  20 Mar 2005 02:25:10 -
@@ -127,7 +127,7 @@
  * xmlC14NErrMemory:
  * @extra:  extra informations
  *
- * Handle a redefinition of attribute error
+ * Handle a redefinition of memory error
  */
 static void
 xmlC14NErrMemory(const char *extra)
@@ -139,6 +139,83 @@
 }
 
 /**
+ * xmlC14NErrParam:
+ * @extra:  extra informations
+ *
+ * Handle a redefinition of param error
+ */
+static void
+xmlC14NErrParam(const char *extra)
+{
+__xmlRaiseError(NULL, NULL, NULL, NULL, NULL, XML_FROM_C14N,
+   XML_ERR_INTERNAL_ERROR, XML_ERR_ERROR, NULL, 0, extra,
+   NULL, NULL, 0, 0,
+   Invalid parameter : %s\n, extra);
+}
+
+/**
+ * xmlC14NErrInternal:
+ * @extra:  extra informations
+ *
+ * Handle a redefinition of internal error
+ */
+static void
+xmlC14NErrInternal(const char *extra)
+{
+__xmlRaiseError(NULL, NULL, NULL, NULL, NULL, XML_FROM_C14N,
+   XML_ERR_INTERNAL_ERROR, XML_ERR_ERROR, NULL, 0, extra,
+   NULL, NULL, 0, 0,
+   Internal error : %s\n, extra);
+}
+
+/**
+ * xmlC14NErrInvalidNode:
+ * @extra:  extra informations
+ *
+ * Handle a redefinition of invalid node error
+ */
+static void
+xmlC14NErrInvalidNode(const char *node_type, const char *extra)
+{
+__xmlRaiseError(NULL, NULL, NULL, NULL, NULL, XML_FROM_C14N,
+   XML_C14N_INVALID_NODE, XML_ERR_ERROR, NULL, 0, extra,
+   NULL, NULL, 0, 0,
+   Node %s is invalid here : %s\n, node_type, extra);
+}
+
+/**
+ * xmlC14NErrUnknownNode:
+ * @extra:  extra informations
+ *
+ * Handle a redefinition of unknown node error
+ */
+static void
+xmlC14NErrUnknownNode(int node_type, const char *extra)
+{
+__xmlRaiseError(NULL, NULL, NULL, NULL, NULL, XML_FROM_C14N,
+   XML_C14N_UNKNOW_NODE, XML_ERR_ERROR, NULL, 0, extra,
+   NULL, NULL, 0, 0,
+   Unknown node type %d found : %s\n, node_type, extra);
+}
+
+/**
+ * xmlC14NErrRelativeNamespace:
+ * @extra:  extra informations
+ *
+ * Handle a redefinition of relative namespace error
+ */
+static void
+xmlC14NErrRelativeNamespace(const char *ns_uri)
+{
+__xmlRaiseError(NULL, NULL, NULL, NULL, NULL, XML_FROM_C14N,
+   XML_C14N_RELATIVE_NAMESPACE, XML_ERR_ERROR, NULL, 0, NULL,
+   NULL, NULL, 0, 0,
+   Relative

Re: [xml] c14n

[Aleksey Sanin]

The most likely reason for c14n failure in your case is
DAV: namespace. Most likely, it failed on xml parsing stage.
Aleksey
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] c14n

[Aleksey Sanin]

Then it fails in relative namespace check inside C14N
(see notes in section 2.1 of c14n spec:
http://www.w3.org/TR/2001/REC-xml-c14n-20010315)
Aleksey
Daniel Veillard wrote:
On Tue, Mar 15, 2005 at 07:26:13AM -0800, Aleksey Sanin wrote:
The most likely reason for c14n failure in your case is
DAV: namespace. Most likely, it failed on xml parsing stage.

  hum, no, it's only a warning, garanteed !
Daniel
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] c14n

[Aleksey Sanin]

either way, it would be nice if the error reported explained the problem
a bit better.

  that I agree with,
Currently, all c14n errors are ifdef'd with DEBUG_C14N. Should
we just turn them on by default?
Aleksey
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] c14n

[Aleksey Sanin]

Indeed. however, I found
http://cvs.apache.org/viewcvs.cgi/*checkout*/ietf-uri/rev-2002/issues.html?rev=HEADcontent-type=text/html#014-empty-opaque_part
so it will be allowed in the future. Any chance of getting this specific
bit in before the new rfc gets published?
I don't have any problem with it. Daniel?
Aleksey
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] c14n

[Aleksey Sanin]

On Tue, Mar 15, 2005 at 10:13:53AM -0800, Aleksey Sanin wrote:
Indeed. however, I found
http://cvs.apache.org/viewcvs.cgi/*checkout*/ietf-uri/rev-2002/issues.html?rev=HEADcontent-type=text/html#014-empty-opaque_part
so it will be allowed in the future. Any chance of getting this specific
bit in before the new rfc gets published?
I don't have any problem with it. Daniel?

  It's a bit more general than just WebDAV, see my other post about 3986,
Yes, I've seen it. But I can hard code DAV: exception in c14n for now.
It's ugly but I am not sure there is other workaround for Martijn.
Aleksey
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] c14n

[Aleksey Sanin]

Sounds good. I'll do it this week (I hope).
Aleksey
Daniel Veillard wrote:
On Tue, Mar 15, 2005 at 10:13:32AM -0800, Aleksey Sanin wrote:
Currently, all c14n errors are ifdef'd with DEBUG_C14N. Should
we just turn them on by default?

Depends:
   1/ do they use the existing structured error reporting framework ?
  I think so but double checking quickly would be good
Yes. Errors are printed out using
   xmlGenericError(xmlGenericErrorContext, ...)
calls.

  Hum, it's not the structured API. The structured one provides more
informations, more generically, see include/libxml/xmlerror.h,
there is a XML_FROM_C14N error domain, and an error range for C14N
(starting at XML_C14N_CREATE_CTXT 1950 to 2000) for error numbers.
Best is to define in c14n.c a couple of intermediate calls which sets
the context up and call __xmlRaiseError or __xmlSimpleError, see for
example xmlTreeErrMemory() and xmlTreeErr() at the beginning of 
the tree.c module. Not really fun to do, defining new enums for errors
and calling the intermediate functions but it makes a huge difference
when the library is embedded in a more complex framework.


   2/ is there any drawback for example speed wise ?
I don't think so. It is only error case which should not happen during
normal run. I'll make sure that only errors will be undefed.

  The main drawback can be the code size inflation, but using intermediate
simpler calls at the beginning of the modules avoid most of the code 
size increase (don't get me started on ELF symbols lookup at runtime ...)


   3/ would that be a problem for upper layer like xmldsig ?
Not to xmldsig. It should be no difference for normal run.

  okay, though the structured error handling might be useful for your
users too ;-)

In general people appreciate receiving error messages when stuff fails.
Yes, I know :)

 especially if they can automate the processing of said errors :-)
Daniel
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml