[xml] xmlSaveToFd: unexpected behavior while redirecting the 'stdout' stream to a file

[YuGiOhJCJ Mailing-List via xml]

Hello,

I think I found an unexpected behavior of the 'xmlSaveToFd' function while 
redirecting the 'stdout' stream to a file, with libxml2 2.9.4.

Here is the code:
---
#include  /* for xmlSaveToFd */
#include  /* for printf */
#include  /* for strlen */
int main()
{
const char *str = "http://www.w3.org/1999/xhtml\;>\n\ntest\n\n\ntest\n\n\n";
xmlSaveCtxtPtr ctxt = NULL;
xmlDocPtr doc = NULL;
printf("Content-Type: application/xhtml+xml;\n\n");
/* The line below is really important.
 * If it is omitted, then the command './main.out 1>1.txt' will 
redirect lines from the 'stdout' stream to the '1.txt' file in the wrong order.
 */
/* fflush(stdout); */
doc = xmlParseMemory(str, strlen(str) + 1);
ctxt = xmlSaveToFd(1, NULL, XML_SAVE_FORMAT);
if(ctxt == NULL)
{
fprintf(stderr, "Unable to create the document saving 
context.\n");
exit(EXIT_FAILURE);
}
if(xmlSaveDoc(ctxt, doc) == -1)
{
fprintf(stderr, "Unable to save the document to the document 
saving context.\n");
exit(EXIT_FAILURE);
}
if(xmlSaveClose(ctxt) == -1)
{
fprintf(stderr, "Unable to close the document.\n");
exit(EXIT_FAILURE);
}
xmlFreeDoc(doc);
xmlCleanupParser();
exit(EXIT_SUCCESS);
}
---

Here is the unexpected behavior while redirecting the 'stdout' stream to a file:
---
$ ./main.out 1>1.txt
$ cat 1.txt 

http://www.w3.org/1999/xhtml;>

test


test


Content-Type: application/xhtml+xml;

---

As you can see above, the string printed with the 'printf' function is shown 
after the string printed with the 'xmlSaveToFd' function.
However, it only happens while redirecting the 'stdout' stream to a file.
Indeed, here is what happens when I run the program without redirecting:
---
$ ./main.out 
Content-Type: application/xhtml+xml;


http://www.w3.org/1999/xhtml;>

test


test


---

A workaround is to call the 'fflush' function just after the call to the 
'printf' function (see the commented line in the code above).
But I really don't understand why it is necessary to flush the steam because 
the '\n' character should already force the stream to be written on the screen.

Is it a bug in libxml2?
Can someone explain this behavior please?

Thank you.
Best regards.
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
https://mail.gnome.org/mailman/listinfo/xml

Re: [xml] xmlSaveToFd: unexpected behavior while redirecting the 'stdout' stream to a file

[YuGiOhJCJ Mailing-List via xml]

Thanks for the explanation.

In order to use buffered IO only, I am now using the 'xmlSaveFile' function 
(instead of 'xmlSaveToFd'):
---
#include  /* for xmlSaveFile */
#include  /* for printf */
#include  /* for strlen */
int main()
{
const char *str = "http://www.w3.org/1999/xhtml\;>\n\ntest\n\n\ntest\n\n\n";
xmlDocPtr doc = NULL;
printf("Content-Type: application/xhtml+xml;\n\n");
doc = xmlParseMemory(str, strlen(str) + 1);
if(xmlSaveFile("-", doc) == -1)
{
fprintf(stderr, "Unable to dump the document to a file.\n");
exit(EXIT_FAILURE);
}
xmlFreeDoc(doc);
xmlCleanupParser();
exit(EXIT_SUCCESS);
}
---

Now, I have the expected behavior while redirecting the 'stdout' stream to a 
file:
---
$ ./main.out 1>1.txt
$ cat 1.txt
Content-Type: application/xhtml+xml;


http://www.w3.org/1999/xhtml;>

test


test


---

However, with this solution, I loose the 'options' parameter that was available 
with the 'xmlSaveToFd' function.
Is there a similar solution with the 'options' parameter please?

On Tue, 3 Apr 2018 14:28:24 +0200
Nick Wellnhofer <wellnho...@aevum.de> wrote:

> On 03/04/2018 14:02, YuGiOhJCJ Mailing-List via xml wrote:
> > printf("Content-Type: application/xhtml+xml;\n\n");
> 
> This operates on the stdout FILE pointer using buffered IO.
> 
> > ctxt = xmlSaveToFd(1, NULL, XML_SAVE_FORMAT);
> 
> This operates on the stdout file descriptor using unbuffered IO. You're 
> mixing 
> buffered and unbuffered IO.
> 
> Nick
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
https://mail.gnome.org/mailman/listinfo/xml

Re: [xml] xmlSaveToFd: unexpected behavior while redirecting the 'stdout' stream to a file

[YuGiOhJCJ Mailing-List via xml]

Thanks, indeed the 'xmlSaveToFilename' function is what I was looking for.

Here is the code:
---
#include  /* for xmlSaveToFilename */
#include  /* for printf */
#include  /* for strlen */
int main()
{
const char *str = "http://www.w3.org/1999/xhtml\;>\n\ntest\n\n\ntest\n\n\n";
xmlSaveCtxtPtr ctxt = NULL;
xmlDocPtr doc = NULL;
printf("Content-Type: application/xhtml+xml;\n\n");
doc = xmlParseMemory(str, strlen(str) + 1);
ctxt = xmlSaveToFilename("-", NULL, XML_SAVE_FORMAT);
if(ctxt == NULL)
{
fprintf(stderr, "Unable to create the document saving 
context.\n");
exit(EXIT_FAILURE);
}
if(xmlSaveDoc(ctxt, doc) == -1)
{
fprintf(stderr, "Unable to save the document to the document 
saving context.\n");
exit(EXIT_FAILURE);
}
if(xmlSaveClose(ctxt) == -1)
{
fprintf(stderr, "Unable to close the document.\n");
exit(EXIT_FAILURE);
}
xmlFreeDoc(doc);
xmlCleanupParser();
exit(EXIT_SUCCESS);
}
---

Here is the expected behavior while redirecting the 'stdout' stream to a file:
---
$ ./main.out 1>1.txt
$ cat 1.txt 
Content-Type: application/xhtml+xml;


http://www.w3.org/1999/xhtml;>

test


test


---

Problem solved!

On Wed, 4 Apr 2018 13:37:23 +0200
Nick Wellnhofer  wrote:

> On 04/04/2018 13:28, YuGiOhJCJ Mailing-List wrote:
> > In order to use buffered IO only, I am now using the 'xmlSaveFile' function 
> > (instead of 'xmlSaveToFd'):
> 
> > However, with this solution, I loose the 'options' parameter that was 
> > available with the 'xmlSaveToFd' function.
> > Is there a similar solution with the 'options' parameter please?
> 
> You can use xmlSaveToFilename with "-" as filename. There's also xmlSaveToIO 
> taking custom IO callbacks.
> 
> Nick
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
https://mail.gnome.org/mailman/listinfo/xml

Re: [xml] xmlSaveToFd: unexpected behavior while redirecting the 'stdout' stream to a file

[YuGiOhJCJ Mailing-List via xml]

Just to clarify the reason why I am using buffered IO only, I am currently 
coding a dynamic website that is written in the C language.
After compiling the C code, I upload the program to a web server (Apache).
On the web browser, when I try to access to the website, there is an error: 
Internal Server Error.
In the Apache logs I see: malformed header from script 'index.cgi': Bad header: 
.
So, I was wondering why I got a bad header whereas it was the first thing 
written by my program to the 'stdout' stream.
Now, I have the answer: I need to use buffered IO only.

Remark: I don't know exactly how Apache is managing the CGI file but I guess it 
does something similar to redirecting the 'stdout' stream to a file because I 
get the same behavior when I try do that.

On Sun, 15 Apr 2018 11:33:51 -0700
"Eric S. Eberhard"  wrote:

> This is a 100% correct answer.  Why anyone on a modern computer would 
> even used buffered anything except extreme cases is beyond me.  I know I 
> cannot do better than AIX no matter how I code it.  Since I can write 
> files of many gigs (and read them) ... I can no longer perceive a true 
> need (except us old guys like to be efficient) for buffering.  The code 
> would be simpler if ripped out.  Oh -- one can easily write say 500 meg, 
> start a new file, etc -- and cat them all together.  I just think 2 gig 
> XML is absurd.
> 
> We do have XML feeds that are indeed well over 2 gig.  And many customer 
> could not parse it, read it, or whatever.  We found a simpler solution 
> -- the files have two tags:\
> 
>  and  ... hence when I start to hit a (user defined) 
> limit of say 500 meg, and I have already done the math ... they will get 
> "47 ... this allows 
> unlimited file sizes to be accepted by all machines.  It also helps in 
> file transport -- because if you are missing #3 -- just get #3.  We also 
> no longer send any important XML in URLs ... we send:
> 
> 
> 
> 
> 
> And let the customer get it when they want.  We found once we changed 
> that reduced the XML traffic by 2/3 (we do 2-4 million transactions per 
> day so it really matters).  If there is an error getting the file, get 
> it again. And again if needed.
> 
> Using date/time stamps is also important -- for example we send XML for 
> order status.  We are FAST.  We can receive XML, print a pick ticket, 
> pick it, ship it (all automated with baggers and boxers and label 
> applicators),  rate shop, produce all shipping documents including 
> invoice, update inventory, etc ... in minutes.  If the customer has 
> "silly" software they expect each step.  We don't.  It is 
> order_1234567.xml period.  Obviously if it is sitting on the UPS truck 
> the odds are good we loaded the order, printed a picking ticket, packed 
> it, invoiced, and placed on the UPS electronic manifest.  So date/time 
> stamps can save a LOT of time.
> 
> Anyone that want help with a large interesting XML project or a small 
> easy quick question is free to contact me.
> 
> Also -- please be CAREFUL with stdout and stderr .. 1>&2 1>filename.txt 
> will get all the output, not just the "good" stuff.
> 
> Sorry, long answer :-)
> 
> Eric
> 
> On 4/4/2018 4:37 AM, Nick Wellnhofer wrote:
> > On 04/04/2018 13:28, YuGiOhJCJ Mailing-List wrote:
> >> In order to use buffered IO only, I am now using the 'xmlSaveFile' 
> >> function (instead of 'xmlSaveToFd'):
> >
> >> However, with this solution, I loose the 'options' parameter that was 
> >> available with the 'xmlSaveToFd' function.
> >> Is there a similar solution with the 'options' parameter please?
> >
> > You can use xmlSaveToFilename with "-" as filename. There's also 
> > xmlSaveToIO taking custom IO callbacks.
> >
> > Nick
> > ___
> > xml mailing list, project page  http://xmlsoft.org/
> > xml@gnome.org
> > https://mail.gnome.org/mailman/listinfo/xml
> >
> 
> -- 
> Eric S. Eberhard
> VICS
> 2933 W Middle Verde Road
> Camp Verde, AZ  86322
> 
> 928-567-3727  work  928-301-7537  cell
> 
> http://www.vicsmba.com/index.html (our work)
> http://www.vicsmba.com/ourpics/index.html (fun pictures)
> 
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
https://mail.gnome.org/mailman/listinfo/xml