Re: [xml] xmlSaveToFd: unexpected behavior while redirecting the 'stdout' stream to a file

[YuGiOhJCJ Mailing-List via xml]

Just to clarify the reason why I am using buffered IO only, I am currently 
coding a dynamic website that is written in the C language.
After compiling the C code, I upload the program to a web server (Apache).
On the web browser, when I try to access to the website, there is an error: 
Internal Server Error.
In the Apache logs I see: malformed header from script 'index.cgi': Bad header: 
.
So, I was wondering why I got a bad header whereas it was the first thing 
written by my program to the 'stdout' stream.
Now, I have the answer: I need to use buffered IO only.

Remark: I don't know exactly how Apache is managing the CGI file but I guess it 
does something similar to redirecting the 'stdout' stream to a file because I 
get the same behavior when I try do that.

On Sun, 15 Apr 2018 11:33:51 -0700
"Eric S. Eberhard"  wrote:

> This is a 100% correct answer.  Why anyone on a modern computer would 
> even used buffered anything except extreme cases is beyond me.  I know I 
> cannot do better than AIX no matter how I code it.  Since I can write 
> files of many gigs (and read them) ... I can no longer perceive a true 
> need (except us old guys like to be efficient) for buffering.  The code 
> would be simpler if ripped out.  Oh -- one can easily write say 500 meg, 
> start a new file, etc -- and cat them all together.  I just think 2 gig 
> XML is absurd.
> 
> We do have XML feeds that are indeed well over 2 gig.  And many customer 
> could not parse it, read it, or whatever.  We found a simpler solution 
> -- the files have two tags:\
> 
>  and  ... hence when I start to hit a (user defined) 
> limit of say 500 meg, and I have already done the math ... they will get 
> "47 ... this allows 
> unlimited file sizes to be accepted by all machines.  It also helps in 
> file transport -- because if you are missing #3 -- just get #3.  We also 
> no longer send any important XML in URLs ... we send:
> 
> 
> 
> 
> 
> And let the customer get it when they want.  We found once we changed 
> that reduced the XML traffic by 2/3 (we do 2-4 million transactions per 
> day so it really matters).  If there is an error getting the file, get 
> it again. And again if needed.
> 
> Using date/time stamps is also important -- for example we send XML for 
> order status.  We are FAST.  We can receive XML, print a pick ticket, 
> pick it, ship it (all automated with baggers and boxers and label 
> applicators),  rate shop, produce all shipping documents including 
> invoice, update inventory, etc ... in minutes.  If the customer has 
> "silly" software they expect each step.  We don't.  It is 
> order_1234567.xml period.  Obviously if it is sitting on the UPS truck 
> the odds are good we loaded the order, printed a picking ticket, packed 
> it, invoiced, and placed on the UPS electronic manifest.  So date/time 
> stamps can save a LOT of time.
> 
> Anyone that want help with a large interesting XML project or a small 
> easy quick question is free to contact me.
> 
> Also -- please be CAREFUL with stdout and stderr .. 1>&2 1>filename.txt 
> will get all the output, not just the "good" stuff.
> 
> Sorry, long answer :-)
> 
> Eric
> 
> On 4/4/2018 4:37 AM, Nick Wellnhofer wrote:
> > On 04/04/2018 13:28, YuGiOhJCJ Mailing-List wrote:
> >> In order to use buffered IO only, I am now using the 'xmlSaveFile' 
> >> function (instead of 'xmlSaveToFd'):
> >
> >> However, with this solution, I loose the 'options' parameter that was 
> >> available with the 'xmlSaveToFd' function.
> >> Is there a similar solution with the 'options' parameter please?
> >
> > You can use xmlSaveToFilename with "-" as filename. There's also 
> > xmlSaveToIO taking custom IO callbacks.
> >
> > Nick
> > ___
> > xml mailing list, project page  http://xmlsoft.org/
> > xml@gnome.org
> > https://mail.gnome.org/mailman/listinfo/xml
> >
> 
> -- 
> Eric S. Eberhard
> VICS
> 2933 W Middle Verde Road
> Camp Verde, AZ  86322
> 
> 928-567-3727  work  928-301-7537  cell
> 
> http://www.vicsmba.com/index.html (our work)
> http://www.vicsmba.com/ourpics/index.html (fun pictures)
> 
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
https://mail.gnome.org/mailman/listinfo/xml

Re: [xml] xmlSaveToFd: unexpected behavior while redirecting the 'stdout' stream to a file

[YuGiOhJCJ Mailing-List via xml]

Thanks, indeed the 'xmlSaveToFilename' function is what I was looking for.

Here is the code:
---
#include  /* for xmlSaveToFilename */
#include  /* for printf */
#include  /* for strlen */
int main()
{
const char *str = "http://www.w3.org/1999/xhtml\;>\n\ntest\n\n\ntest\n\n\n";
xmlSaveCtxtPtr ctxt = NULL;
xmlDocPtr doc = NULL;
printf("Content-Type: application/xhtml+xml;\n\n");
doc = xmlParseMemory(str, strlen(str) + 1);
ctxt = xmlSaveToFilename("-", NULL, XML_SAVE_FORMAT);
if(ctxt == NULL)
{
fprintf(stderr, "Unable to create the document saving 
context.\n");
exit(EXIT_FAILURE);
}
if(xmlSaveDoc(ctxt, doc) == -1)
{
fprintf(stderr, "Unable to save the document to the document 
saving context.\n");
exit(EXIT_FAILURE);
}
if(xmlSaveClose(ctxt) == -1)
{
fprintf(stderr, "Unable to close the document.\n");
exit(EXIT_FAILURE);
}
xmlFreeDoc(doc);
xmlCleanupParser();
exit(EXIT_SUCCESS);
}
---

Here is the expected behavior while redirecting the 'stdout' stream to a file:
---
$ ./main.out 1>1.txt
$ cat 1.txt 
Content-Type: application/xhtml+xml;


http://www.w3.org/1999/xhtml;>

test


test


---

Problem solved!

On Wed, 4 Apr 2018 13:37:23 +0200
Nick Wellnhofer  wrote:

> On 04/04/2018 13:28, YuGiOhJCJ Mailing-List wrote:
> > In order to use buffered IO only, I am now using the 'xmlSaveFile' function 
> > (instead of 'xmlSaveToFd'):
> 
> > However, with this solution, I loose the 'options' parameter that was 
> > available with the 'xmlSaveToFd' function.
> > Is there a similar solution with the 'options' parameter please?
> 
> You can use xmlSaveToFilename with "-" as filename. There's also xmlSaveToIO 
> taking custom IO callbacks.
> 
> Nick
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
https://mail.gnome.org/mailman/listinfo/xml

Re: [xml] xmlSaveToFd: unexpected behavior while redirecting the 'stdout' stream to a file

[YuGiOhJCJ Mailing-List via xml]

Thanks for the explanation.

In order to use buffered IO only, I am now using the 'xmlSaveFile' function 
(instead of 'xmlSaveToFd'):
---
#include  /* for xmlSaveFile */
#include  /* for printf */
#include  /* for strlen */
int main()
{
const char *str = "http://www.w3.org/1999/xhtml\;>\n\ntest\n\n\ntest\n\n\n";
xmlDocPtr doc = NULL;
printf("Content-Type: application/xhtml+xml;\n\n");
doc = xmlParseMemory(str, strlen(str) + 1);
if(xmlSaveFile("-", doc) == -1)
{
fprintf(stderr, "Unable to dump the document to a file.\n");
exit(EXIT_FAILURE);
}
xmlFreeDoc(doc);
xmlCleanupParser();
exit(EXIT_SUCCESS);
}
---

Now, I have the expected behavior while redirecting the 'stdout' stream to a 
file:
---
$ ./main.out 1>1.txt
$ cat 1.txt
Content-Type: application/xhtml+xml;


http://www.w3.org/1999/xhtml;>

test


test


---

However, with this solution, I loose the 'options' parameter that was available 
with the 'xmlSaveToFd' function.
Is there a similar solution with the 'options' parameter please?

On Tue, 3 Apr 2018 14:28:24 +0200
Nick Wellnhofer  wrote:

> On 03/04/2018 14:02, YuGiOhJCJ Mailing-List via xml wrote:
> > printf("Content-Type: application/xhtml+xml;\n\n");
> 
> This operates on the stdout FILE pointer using buffered IO.
> 
> > ctxt = xmlSaveToFd(1, NULL, XML_SAVE_FORMAT);
> 
> This operates on the stdout file descriptor using unbuffered IO. You're 
> mixing 
> buffered and unbuffered IO.
> 
> Nick
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
https://mail.gnome.org/mailman/listinfo/xml