Re: [petsc-dev] [petsc-users] PetscPrintf
Thanks Barry. I will update my petsc. By the way, I simply replaced the functions PetscFormatConvert() and PetscVFPrintfDefault() with those in petsc-3.6 and it works for me. Best, Rongliang On 04/17/2018 05:53 AM, Smith, Barry F. wrote: Rongliang, I have prepared a fix for this in the branch barry/bugfix-petscformatconvert which comes from the master branch of PETSc. Unfortunately it requires many changes so cannot be back ported to previous releases. Thanks for reporting the problem, Barry On Apr 11, 2018, at 6:17 PM, Rongliang Chenwrote: Dear All, When I tried to print a long string using PetscPrintf() I found that it truncated the string. Attached is a simple example for this (run with single processor). I used PetscPrintf() and printf() to print the same string and the printf() seems OK. I am using petsc-3.8.4. Best, Rongliang
Re: [petsc-dev] [petsc-users] PetscPrintf
Rongliang, I have prepared a fix for this in the branch barry/bugfix-petscformatconvert which comes from the master branch of PETSc. Unfortunately it requires many changes so cannot be back ported to previous releases. Thanks for reporting the problem, Barry > On Apr 11, 2018, at 6:17 PM, Rongliang Chenwrote: > > Dear All, > > > When I tried to print a long string using PetscPrintf() I found that it > truncated the string. Attached is a simple example for this (run with single > processor). I used PetscPrintf() and printf() to print the same string and > the printf() seems OK. I am using petsc-3.8.4. > > > Best, > > Rongliang > >
Re: [petsc-dev] [petsc-users] PetscPrintf
Very small step in the correct direction https://bitbucket.org/petsc/petsc/pull-requests/931/barry-bugfix-petscformatconvert/diff > On Apr 15, 2018, at 5:41 AM, Patrick Sananwrote: > > How about the logic of this analysis? > > 1. We are trying to use the same functions (in particular, PetscVFPrintf) for > two purposes: >a. printing error messages (don't want to malloc) >b. use by public API printing functions (don't want length restrictions) > > 2. Right now, PetscVFPrintf works fine for a but not for b. We could make it > work for b and not for a by malloc'ing a longer string. > > 3. Printing from error handlers happens through PetscErrorPrintf (default : > http://www.mcs.anl.gov/petsc/petsc-dev/src/sys/error/errtrace.c.html#PetscErrorPrintfDefault > ), so if there's a special requirement for printing error messages, we can > impose it here. > > A solution could then be something which skips the malloc only when printing > an error, e.g. > > 1. Add an argument to PetscVFPrintf (say "PetscBool noMalloc") [1] > 2. API (PetscPrintf(), etc.) functions use noMalloc = PETSC_FALSE > 3. error functions (PetscErrorPrintf() ) functions use noMalloc = PETSC_TRUE > > > [1] And probably do the same thing in PetscVSNPrintf since, as Dr. Zhang > pointed out, this could also call malloc while handling an error, if the > string was long enough > > > > > 2018-04-13 15:59 GMT+02:00 Junchao Zhang : > On Thu, Apr 12, 2018 at 9:48 AM, Smith, Barry F. wrote: > > > > On Apr 12, 2018, at 3:59 AM, Patrick Sanan wrote: > > > > I also happened to stumble across this yesterday. Is the length restriction > > for the default printer (l assume from the array of 8*1024 chars in > > PetscVFPrintfDefault() ) intended behavior to be documented, or a bug to be > > fixed? > > You could call it either. My problems are > > 1) that given a format string I don't know in advance how much work space is > needed so cannot accurately malloc, for sure, enough space > > 2) since this can be called in an error handler I really don't want it > calling malloc(). > PetscVSNPrintf does still contain a malloc "122 ierr = > PetscMalloc1(oldLength, );CHKERRQ(ierr);" > Also, vsnprintf returns "the number of characters that would have been > written if n had been sufficiently large". I don't know why you void'ed it. > We can not make the 8K chars a requirement since users don't know how many > chars they want to print upfront. > Anyway, crash is better than silent errors. > >Hence it lives in this limbo. I don't even know a way to add a good error > checking that detects if the buffer is long enough. All in all it is bad ugly > code, any suggestions on improvements would be appreciated. > >Barry > > > > > 2018-04-12 2:16 GMT+02:00 Rongliang Chen : > > Thanks Barry. I found petsc-3.6 and older versions did not have this > > restriction. > > > > Best, > > Rongliang > > > > > > On 04/12/2018 07:22 AM, Smith, Barry F. wrote: > >Yes, PetscPrintf() and related functions have a maximum string length of > > about 8000 characters. > > > > Barry > > > > > > On Apr 11, 2018, at 6:17 PM, Rongliang Chen > > wrote: > > > > Dear All, > > > > > > When I tried to print a long string using PetscPrintf() I found that it > > truncated the string. Attached is a simple example for this (run with > > single processor). I used PetscPrintf() and printf() to print the same > > string and the printf() seems OK. I am using petsc-3.8.4. > > > > > > Best, > > > > Rongliang > > > > > > > > > > > > >
Re: [petsc-dev] [petsc-users] PetscPrintf
To me the difficulty is knowing how large an array to allocate (in both PetscVSNPrintf() and PetscVFPrintfDefault()). The macro PETSC_MAX_LENGTH_FORMAT() is used in PetscVSNPrintf() but it has nothing to do with reality. One could walk through the formats (and their string values) to get a much better estimate for total length of buffer needed. Maybe a routine something like va_list Argp; va_start(Argp,format); ierr = PetscDetermineOutputLength(format,Argp,size_t *neededlength);CHKERRQ(ierr); PetscMalloc(neededlength,); But I am not convinced we can't just have an upper bound on the possible format length (so long as we can error out properly if the user requests a something that doesn't fit). Barry Note: there are horrible other buffer size code fragments such as next->size = -1; while ((PetscInt)fullLength >= next->size) { next->size = fullLength+1; ierr = PetscMalloc1(next->size, >string);CHKERRQ(ierr); va_start(Argp,format); ierr = PetscMemzero(next->string,next->size);CHKERRQ(ierr); ierr = PetscVSNPrintf(next->string,next->size,format, ,Argp);CHKERRQ(ierr); va_end(Argp); } that seems to try (very badly) to allocate larger and larger buffers (without freeing the previous?) until it has enough room for the result ? Any fix should resolve this as well. Maybe two versions of PetscVSNPrintf() one that takes a buffer (and respects it) and one that allocates a buffer large enough. > On Apr 15, 2018, at 5:41 AM, Patrick Sananwrote: > > How about the logic of this analysis? > > 1. We are trying to use the same functions (in particular, PetscVFPrintf) for > two purposes: >a. printing error messages (don't want to malloc) >b. use by public API printing functions (don't want length restrictions) > > 2. Right now, PetscVFPrintf works fine for a but not for b. We could make it > work for b and not for a by malloc'ing a longer string. > > 3. Printing from error handlers happens through PetscErrorPrintf (default : > http://www.mcs.anl.gov/petsc/petsc-dev/src/sys/error/errtrace.c.html#PetscErrorPrintfDefault > ), so if there's a special requirement for printing error messages, we can > impose it here. > > A solution could then be something which skips the malloc only when printing > an error, e.g. > > 1. Add an argument to PetscVFPrintf (say "PetscBool noMalloc") [1] > 2. API (PetscPrintf(), etc.) functions use noMalloc = PETSC_FALSE > 3. error functions (PetscErrorPrintf() ) functions use noMalloc = PETSC_TRUE > > > [1] And probably do the same thing in PetscVSNPrintf since, as Dr. Zhang > pointed out, this could also call malloc while handling an error, if the > string was long enough > > > > > 2018-04-13 15:59 GMT+02:00 Junchao Zhang : > On Thu, Apr 12, 2018 at 9:48 AM, Smith, Barry F. wrote: > > > > On Apr 12, 2018, at 3:59 AM, Patrick Sanan wrote: > > > > I also happened to stumble across this yesterday. Is the length restriction > > for the default printer (l assume from the array of 8*1024 chars in > > PetscVFPrintfDefault() ) intended behavior to be documented, or a bug to be > > fixed? > > You could call it either. My problems are > > 1) that given a format string I don't know in advance how much work space is > needed so cannot accurately malloc, for sure, enough space > > 2) since this can be called in an error handler I really don't want it > calling malloc(). > PetscVSNPrintf does still contain a malloc "122 ierr = > PetscMalloc1(oldLength, );CHKERRQ(ierr);" > Also, vsnprintf returns "the number of characters that would have been > written if n had been sufficiently large". I don't know why you void'ed it. > We can not make the 8K chars a requirement since users don't know how many > chars they want to print upfront. > Anyway, crash is better than silent errors. > >Hence it lives in this limbo. I don't even know a way to add a good error > checking that detects if the buffer is long enough. All in all it is bad ugly > code, any suggestions on improvements would be appreciated. > >Barry > > > > > 2018-04-12 2:16 GMT+02:00 Rongliang Chen : > > Thanks Barry. I found petsc-3.6 and older versions did not have this > > restriction. > > > > Best, > > Rongliang > > > > > > On 04/12/2018 07:22 AM, Smith, Barry F. wrote: > >Yes, PetscPrintf() and related functions have a maximum string length of > > about 8000 characters. > > > > Barry > > > > > > On Apr 11, 2018, at 6:17 PM, Rongliang Chen > > wrote: > > > > Dear All, > > > > > > When I tried to print a long string using PetscPrintf() I found that it > > truncated the string. Attached is a simple example for this (run with > > single processor). I used PetscPrintf() and printf() to print the same > > string and the printf() seems OK. I
Re: [petsc-dev] [petsc-users] PetscPrintf
How about the logic of this analysis? 1. We are trying to use the same functions (in particular, PetscVFPrintf) for two purposes: a. printing error messages (don't want to malloc) b. use by public API printing functions (don't want length restrictions) 2. Right now, PetscVFPrintf works fine for a but not for b. We could make it work for b and not for a by malloc'ing a longer string. 3. Printing from error handlers happens through PetscErrorPrintf (default : http://www.mcs.anl.gov/petsc/petsc-dev/src/sys/error/errtrace.c.html#PetscErrorPrintfDefault ), so if there's a special requirement for printing error messages, we can impose it here. A solution could then be something which skips the malloc only when printing an error, e.g. 1. Add an argument to PetscVFPrintf (say "PetscBool noMalloc") [1] 2. API (PetscPrintf(), etc.) functions use noMalloc = PETSC_FALSE 3. error functions (PetscErrorPrintf() ) functions use noMalloc = PETSC_TRUE [1] And probably do the same thing in PetscVSNPrintf since, as Dr. Zhang pointed out, this could also call malloc while handling an error, if the string was long enough 2018-04-13 15:59 GMT+02:00 Junchao Zhang: > On Thu, Apr 12, 2018 at 9:48 AM, Smith, Barry F. > wrote: > >> >> >> > On Apr 12, 2018, at 3:59 AM, Patrick Sanan >> wrote: >> > >> > I also happened to stumble across this yesterday. Is the length >> restriction for the default printer (l assume from the array of 8*1024 >> chars in PetscVFPrintfDefault() ) intended behavior to be documented, or a >> bug to be fixed? >> >> You could call it either. My problems are >> >> 1) that given a format string I don't know in advance how much work space >> is needed so cannot accurately malloc, for sure, enough space >> >> 2) since this can be called in an error handler I really don't want it >> calling malloc(). >> > PetscVSNPrintf does still contain a malloc "122 ierr = > PetscMalloc1(oldLength, );CHKERRQ(ierr);" > Also, vsnprintf returns "the number of characters that would have been > written if n had been sufficiently large". I don't know why you void'ed > it. > We can not make the 8K chars a requirement since users don't know how many > chars they want to print upfront. > Anyway, crash is better than silent errors. > >> >>Hence it lives in this limbo. I don't even know a way to add a good >> error checking that detects if the buffer is long enough. All in all it is >> bad ugly code, any suggestions on improvements would be appreciated. >> >>Barry >> >> > >> > 2018-04-12 2:16 GMT+02:00 Rongliang Chen : >> > Thanks Barry. I found petsc-3.6 and older versions did not have this >> restriction. >> > >> > Best, >> > Rongliang >> > >> > >> > On 04/12/2018 07:22 AM, Smith, Barry F. wrote: >> >Yes, PetscPrintf() and related functions have a maximum string >> length of about 8000 characters. >> > >> > Barry >> > >> > >> > On Apr 11, 2018, at 6:17 PM, Rongliang Chen >> wrote: >> > >> > Dear All, >> > >> > >> > When I tried to print a long string using PetscPrintf() I found that it >> truncated the string. Attached is a simple example for this (run with >> single processor). I used PetscPrintf() and printf() to print the same >> string and the printf() seems OK. I am using petsc-3.8.4. >> > >> > >> > Best, >> > >> > Rongliang >> > >> > >> > >> > >> > >> >> >
Re: [petsc-dev] [petsc-users] PetscPrintf
On Thu, Apr 12, 2018 at 9:48 AM, Smith, Barry F.wrote: > > > > On Apr 12, 2018, at 3:59 AM, Patrick Sanan > wrote: > > > > I also happened to stumble across this yesterday. Is the length > restriction for the default printer (l assume from the array of 8*1024 > chars in PetscVFPrintfDefault() ) intended behavior to be documented, or a > bug to be fixed? > > You could call it either. My problems are > > 1) that given a format string I don't know in advance how much work space > is needed so cannot accurately malloc, for sure, enough space > > 2) since this can be called in an error handler I really don't want it > calling malloc(). > PetscVSNPrintf does still contain a malloc "122 ierr = PetscMalloc1(oldLength, );CHKERRQ(ierr);" Also, vsnprintf returns "the number of characters that would have been written if n had been sufficiently large". I don't know why you void'ed it. We can not make the 8K chars a requirement since users don't know how many chars they want to print upfront. Anyway, crash is better than silent errors. > >Hence it lives in this limbo. I don't even know a way to add a good > error checking that detects if the buffer is long enough. All in all it is > bad ugly code, any suggestions on improvements would be appreciated. > >Barry > > > > > 2018-04-12 2:16 GMT+02:00 Rongliang Chen : > > Thanks Barry. I found petsc-3.6 and older versions did not have this > restriction. > > > > Best, > > Rongliang > > > > > > On 04/12/2018 07:22 AM, Smith, Barry F. wrote: > >Yes, PetscPrintf() and related functions have a maximum string length > of about 8000 characters. > > > > Barry > > > > > > On Apr 11, 2018, at 6:17 PM, Rongliang Chen > wrote: > > > > Dear All, > > > > > > When I tried to print a long string using PetscPrintf() I found that it > truncated the string. Attached is a simple example for this (run with > single processor). I used PetscPrintf() and printf() to print the same > string and the printf() seems OK. I am using petsc-3.8.4. > > > > > > Best, > > > > Rongliang > > > > > > > > > > > >
Re: [petsc-dev] [petsc-users] PetscPrintf
"Smith, Barry F."writes: >> Is the postprocessing of output in PetscVSNPrintf really necessary? > > PetscVSNPrintf does two things > > 1) handles %D to have a common way of printing PetscInt regardless of if they > are 32 or 64 bit integers This is just modifying the format string, so not an issue. > 2) insures that all floating point numbers printed have a . in them so > Petscdiff can accurately determine what is a > floating point number in output (standard C formats don't put the . in > floating point numbers like 1. that don't need them. I could not find > any decent numerical diff for PETSc output to use in place of our cumbersome > home-brew thing. I don't think it's possible to write a printf format string that behaves like %g except always includes the decimal point. We could insist on using %e or %.1f style format strings always -- many uses of %g are gratuitous and not actually better for the user. But it would be disruptive and occasionally weird. One alternative would be to escape everything but the %g, count and extend the storage for the %g, snprintf that part, then pass along the rest. But that won't reduced the dirtiness. > If there are alternative ways of handling 1) and 2) then PetscVSNPrintf() and > its cumbersomeness are not needed. > > Barry > > Note that PetscVSNPrintf() does not handle printing of PetscReal, we have to > manually put a caste of (double) in the calling sequence (for __float128). I > tried to add this support with %G but gave up because it was a rat hole. > > > > > >> On Apr 12, 2018, at 8:22 PM, Jed Brown wrote: >> >> "Smith, Barry F." writes: >> On Apr 12, 2018, at 3:59 AM, Patrick Sanan wrote: I also happened to stumble across this yesterday. Is the length restriction for the default printer (l assume from the array of 8*1024 chars in PetscVFPrintfDefault() ) intended behavior to be documented, or a bug to be fixed? >>> >>> You could call it either. My problems are >>> >>> 1) that given a format string I don't know in advance how much work space >>> is needed so cannot accurately malloc, for sure, enough space >>> >>> 2) since this can be called in an error handler I really don't want it >>> calling malloc(). >>> >>> Hence it lives in this limbo. I don't even know a way to add a good error >>> checking that detects if the buffer is long enough. All in all it is bad >>> ugly code, any suggestions on improvements would be appreciated. >> >> Is the postprocessing of output in PetscVSNPrintf really necessary? >> Without it, you would call vfprintf instead of vsnprintf followed by >> fprintf("%s", string) [1]. >> >> >> [1] fputs would be preferable. >> >>> >>> Barry >>> 2018-04-12 2:16 GMT+02:00 Rongliang Chen : Thanks Barry. I found petsc-3.6 and older versions did not have this restriction. Best, Rongliang On 04/12/2018 07:22 AM, Smith, Barry F. wrote: Yes, PetscPrintf() and related functions have a maximum string length of about 8000 characters. Barry On Apr 11, 2018, at 6:17 PM, Rongliang Chen wrote: Dear All, When I tried to print a long string using PetscPrintf() I found that it truncated the string. Attached is a simple example for this (run with single processor). I used PetscPrintf() and printf() to print the same string and the printf() seems OK. I am using petsc-3.8.4. Best, Rongliang
Re: [petsc-dev] [petsc-users] PetscPrintf
> Is the postprocessing of output in PetscVSNPrintf really necessary? PetscVSNPrintf does two things 1) handles %D to have a common way of printing PetscInt regardless of if they are 32 or 64 bit integers 2) insures that all floating point numbers printed have a . in them so Petscdiff can accurately determine what is a floating point number in output (standard C formats don't put the . in floating point numbers like 1. that don't need them. I could not find any decent numerical diff for PETSc output to use in place of our cumbersome home-brew thing. If there are alternative ways of handling 1) and 2) then PetscVSNPrintf() and its cumbersomeness are not needed. Barry Note that PetscVSNPrintf() does not handle printing of PetscReal, we have to manually put a caste of (double) in the calling sequence (for __float128). I tried to add this support with %G but gave up because it was a rat hole. > On Apr 12, 2018, at 8:22 PM, Jed Brownwrote: > > "Smith, Barry F." writes: > >>> On Apr 12, 2018, at 3:59 AM, Patrick Sanan wrote: >>> >>> I also happened to stumble across this yesterday. Is the length restriction >>> for the default printer (l assume from the array of 8*1024 chars in >>> PetscVFPrintfDefault() ) intended behavior to be documented, or a bug to be >>> fixed? >> >> You could call it either. My problems are >> >> 1) that given a format string I don't know in advance how much work space is >> needed so cannot accurately malloc, for sure, enough space >> >> 2) since this can be called in an error handler I really don't want it >> calling malloc(). >> >> Hence it lives in this limbo. I don't even know a way to add a good error >> checking that detects if the buffer is long enough. All in all it is bad >> ugly code, any suggestions on improvements would be appreciated. > > Is the postprocessing of output in PetscVSNPrintf really necessary? > Without it, you would call vfprintf instead of vsnprintf followed by > fprintf("%s", string) [1]. > > > [1] fputs would be preferable. > >> >> Barry >> >>> >>> 2018-04-12 2:16 GMT+02:00 Rongliang Chen : >>> Thanks Barry. I found petsc-3.6 and older versions did not have this >>> restriction. >>> >>> Best, >>> Rongliang >>> >>> >>> On 04/12/2018 07:22 AM, Smith, Barry F. wrote: >>> Yes, PetscPrintf() and related functions have a maximum string length of >>> about 8000 characters. >>> >>>Barry >>> >>> >>> On Apr 11, 2018, at 6:17 PM, Rongliang Chen >>> wrote: >>> >>> Dear All, >>> >>> >>> When I tried to print a long string using PetscPrintf() I found that it >>> truncated the string. Attached is a simple example for this (run with >>> single processor). I used PetscPrintf() and printf() to print the same >>> string and the printf() seems OK. I am using petsc-3.8.4. >>> >>> >>> Best, >>> >>> Rongliang >>> >>>
Re: [petsc-dev] [petsc-users] PetscPrintf
"Smith, Barry F."writes: >> On Apr 12, 2018, at 3:59 AM, Patrick Sanan wrote: >> >> I also happened to stumble across this yesterday. Is the length restriction >> for the default printer (l assume from the array of 8*1024 chars in >> PetscVFPrintfDefault() ) intended behavior to be documented, or a bug to be >> fixed? > > You could call it either. My problems are > > 1) that given a format string I don't know in advance how much work space is > needed so cannot accurately malloc, for sure, enough space > > 2) since this can be called in an error handler I really don't want it > calling malloc(). > >Hence it lives in this limbo. I don't even know a way to add a good error > checking that detects if the buffer is long enough. All in all it is bad ugly > code, any suggestions on improvements would be appreciated. Is the postprocessing of output in PetscVSNPrintf really necessary? Without it, you would call vfprintf instead of vsnprintf followed by fprintf("%s", string) [1]. [1] fputs would be preferable. > >Barry > >> >> 2018-04-12 2:16 GMT+02:00 Rongliang Chen : >> Thanks Barry. I found petsc-3.6 and older versions did not have this >> restriction. >> >> Best, >> Rongliang >> >> >> On 04/12/2018 07:22 AM, Smith, Barry F. wrote: >>Yes, PetscPrintf() and related functions have a maximum string length of >> about 8000 characters. >> >> Barry >> >> >> On Apr 11, 2018, at 6:17 PM, Rongliang Chen wrote: >> >> Dear All, >> >> >> When I tried to print a long string using PetscPrintf() I found that it >> truncated the string. Attached is a simple example for this (run with single >> processor). I used PetscPrintf() and printf() to print the same string and >> the printf() seems OK. I am using petsc-3.8.4. >> >> >> Best, >> >> Rongliang >> >> >> >> >>
Re: [petsc-dev] [petsc-users] PetscPrintf
> On Apr 12, 2018, at 3:59 AM, Patrick Sananwrote: > > I also happened to stumble across this yesterday. Is the length restriction > for the default printer (l assume from the array of 8*1024 chars in > PetscVFPrintfDefault() ) intended behavior to be documented, or a bug to be > fixed? You could call it either. My problems are 1) that given a format string I don't know in advance how much work space is needed so cannot accurately malloc, for sure, enough space 2) since this can be called in an error handler I really don't want it calling malloc(). Hence it lives in this limbo. I don't even know a way to add a good error checking that detects if the buffer is long enough. All in all it is bad ugly code, any suggestions on improvements would be appreciated. Barry > > 2018-04-12 2:16 GMT+02:00 Rongliang Chen : > Thanks Barry. I found petsc-3.6 and older versions did not have this > restriction. > > Best, > Rongliang > > > On 04/12/2018 07:22 AM, Smith, Barry F. wrote: >Yes, PetscPrintf() and related functions have a maximum string length of > about 8000 characters. > > Barry > > > On Apr 11, 2018, at 6:17 PM, Rongliang Chen wrote: > > Dear All, > > > When I tried to print a long string using PetscPrintf() I found that it > truncated the string. Attached is a simple example for this (run with single > processor). I used PetscPrintf() and printf() to print the same string and > the printf() seems OK. I am using petsc-3.8.4. > > > Best, > > Rongliang > > > > >
Re: [petsc-dev] [petsc-users] PetscPrintf
I also happened to stumble across this yesterday. Is the length restriction for the default printer (l assume from the array of 8*1024 chars in PetscVFPrintfDefault() ) intended behavior to be documented, or a bug to be fixed? 2018-04-12 2:16 GMT+02:00 Rongliang Chen: > Thanks Barry. I found petsc-3.6 and older versions did not have this > restriction. > > Best, > Rongliang > > > On 04/12/2018 07:22 AM, Smith, Barry F. wrote: > >>Yes, PetscPrintf() and related functions have a maximum string length >> of about 8000 characters. >> >> Barry >> >> >> On Apr 11, 2018, at 6:17 PM, Rongliang Chen >>> wrote: >>> >>> Dear All, >>> >>> >>> When I tried to print a long string using PetscPrintf() I found that it >>> truncated the string. Attached is a simple example for this (run with >>> single processor). I used PetscPrintf() and printf() to print the same >>> string and the printf() seems OK. I am using petsc-3.8.4. >>> >>> >>> Best, >>> >>> Rongliang >>> >>> >>> >> > >