Hi Ian,
> syslogparser.go
> =============
>
> func (obj SyslogParser) LoadPatternDB(opts Syslog, workerId int) {
log.Info("Loading pattern db with path:", opts.Patterndb)
> patterndbpath := C.CString(opts.Patterndb) . <<<<<<<<<<<<<<< . STEP 1
> defer C.free(unsafe.Pointer(patterndbpath)) . <<<<<<<<<<<<<<<< STEP 2
> InitStruct := (*C.Accumulatedparams)(C.calloc(1,
C.sizeof_struct_Accumulatedparams))
> InitStruct.callback = (C.key_value_cb)(C.callback)
> InitStruct.data = C.int(workerId)
> C.load_pattern_db(patterndbpath,
(*C.Accumulatedparams)(unsafe.Pointer(InitStruct)), C.int(workerId))
> }
>
> Since I am running this program in multiple go routines, when we do above
STEP 1 :
>
> 1)Will the C memory allocate different addresses for 'patterndbpath' in
both the routines ...Am I right ?
Yes.
> 2)You said 'It may be calling free twice on the same pointer', but I am
assigning a different pointer in STEP 1 everytime for each routine...Am I
right ?
Yes, you will see a different pointer each time you call C.calloc, and
in the code that I see that pointer is only freed once, but I have no
idea what the rest of your code is doing.
I am calling free only once and that is for 'patterndbpath' in this
particular method.In the entire codebase I am NOT calling free for
InitStruct (Memory allocated for the struct)
I am pointing out this particular method 'LoadPatternDB' of Go code base
because I had put a 'Info' log in this method at the starting and that got
printed before the corruption error came up.
And my testing failed at the third iteration:
1)WorkerID0 completed the work
2)WorkerID1 completed the work
3)WorkerID1 failed in this particular method with 'fatal error: bin/main':
double free or corruption (fasttop)'
Following is one of the methods called before this method :
func (obj SyslogParser) InitializeEngine() {
patterndb := C.CString(obj.Patterndb)
defer C.free(unsafe.Pointer(patterndb))
modulepath := C.CString(obj.Modulepath)
defer C.free(unsafe.Pointer(modulepath))
C.initialize_engine(patterndb, modulepath)
}
Not deallocating the memory for the struct can be the issue for 'fatal
error: bin/main': double free or corruption (fasttop)' ?
> 3)Is it possible that one Go routine has already freed up the address
that the other Go routine is trying to free and that is causing this
'fatal error: bin/main': double free or corruption (fasttop)'
Yes, that is possible. It won't happen with the code I see above but
I have no idea what the rest of your code is doing.
How can it not happen with this code ?I did not understand this point.
Can you please clarify this point ?
> Is this because of the race condition between two routines that this
error is seen ?
It is possible.
> 4)If you show us a small, complete, self-contained example, that does not
call any other code, then we may be able to see the problem.
> >> As you have mentioned problem is happening when we call 'C.free'.Do
you mean by not calling internal syslog-ng methods and just calling a
normal C program with multiple go routines ?
> Please correct me here if I am wrong
I mean any small, self-contained example where we can see all the code.
>>>I will try to send a very small, basic code but not sure how to
reproduce the issue with that.
You seem to be looking at a memory corruption or double free problem.
These are normal, ordinary, problems when writing C code. We can't
debug your problem for you.
On Fri, Jul 12, 2019 at 7:54 PM Ian Lance Taylor <i...@golang.org> wrote:
> On Thu, Jul 11, 2019 at 11:41 PM Nitish Saboo <nitish.sabo...@gmail.com>
> wrote:
> >
> > syslogparser.go
> > =============
> >
> > func (obj SyslogParser) LoadPatternDB(opts Syslog, workerId int) {
> > patterndbpath := C.CString(opts.Patterndb) . <<<<<<<<<<<<<<< . STEP 1
> > defer C.free(unsafe.Pointer(patterndbpath)) . <<<<<<<<<<<<<<<< STEP 2
> > InitStruct := (*C.Accumulatedparams)(C.calloc(1,
> C.sizeof_struct_Accumulatedparams))
> > InitStruct.callback = (C.key_value_cb)(C.callback)
> > InitStruct.data = C.int(workerId)
> > C.load_pattern_db(patterndbpath,
> (*C.Accumulatedparams)(unsafe.Pointer(InitStruct)), C.int(workerId))
> > }
> >
> > Since I am running this program in multiple go routines, when we do
> above STEP 1 :
> >
> > 1)Will the C memory allocate different addresses for 'patterndbpath' in
> both the routines ...Am I right ?
>
> Yes.
>
> > 2)You said 'It may be calling free twice on the same pointer', but I am
> assigning a different pointer in STEP 1 everytime for each routine...Am I
> right ?
>
> Yes, you will see a different pointer each time you call C.calloc, and
> in the code that I see that pointer is only freed once, but I have no
> idea what the rest of your code is doing.
>
> > 3)Is it possible that one Go routine has already freed up the address
> that the other Go routine is trying to free and that is causing this
> 'fatal error: bin/main': double free or corruption (fasttop)'
>
> Yes, that is possible. It won't happen with the code I see above but
> I have no idea what the rest of your code is doing.
>
> > Is this because of the race condition between two routines that this
> error is seen ?
>
> It is possible.
>
> > 4)If you show us a small, complete, self-contained example, that does
> not call any other code, then we may be able to see the problem.
> > >> As you have mentioned problem is happening when we call 'C.free'.Do
> you mean by not calling internal syslog-ng methods and just calling a
> normal C program with multiple go routines ?
> > Please correct me here if I am wrong
>
> I mean any small, self-contained example where we can see all the code.
>
> You seem to be looking at a memory corruption or double free problem.
> These are normal, ordinary, problems when writing C code. We can't
> debug your problem for you.
>
> Ian
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/CALjMrq6eM1_ykJYmNVYqnFnjxqRvRCJF8Bqgh6k2N7kBV%2B_3Dw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
