Dear all, trying to make samba work and getting rid of my error message [2019/04/01 09:51:41.112485, 0] ../../lib/tdb_wrap/tdb_wrap.c:64(tdb_wrap_log) tdb(/var/samba/lock/gencache.tdb): tdb_open_ex: tdb_mutex_init failed for /var/samba/lock/gencache.tdb: Device busy
I followed the info of Youzhong. It definitely works. So thankx. BUT: The OS (OpenIndiana; state of April,19th) is patched against the illumos issue 9959 So there might be some mechanism (maybe MUTEX_MAGIC) that avoids the initialization of the mutex before init. I am really not deep in all that pthread stuff… Is there the possiblity that there is a bug in the code base of issue 9959? Best regards Rouven > Am 18.04.2019 um 19:20 schrieb Robert Mustacchi <[email protected]>: > > On 4/18/19 10:12 , Youzhong Yang wrote: >> Little bit confused. The page https://www.openindiana.org/download/ shows >> latest ISO is dated 2018.10, but 9959 fix was submitted on 11/26/2018, I >> guess Hipster doesn't have it. > > The ISO is just the install image. Generally, after using that to > bootstrap, then folks will update to latest bits via IPS. As a result, > the ISO isn't updated (and generally doesn't have to be) quite as often. > > Robert > >> On Thu, Apr 18, 2019 at 12:38 PM Rouven WEILER <[email protected]> >> wrote: >> >>> On my system the issue is solved and the sources are patched accordingly. >>> Thus, the issue seems to be somewhere else. >>> >>> Am 18.04.2019 um 18:27 schrieb Rouven WEILER <[email protected]>: >>> >>> concerning openindiana I am using the most recent one. I right now asked >>> if the issue is solved in the latest version. >>> >>> another interesting thing is that older samba versions do not throw EBUSY, >>> although on the same underlying system. >>> >>> >>> >>> Am 18.04.2019 um 18:07 schrieb Youzhong Yang <[email protected]>: >>> >>> That is interesting. I am curious about how pthread_mutexattr_init ends up >>> returning EBUSY. What verison of Openindiana are you running? is there a >>> link pointing to the fix that it has for 9959? >>> >>> On Thu, Apr 18, 2019 at 11:53 AM Rouven WEILER <[email protected]> >>> wrote: >>> >>>> I understand and see that it works, but as written below >>>> pthread_mutexattr_init throws EBUSY, not pthread_mutex_init. >>>> >>>> Also Openindiana seems to include the patch for issue 9959 mentioned. I >>>> will clarify that to make sure. >>>> >>>> I just want to make sure that the patch you suggest aims for the issue >>>> and is not a workaround. >>>> >>>> >>>> Am 18.04.2019 um 16:09 schrieb Youzhong Yang <[email protected]>: >>>> >>>> I reproduced the issue on a smartos version which does not have the fix >>>> for https://www.illumos.org/issues/9959. >>>> >>>> The pthread_mutex_init() at line 590 returns EBUSY: >>>> ret = pthread_mutex_init(chain, &ma); >>>> >>>> https://git.samba.org/?p=samba.git;a=blob;f=lib/tdb/common/mutex.c;h=8a122d5d6a2ec321bce25cf06062acc808f1e126;hb=HEAD#l590 >>>> >>>> >>>> Then I applied the following patch to lib/tdb/common/mutex.c: >>>> >>>> # git grep -B 1 -n pthread_mutex_init lib/tdb/common/mutex.c >>>> lib/tdb/common/mutex.c-590- bzero(chain, >>>> sizeof(pthread_mutex_t)); >>>> lib/tdb/common/mutex.c:591: ret = pthread_mutex_init(chain, >>>> &ma); >>>> -- >>>> lib/tdb/common/mutex.c-599- bzero(&m->allrecord_mutex, >>>> sizeof(pthread_mutex_t)); >>>> lib/tdb/common/mutex.c:600: ret = >>>> pthread_mutex_init(&m->allrecord_mutex, &ma); >>>> -- >>>> lib/tdb/common/mutex.c-693- bzero(&m, sizeof(pthread_mutex_t)); >>>> lib/tdb/common/mutex.c:694: ret = pthread_mutex_init(&m, &ma); >>>> -- >>>> lib/tdb/common/mutex.c-891- bzero(ptr, sizeof(pthread_mutex_t)); >>>> lib/tdb/common/mutex.c:892: ret = pthread_mutex_init(ptr, &ma); >>>> >>>> It works like a charm. Fair enough? >>>> >>>> So again I recommend either of the following two: >>>> - apply 9959 fix to your OS image, or use a newer version which has 9959 >>>> fix >>>> - bzero pthread_mutex_t struct before calling pthread_mutex_init(). >>>> >>>> In my environment I applied bzero patch to all pthread_mutex_init() calls >>>> in the entire samba code base, you know, who knows what is going to happen. >>>> >>>> >>>> >>>> On Thu, Apr 18, 2019 at 4:29 AM Rouven WEILER <[email protected]> >>>> wrote: >>>> >>>>> That sounds reasonable! >>>>> Could you help me to dtrace that in a feasible way? Up to now I am not >>>>> used to dtrace. Except knowing what is and that its powerful. >>>>> >>>>> Am 18.04.2019 um 09:29 schrieb Jason King <[email protected]>: >>>>> >>>>> It appears that the robust mutexes are mmapped from the .tdb file. >>>>> EBUSY means that the mutex has already been intialized. It also appears >>>>> that if the TDB_CLEAR_IF_FIRST flag is _not_ set when opening the tdb, it >>>>> will try to initialize the mutexes. I’m wondering if the tdb file gets >>>>> created, the memory image of the initialized mutexes is getting saved, >>>>> then >>>>> when it’s opened again, if that flag is not set, it tries to re-initialize >>>>> the mutexes and fails (because they are already constructed) legitimately >>>>> returning EBUSY. If the samba code is in fact trying to reinitialize an >>>>> already initialized mutex (dtrace might be able to help determine this — >>>>> at >>>>> this point it’s merely a supposition), that would explain the error. >>>>> >>>>> >>>>> >>>>> >>>>> From: Rouven WEILER <[email protected]> <[email protected]> >>>>> Reply: illumos-developer <[email protected]> >>>>> <[email protected]> >>>>> Date: April 18, 2019 at 12:48:48 AM >>>>> To: Youzhong Yang <[email protected]> <[email protected]> >>>>> Cc: Robert Mustacchi <[email protected]> <[email protected]>, Toomas Soome via >>>>> illumos-developer <[email protected]> >>>>> <[email protected]> >>>>> Subject: [developer] Aw: Re: [Samba] gencache.tdb: device busy >>>>> >>>>> You are totally right. That's the same I got from the man pages. >>>>> >>>>> The responsible code lines of samba also seems to bear no issue and also >>>>> this issue arises on samba version 4.10.x not on earlier ones. >>>>> In addition, there seems to be only a problem with >>>>> illumos/openindiana... >>>>> That's why I do not know how to get that fixed. Up to now I hadn't time >>>>> to dtrace. >>>>> >>>>> Only thing I got so far: When the mutexattr is already set the return >>>>> value will result on undefined behaviour. >>>>> But why only after version 4.10.x on openindiana/illumos? >>>>> >>>>> Just for completion here the codelines of samba: >>>>> https://git.samba.org/?p=samba.git;a=blob;f=lib/tdb/common/mutex.c;h=8a122d5d6a2ec321bce25cf06062acc808f1e126;hb=HEAD#l570 >>>>> >>>>> >>>>> >>>>> Gesendet: Mittwoch, 17. April 2019 um 19:19 Uhr >>>>> Von: "Youzhong Yang" <[email protected]> >>>>> An: "Rouven WEILER" <[email protected]> >>>>> Cc: "Robert Mustacchi" <[email protected]>, "Toomas Soome via >>>>> illumos-developer" <[email protected]> >>>>> Betreff: Re: [Samba] gencache.tdb: device busy >>>>> >>>>> Unless I am totally wrong, pthread_mutexattr_init() shouldn't return >>>>> EBUSY: >>>>> >>>>> http://src.illumos.org/source/xref/illumos-gate/usr/src/lib/libc/port/threads/pthr_mutex.c#37 >>>>> >>>>> >>>>> On Tue, Apr 16, 2019 at 2:45 PM Rouven WEILER <[email protected][ >>>>> mailto:[email protected] <[email protected]>]> wrote:Hi Robert >>>>> & Youzhong, >>>>> >>>>> I got a response from the Samba-Devs. >>>>> Would the issue mentioned by Youzhong also hot pthread_mutexattr_t? >>>>> >>>>> Concerning dtrace I hadn’t the time yet but I will check for that. >>>>> Is there maybe a way to trace all forks of smbd? Usually the issue >>>>> arises after the fork… >>>>> >>>>> FIND BELOW THE STATEMENT FROM samba.org[http://samba.org]: >>>>> >>>>> This doesn't look exactly the same. In your >>>>> log below you say: >>>>> >>>>> ret = pthread_mutexattr_init(&ma); in line 570 >>>>> I got errorno 16 --> device busy. >>>>> >>>>> But the bug report is about needing the value >>>>> passed to pthread_mutex_init() needing to be >>>>> zero, not the value passed to pthread_mutexattr_init(). >>>>> >>>>> Can you just try adding '= {0}' to all the >>>>> places where we declare: >>>>> >>>>> pthread_mutexattr_t ma >>>>> >>>>> (i.e. make it say) >>>>> >>>>> pthread_mutexattr_t ma = {0}; >>>>> >>>>> and see if it makes your problem go away ? >>>>> >>>>> END OF STATEMENT >>>>> >>>>> >>>>>> Anfang der weitergeleiteten Nachricht: >>>>>> >>>>>> Von: Jeremy Allison via samba <[email protected][ >>>>> mailto:[email protected] <[email protected]>]> >>>>>> Betreff: Aw: [Samba] gencache.tdb: device busy >>>>>> Datum: 16. April 2019 um 02:07:05 MESZ >>>>>> An: Rouven WEILER <[email protected][mailto:[email protected] >>>>> <[email protected]>]> >>>>>> Kopie: [email protected][mailto:[email protected] >>>>> <[email protected]>] >>>>>> Antwort an: Jeremy Allison <[email protected][mailto:[email protected] >>>>> <[email protected]>]> >>>>>> >>>>>> On Fri, Apr 12, 2019 at 09:46:04AM +0200, Rouven WEILER via samba >>>>> wrote: >>>>>>> >>>>>>> >>>>>>> Hi Jeremy, >>>>>>> >>>>>>> I got some info on that topic from the illumos devs: >>>>>>> >>>>>>> >>>>>>>> It's a sporadic issue, you're lucky enough to not encounter it on >>>>> 4.9.5. >>>>>>>> >>>>>>>> I confirmed in 4.10.2, it happens: >>>>>>>> >>>>>>>> winbindd.log: tdb(/tmw-nas-3p/samba/var/lock/gencache.tdb): >>>>> tdb_open_ex: tdb_mutex_init failed for >>>>> /tmw-nas-3p/samba/var/lock/gencache.tdb: Device busy >>>>>>>> >>>>>>>> So either apply OS fix, or apply a software fix (zeroing >>>>> pthread_mutex_t before pthread_mutex_init()). >>>>>>> >>>>>>> This all should be based on the following issue: >>>>>>> >>>>>>> >>>>> https://www.illumos.org/issues/9959[https://www.illumos.org/issues/9959] >>>>> >>>>>>> >>>>>>> As far as I understand the issue making an software fix in samba for >>>>> the OS seems more reasonable since the problem can occur sporadic and thus >>>>> again... >>>>>>> Could you tell me how to help to fix that bug? >>>>>> >>>>>> This doesn't look exactly the same. In your >>>>>> log below you say: >>>>>> >>>>>> ret = pthread_mutexattr_init(&ma); in line 570 >>>>>> I got errorno 16 --> device busy. >>>>>> >>>>>> But the bug report is about needing the value >>>>>> passed to pthread_mutex_init() needing to be >>>>>> zero, not the value passed to pthread_mutexattr_init(). >>>>>> >>>>>> Can you just try adding '= {0}' to all the >>>>>> places where we declare: >>>>>> >>>>>> pthread_mutexattr_t ma >>>>>> >>>>>> (i.e. make it say) >>>>>> >>>>>> pthread_mutexattr_t ma = {0}; >>>>>> >>>>>> and see if it makes your problem go away ? >>>>>> >>>>>> >>>>>>> Gesendet: Dienstag, 02. April 2019 um 18:59 Uhr >>>>>>> Von: "Jeremy Allison" <[email protected][mailto:[email protected] >>>>> <[email protected]>]> >>>>>>> An: "Rouven WEILER" <[email protected][ >>>>> mailto:[email protected] <[email protected]>]> >>>>>>> Cc: [email protected][mailto:[email protected] >>>>> <[email protected]>] >>>>>>> Betreff: Re: Re: [Samba] gencache.tdb: device busy >>>>>>> On Tue, Apr 02, 2019 at 01:24:19PM +0200, Rouven WEILER wrote: >>>>>>>> >>>>>>>> >>>>>>>> I added LOG statements to mutex.c printing the strerror from the >>>>> return values. >>>>>>>> >>>>>>>> From >>>>>>>> >>>>>>>> ret = pthread_mutexattr_init(&ma); in line 570 >>>>>>>> >>>>>>>> I got errorno 16 --> device busy. >>>>>>>> >>>>>>>> But due the manual of pthread_mutex_attr_init() there should only be >>>>> ENOMEM and EINVAL... >>>>>>>> >>>>>>>> Doing gdb smbd -> run >>>>>>>> Doe not reveal anything, but I never used gdb for debugging >>>>> anything. I just gave it a shot... >>>>>>> >>>>>>> Sorry, looks a little like the libc/pthread_mutex support >>>>>>> on Solaris is broken. You might want to raise this on a >>>>>>> Solaris/Illumos forum. >>>>>>> >>>>>>> -- >>>>>>> To unsubscribe from this list go to the following URL and read the >>>>>>> instructions: >>>>> https://lists.samba.org/mailman/options/samba[https://lists.samba.org/mailman/options/samba] >>>>> >>>>>> >>>>>> -- >>>>>> To unsubscribe from this list go to the following URL and read the >>>>>> instructions: >>>>> https://lists.samba.org/mailman/options/samba[https://lists.samba.org/mailman/options/samba] >>>>> >>>>> >>>>> ------------------------------------------ >>>>> illumos: illumos-developer >>>>> Permalink: >>>>> https://illumos.topicbox.com/groups/developer/T38b7c9b3f92fa699-M99a2afeb19714648c051ab5f >>>>> Delivery options: >>>>> https://illumos.topicbox.com/groups/developer/subscription >>>>> >>>>> *illumos <https://illumos.topicbox.com/latest>* / illumos-developer / >>> see discussions <https://illumos.topicbox.com/groups/developer> + >>> participants <https://illumos.topicbox.com/groups/developer/members> + >>> delivery >>> options <https://illumos.topicbox.com/groups/developer/subscription> >>> Permalink >>> <https://illumos.topicbox.com/groups/developer/T38b7c9b3f92fa699-M2372380a78d4d5078f031f0c> >>> >>> > > ------------------------------------------ > illumos: illumos-developer > Permalink: > https://illumos.topicbox.com/groups/developer/T38b7c9b3f92fa699-M1048b12f29cda459a4de0893 > Delivery options: https://illumos.topicbox.com/groups/developer/subscription
_______________________________________________ oi-dev mailing list [email protected] https://openindiana.org/mailman/listinfo/oi-dev
