Any chance to get to know what happens?
> Am 21.04.2019 um 14:12 schrieb Richard Lowe <richl...@richlowe.net>: > > It's possible there's a bug. It's also possible you're getting really > unlucky and the uninitialized mutex looks valid because the magic is valid. > >> On Sun, Apr 21, 2019, 06:10 Rouven WEILER <rouven_wei...@gmx.net> wrote: >> Dear all, >> >> trying to make samba work and getting rid of my error message >> [2019/04/01 09:51:41.112485, 0] >> ../../lib/tdb_wrap/tdb_wrap.c:64(tdb_wrap_log) >> tdb(/var/samba/lock/gencache.tdb): tdb_open_ex: tdb_mutex_init failed for >> /var/samba/lock/gencache.tdb: Device busy >> >> I followed the info of Youzhong. It definitely works. So thankx. >> BUT: The OS (OpenIndiana; state of April,19th) is patched against the >> illumos issue 9959 >> >> So there might be some mechanism (maybe MUTEX_MAGIC) that avoids the >> initialization of the mutex before init. >> I am really not deep in all that pthread stuff… >> >> Is there the possiblity that there is a bug in the code base of issue 9959? >> >> >> Best regards >> >> Rouven >> >> >>> Am 18.04.2019 um 19:20 schrieb Robert Mustacchi <r...@joyent.com>: >>> >>> On 4/18/19 10:12 , Youzhong Yang wrote: >>>> Little bit confused. The page https://www.openindiana.org/download/ shows >>>> latest ISO is dated 2018.10, but 9959 fix was submitted on 11/26/2018, I >>>> guess Hipster doesn't have it. >>> >>> The ISO is just the install image. Generally, after using that to >>> bootstrap, then folks will update to latest bits via IPS. As a result, >>> the ISO isn't updated (and generally doesn't have to be) quite as often. >>> >>> Robert >>> >>>> On Thu, Apr 18, 2019 at 12:38 PM Rouven WEILER <rouven_wei...@gmx.net> >>>> wrote: >>>> >>>>> On my system the issue is solved and the sources are patched accordingly. >>>>> Thus, the issue seems to be somewhere else. >>>>> >>>>> Am 18.04.2019 um 18:27 schrieb Rouven WEILER <rouven_wei...@gmx.net>: >>>>> >>>>> concerning openindiana I am using the most recent one. I right now asked >>>>> if the issue is solved in the latest version. >>>>> >>>>> another interesting thing is that older samba versions do not throw EBUSY, >>>>> although on the same underlying system. >>>>> >>>>> >>>>> >>>>> Am 18.04.2019 um 18:07 schrieb Youzhong Yang <youzh...@gmail.com>: >>>>> >>>>> That is interesting. I am curious about how pthread_mutexattr_init ends up >>>>> returning EBUSY. What verison of Openindiana are you running? is there a >>>>> link pointing to the fix that it has for 9959? >>>>> >>>>> On Thu, Apr 18, 2019 at 11:53 AM Rouven WEILER <rouven_wei...@gmx.net> >>>>> wrote: >>>>> >>>>>> I understand and see that it works, but as written below >>>>>> pthread_mutexattr_init throws EBUSY, not pthread_mutex_init. >>>>>> >>>>>> Also Openindiana seems to include the patch for issue 9959 mentioned. I >>>>>> will clarify that to make sure. >>>>>> >>>>>> I just want to make sure that the patch you suggest aims for the issue >>>>>> and is not a workaround. >>>>>> >>>>>> >>>>>> Am 18.04.2019 um 16:09 schrieb Youzhong Yang <youzh...@gmail.com>: >>>>>> >>>>>> I reproduced the issue on a smartos version which does not have the fix >>>>>> for https://www.illumos.org/issues/9959. >>>>>> >>>>>> The pthread_mutex_init() at line 590 returns EBUSY: >>>>>> ret = pthread_mutex_init(chain, &ma); >>>>>> >>>>>> https://git.samba.org/?p=samba.git;a=blob;f=lib/tdb/common/mutex.c;h=8a122d5d6a2ec321bce25cf06062acc808f1e126;hb=HEAD#l590 >>>>>> >>>>>> >>>>>> Then I applied the following patch to lib/tdb/common/mutex.c: >>>>>> >>>>>> # git grep -B 1 -n pthread_mutex_init lib/tdb/common/mutex.c >>>>>> lib/tdb/common/mutex.c-590- bzero(chain, >>>>>> sizeof(pthread_mutex_t)); >>>>>> lib/tdb/common/mutex.c:591: ret = pthread_mutex_init(chain, >>>>>> &ma); >>>>>> -- >>>>>> lib/tdb/common/mutex.c-599- bzero(&m->allrecord_mutex, >>>>>> sizeof(pthread_mutex_t)); >>>>>> lib/tdb/common/mutex.c:600: ret = >>>>>> pthread_mutex_init(&m->allrecord_mutex, &ma); >>>>>> -- >>>>>> lib/tdb/common/mutex.c-693- bzero(&m, sizeof(pthread_mutex_t)); >>>>>> lib/tdb/common/mutex.c:694: ret = pthread_mutex_init(&m, &ma); >>>>>> -- >>>>>> lib/tdb/common/mutex.c-891- bzero(ptr, sizeof(pthread_mutex_t)); >>>>>> lib/tdb/common/mutex.c:892: ret = pthread_mutex_init(ptr, &ma); >>>>>> >>>>>> It works like a charm. Fair enough? >>>>>> >>>>>> So again I recommend either of the following two: >>>>>> - apply 9959 fix to your OS image, or use a newer version which has 9959 >>>>>> fix >>>>>> - bzero pthread_mutex_t struct before calling pthread_mutex_init(). >>>>>> >>>>>> In my environment I applied bzero patch to all pthread_mutex_init() calls >>>>>> in the entire samba code base, you know, who knows what is going to >>>>>> happen. >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Apr 18, 2019 at 4:29 AM Rouven WEILER <rouven_wei...@gmx.net> >>>>>> wrote: >>>>>> >>>>>>> That sounds reasonable! >>>>>>> Could you help me to dtrace that in a feasible way? Up to now I am not >>>>>>> used to dtrace. Except knowing what is and that its powerful. >>>>>>> >>>>>>> Am 18.04.2019 um 09:29 schrieb Jason King <jason.brian.k...@gmail.com>: >>>>>>> >>>>>>> It appears that the robust mutexes are mmapped from the .tdb file. >>>>>>> EBUSY means that the mutex has already been intialized. It also appears >>>>>>> that if the TDB_CLEAR_IF_FIRST flag is _not_ set when opening the tdb, >>>>>>> it >>>>>>> will try to initialize the mutexes. I’m wondering if the tdb file gets >>>>>>> created, the memory image of the initialized mutexes is getting saved, >>>>>>> then >>>>>>> when it’s opened again, if that flag is not set, it tries to >>>>>>> re-initialize >>>>>>> the mutexes and fails (because they are already constructed) >>>>>>> legitimately >>>>>>> returning EBUSY. If the samba code is in fact trying to reinitialize an >>>>>>> already initialized mutex (dtrace might be able to help determine this >>>>>>> — at >>>>>>> this point it’s merely a supposition), that would explain the error. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> From: Rouven WEILER <rouven_wei...@gmx.net> <rouven_wei...@gmx.net> >>>>>>> Reply: illumos-developer <develo...@lists.illumos.org> >>>>>>> <develo...@lists.illumos.org> >>>>>>> Date: April 18, 2019 at 12:48:48 AM >>>>>>> To: Youzhong Yang <youzh...@gmail.com> <youzh...@gmail.com> >>>>>>> Cc: Robert Mustacchi <r...@joyent.com> <r...@joyent.com>, Toomas Soome >>>>>>> via >>>>>>> illumos-developer <develo...@lists.illumos.org> >>>>>>> <develo...@lists.illumos.org> >>>>>>> Subject: [developer] Aw: Re: [Samba] gencache.tdb: device busy >>>>>>> >>>>>>> You are totally right. That's the same I got from the man pages. >>>>>>> >>>>>>> The responsible code lines of samba also seems to bear no issue and also >>>>>>> this issue arises on samba version 4.10.x not on earlier ones. >>>>>>> In addition, there seems to be only a problem with >>>>>>> illumos/openindiana... >>>>>>> That's why I do not know how to get that fixed. Up to now I hadn't time >>>>>>> to dtrace. >>>>>>> >>>>>>> Only thing I got so far: When the mutexattr is already set the return >>>>>>> value will result on undefined behaviour. >>>>>>> But why only after version 4.10.x on openindiana/illumos? >>>>>>> >>>>>>> Just for completion here the codelines of samba: >>>>>>> https://git.samba.org/?p=samba.git;a=blob;f=lib/tdb/common/mutex.c;h=8a122d5d6a2ec321bce25cf06062acc808f1e126;hb=HEAD#l570 >>>>>>> >>>>>>> >>>>>>> >>>>>>> Gesendet: Mittwoch, 17. April 2019 um 19:19 Uhr >>>>>>> Von: "Youzhong Yang" <youzh...@gmail.com> >>>>>>> An: "Rouven WEILER" <rouven_wei...@gmx.net> >>>>>>> Cc: "Robert Mustacchi" <r...@joyent.com>, "Toomas Soome via >>>>>>> illumos-developer" <develo...@lists.illumos.org> >>>>>>> Betreff: Re: [Samba] gencache.tdb: device busy >>>>>>> >>>>>>> Unless I am totally wrong, pthread_mutexattr_init() shouldn't return >>>>>>> EBUSY: >>>>>>> >>>>>>> http://src.illumos.org/source/xref/illumos-gate/usr/src/lib/libc/port/threads/pthr_mutex.c#37 >>>>>>> >>>>>>> >>>>>>> On Tue, Apr 16, 2019 at 2:45 PM Rouven WEILER <rouven_wei...@gmx.net[ >>>>>>> mailto:rouven_wei...@gmx.net <rouven_wei...@gmx.net>]> wrote:Hi Robert >>>>>>> & Youzhong, >>>>>>> >>>>>>> I got a response from the Samba-Devs. >>>>>>> Would the issue mentioned by Youzhong also hot pthread_mutexattr_t? >>>>>>> >>>>>>> Concerning dtrace I hadn’t the time yet but I will check for that. >>>>>>> Is there maybe a way to trace all forks of smbd? Usually the issue >>>>>>> arises after the fork… >>>>>>> >>>>>>> FIND BELOW THE STATEMENT FROM samba.org[http://samba.org]: >>>>>>> >>>>>>> This doesn't look exactly the same. In your >>>>>>> log below you say: >>>>>>> >>>>>>> ret = pthread_mutexattr_init(&ma); in line 570 >>>>>>> I got errorno 16 --> device busy. >>>>>>> >>>>>>> But the bug report is about needing the value >>>>>>> passed to pthread_mutex_init() needing to be >>>>>>> zero, not the value passed to pthread_mutexattr_init(). >>>>>>> >>>>>>> Can you just try adding '= {0}' to all the >>>>>>> places where we declare: >>>>>>> >>>>>>> pthread_mutexattr_t ma >>>>>>> >>>>>>> (i.e. make it say) >>>>>>> >>>>>>> pthread_mutexattr_t ma = {0}; >>>>>>> >>>>>>> and see if it makes your problem go away ? >>>>>>> >>>>>>> END OF STATEMENT >>>>>>> >>>>>>> >>>>>>>> Anfang der weitergeleiteten Nachricht: >>>>>>>> >>>>>>>> Von: Jeremy Allison via samba <sa...@lists.samba.org[ >>>>>>> mailto:sa...@lists.samba.org <sa...@lists.samba.org>]> >>>>>>>> Betreff: Aw: [Samba] gencache.tdb: device busy >>>>>>>> Datum: 16. April 2019 um 02:07:05 MESZ >>>>>>>> An: Rouven WEILER <rouven_wei...@gmx.net[mailto:rouven_wei...@gmx.net >>>>>>> <rouven_wei...@gmx.net>]> >>>>>>>> Kopie: sa...@lists.samba.org[mailto:sa...@lists.samba.org >>>>>>> <sa...@lists.samba.org>] >>>>>>>> Antwort an: Jeremy Allison <j...@samba.org[mailto:j...@samba.org >>>>>>> <j...@samba.org>]> >>>>>>>> >>>>>>>> On Fri, Apr 12, 2019 at 09:46:04AM +0200, Rouven WEILER via samba >>>>>>> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> Hi Jeremy, >>>>>>>>> >>>>>>>>> I got some info on that topic from the illumos devs: >>>>>>>>> >>>>>>>>> >>>>>>>>>> It's a sporadic issue, you're lucky enough to not encounter it on >>>>>>> 4.9.5. >>>>>>>>>> >>>>>>>>>> I confirmed in 4.10.2, it happens: >>>>>>>>>> >>>>>>>>>> winbindd.log: tdb(/tmw-nas-3p/samba/var/lock/gencache.tdb): >>>>>>> tdb_open_ex: tdb_mutex_init failed for >>>>>>> /tmw-nas-3p/samba/var/lock/gencache.tdb: Device busy >>>>>>>>>> >>>>>>>>>> So either apply OS fix, or apply a software fix (zeroing >>>>>>> pthread_mutex_t before pthread_mutex_init()). >>>>>>>>> >>>>>>>>> This all should be based on the following issue: >>>>>>>>> >>>>>>>>> >>>>>>> https://www.illumos.org/issues/9959[https://www.illumos.org/issues/9959] >>>>>>> >>>>>>>>> >>>>>>>>> As far as I understand the issue making an software fix in samba for >>>>>>> the OS seems more reasonable since the problem can occur sporadic and >>>>>>> thus >>>>>>> again... >>>>>>>>> Could you tell me how to help to fix that bug? >>>>>>>> >>>>>>>> This doesn't look exactly the same. In your >>>>>>>> log below you say: >>>>>>>> >>>>>>>> ret = pthread_mutexattr_init(&ma); in line 570 >>>>>>>> I got errorno 16 --> device busy. >>>>>>>> >>>>>>>> But the bug report is about needing the value >>>>>>>> passed to pthread_mutex_init() needing to be >>>>>>>> zero, not the value passed to pthread_mutexattr_init(). >>>>>>>> >>>>>>>> Can you just try adding '= {0}' to all the >>>>>>>> places where we declare: >>>>>>>> >>>>>>>> pthread_mutexattr_t ma >>>>>>>> >>>>>>>> (i.e. make it say) >>>>>>>> >>>>>>>> pthread_mutexattr_t ma = {0}; >>>>>>>> >>>>>>>> and see if it makes your problem go away ? >>>>>>>> >>>>>>>> >>>>>>>>> Gesendet: Dienstag, 02. April 2019 um 18:59 Uhr >>>>>>>>> Von: "Jeremy Allison" <j...@samba.org[mailto:j...@samba.org >>>>>>> <j...@samba.org>]> >>>>>>>>> An: "Rouven WEILER" <rouven_wei...@gmx.net[ >>>>>>> mailto:rouven_wei...@gmx.net <rouven_wei...@gmx.net>]> >>>>>>>>> Cc: sa...@lists.samba.org[mailto:sa...@lists.samba.org >>>>>>> <sa...@lists.samba.org>] >>>>>>>>> Betreff: Re: Re: [Samba] gencache.tdb: device busy >>>>>>>>>> On Tue, Apr 02, 2019 at 01:24:19PM +0200, Rouven WEILER wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I added LOG statements to mutex.c printing the strerror from the >>>>>>> return values. >>>>>>>>>> >>>>>>>>>> From >>>>>>>>>> >>>>>>>>>> ret = pthread_mutexattr_init(&ma); in line 570 >>>>>>>>>> >>>>>>>>>> I got errorno 16 --> device busy. >>>>>>>>>> >>>>>>>>>> But due the manual of pthread_mutex_attr_init() there should only be >>>>>>> ENOMEM and EINVAL... >>>>>>>>>> >>>>>>>>>> Doing gdb smbd -> run >>>>>>>>>> Doe not reveal anything, but I never used gdb for debugging >>>>>>> anything. I just gave it a shot... >>>>>>>>> >>>>>>>>> Sorry, looks a little like the libc/pthread_mutex support >>>>>>>>> on Solaris is broken. You might want to raise this on a >>>>>>>>> Solaris/Illumos forum. >>>>>>>>> >>>>>>>>> -- >>>>>>>>> To unsubscribe from this list go to the following URL and read the >>>>>>>>> instructions: >>>>>>> https://lists.samba.org/mailman/options/samba[https://lists.samba.org/mailman/options/samba] >>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> To unsubscribe from this list go to the following URL and read the >>>>>>>> instructions: >>>>>>> https://lists.samba.org/mailman/options/samba[https://lists.samba.org/mailman/options/samba] >>>>>>> >>>>>>> >>>>>>> ------------------------------------------ >>>>>>> illumos: illumos-developer >>>>>>> Permalink: >>>>>>> https://illumos.topicbox.com/groups/developer/T38b7c9b3f92fa699-M99a2afeb19714648c051ab5f >>>>>>> Delivery options: >>>>>>> https://illumos.topicbox.com/groups/developer/subscription >>>>>>> >>>>>>> *illumos <https://illumos.topicbox.com/latest>* / illumos-developer / >>>>> see discussions <https://illumos.topicbox.com/groups/developer> + >>>>> participants <https://illumos.topicbox.com/groups/developer/members> + >>>>> delivery >>>>> options <https://illumos.topicbox.com/groups/developer/subscription> >>>>> Permalink >>>>> <https://illumos.topicbox.com/groups/developer/T38b7c9b3f92fa699-M2372380a78d4d5078f031f0c> >>>>> >>>>> >>> >>> ------------------------------------------ >>> illumos: illumos-developer >>> Permalink: >>> https://illumos.topicbox.com/groups/developer/T38b7c9b3f92fa699-M1048b12f29cda459a4de0893 >>> Delivery options: https://illumos.topicbox.com/groups/developer/subscription >> >> _______________________________________________ >> oi-dev mailing list >> oi-dev@openindiana.org >> https://openindiana.org/mailman/listinfo/oi-dev > _______________________________________________ > oi-dev mailing list > oi-dev@openindiana.org > https://openindiana.org/mailman/listinfo/oi-dev
_______________________________________________ oi-dev mailing list oi-dev@openindiana.org https://openindiana.org/mailman/listinfo/oi-dev