I understand, and I appreciate you all for that! And I would guess that there is possibly more that can be done, but it would be debugging work that would be pertinent to the pintool environment. It is something I will have to tackle. I appreciate your willingness to help me out and provide suggestions :)
On Sat, Jan 25, 2014 at 1:19 AM, Michel Pelletier < [email protected]> wrote: > We as a community have probably provided the most that can be reasonable > expected. You're working with a new, apparently closed source library that > few of us, or perhaps none at all, have heard of. If it's not possible to > provide us with an isolated test case then there's likely little we can do > besides provide some broad debugging support. > > I looked at you diff against zeromq master on github, it looked reasonable > enough. Your best bet now would be to probably use some debugging tools > to dig deeper. As Pieter pointed out it seems your socket is invalid or > closed. Have you investigated those possibilities? > > -Michel > > > On Fri, Jan 24, 2014 at 2:51 PM, Kenneth Adam Miller < > [email protected]> wrote: > >> Is anybody still willing to work on this with me? >> >> >> On Fri, Jan 24, 2014 at 12:57 PM, Kenneth Adam Miller < >> [email protected]> wrote: >> >>> In that case, it would be a separately named library; the developers of >>> their application wouldn't execute their application in a pintool context, >>> so they would use the vanilla library, because they would never be able to >>> get their application off the ground due to link errors from undefined >>> symbols and runtime unresolved references. They would never think to >>> themselves "I'm going to run my normal everyday application which uses >>> regular threads in a regular environment with an obscure variant of >>> zeromq." As a result, the vanilla zeromq would spawn threads the normal >>> way, and hence they would get instrumented. Sure you would have some >>> unnecessary data/space consumption from replicated >>> >>> This particular library could just be named to something like libpinzmq >>> or libzmq-pintool. Then it should work fine. >>> >>> >>> On Fri, Jan 24, 2014 at 12:51 PM, Lindley French <[email protected]>wrote: >>> >>>> Ideally, it would be possible to use ZMQ simultaneously inside of and >>>> outside of a pintool context. I'm not familiar with pintools, but if it's >>>> an instrumentation framework, what if they code you're trying to instrument >>>> *also* uses ZMQ? >>>> >>>> >>>> On Fri, Jan 24, 2014 at 1:44 PM, Kenneth Adam Miller < >>>> [email protected]> wrote: >>>> >>>>> No; It calls PIN_SpawnInternalThread, which only makes sense in a >>>>> pintool's executing environment. Basically, if you use this particular >>>>> modified version of zmq, you specifically want to use zmq in a pintool >>>>> context. >>>>> >>>>> I wouldn't say my changes are "harm", they are what I see as necessary >>>>> in order to realize pintool utilization of zeromq. I'm just not as >>>>> familiar with the zeromq code base as perhaps you guys are. :-) >>>>> >>>>> >>>>> On Fri, Jan 24, 2014 at 12:42 PM, Lindley French >>>>> <[email protected]>wrote: >>>>> >>>>>> Question: Does your modified code work properly outside of a pintool >>>>>> context? EG, does it at least pass the "do no harm" test, before you >>>>>> worry >>>>>> about whether it does what you want it to do? >>>>>> >>>>>> >>>>>> On Fri, Jan 24, 2014 at 1:37 PM, Kenneth Adam Miller < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> It must use pintools, because I had to refactor libzmq in order to >>>>>>> play nice with pintool; that's what the question is about. The 200 line >>>>>>> program perfectly replicates the issue of not being able to create a >>>>>>> router-proxy-dealer setup exactly as laid out in the manual; I get the >>>>>>> same >>>>>>> error using that sample as in my currently closed source pintool. >>>>>>> >>>>>>> The reason why I have to use a pintool to demonstrate the problem is >>>>>>> because I must give an example that shows the augmented libzmq >>>>>>> operating, >>>>>>> and the augmented libzmq spawns threads using the PIN thread API (just >>>>>>> as >>>>>>> the pintool manual specifies that thread control must in a pintool >>>>>>> executing environment). I just realized I didn't try to explain that >>>>>>> thread >>>>>>> spawning and thread control primitives must be done through the PIN api; >>>>>>> basically because PIN is a dynamic instrumentation framework, if you >>>>>>> spawn >>>>>>> a thread, even in the instrumentation or analysis routines that you >>>>>>> define, >>>>>>> using the normal facilities, then PIN will have a recursive condition >>>>>>> against the thread that you innocently meant to spawn to help you do >>>>>>> analysis or instrumentation faster. In order to prevent this, they >>>>>>> provide >>>>>>> the PIN thread APIs. >>>>>>> >>>>>>> I like zeromq, and I would like to use it to coordinate my analysis >>>>>>> and instrumentation routines. The problem is, the zeromq library uses >>>>>>> some >>>>>>> thread control and primitives that it shouldn't. >>>>>>> >>>>>>> I'm sorry, but I can't make the example any less complicated by >>>>>>> moving the example out of a pintool context. I can try and eliminate >>>>>>> some >>>>>>> code lines... If you have any questions about pintool itself, please >>>>>>> feel >>>>>>> free to ask. I'm the one getting help from you guys in the long run, so >>>>>>> I'm >>>>>>> more than happy to help you help me :-) >>>>>>> >>>>>>> To be honest, I acknowledge that I'm bringing in a complex library >>>>>>> that has nothing to do with this mailing list itself, but this porting >>>>>>> needs to occur in order for the pintool community to benefit from the >>>>>>> capabilities that zmq offers. I'm highly motivated by zeromq's success >>>>>>> and >>>>>>> flexibility, enough that I'm willing to dive an into unfamiliar code dig >>>>>>> through and find what I think needs to be changed and change it. >>>>>>> >>>>>>> >>>>>>> On Fri, Jan 24, 2014 at 12:02 PM, Pieter Hintjens >>>>>>> <[email protected]>wrote: >>>>>>> >>>>>>>> I saw the repository and saw some code but a 200+ line program isn't >>>>>>>> really a minimal test case. As far as I understand, using pintools >>>>>>>> should be transparent to users, since it doesn't modify the ZMQ API, >>>>>>>> right? So you can make a minimal test case that doesn't require any >>>>>>>> knowledge of pintools to understand. >>>>>>>> >>>>>>>> On Fri, Jan 24, 2014 at 8:39 AM, Kenneth Adam Miller >>>>>>>> <[email protected]> wrote: >>>>>>>> > The example is on the github repo where it has >>>>>>>> PinTool-ErrorReplication. >>>>>>>> > >>>>>>>> > >>>>>>>> > On Fri, Jan 24, 2014 at 8:29 AM, Pieter Hintjens <[email protected]> >>>>>>>> wrote: >>>>>>>> >> >>>>>>>> >> Hi Kenneth, >>>>>>>> >> >>>>>>>> >> Could you post a minimal example of the code that causes an >>>>>>>> error? If >>>>>>>> >> possible, in C, using the low level API. First step is to let us >>>>>>>> >> simply inspect this and check for obvious misuse of sockets. >>>>>>>> >> >>>>>>>> >> The "socket operation on non-socket" can happen when you pass an >>>>>>>> >> invalid argument to a zmq socket call, or when you are trying to >>>>>>>> use a >>>>>>>> >> closed socket. >>>>>>>> >> >>>>>>>> >> -Pieter >>>>>>>> >> >>>>>>>> >> On Fri, Jan 24, 2014 at 7:28 AM, Kenneth Adam Miller >>>>>>>> >> <[email protected]> wrote: >>>>>>>> >> > I would like to use zeromq to work with intel pintool ( >>>>>>>> pintool.org). >>>>>>>> >> > Pintool >>>>>>>> >> > is a dynamic binary instrumentation framework; it allows the >>>>>>>> developer >>>>>>>> >> > to >>>>>>>> >> > specify some callbacks at many levels of execution of any >>>>>>>> binary >>>>>>>> >> > program. >>>>>>>> >> > But to allow zeromq to work with it, you have to restrict >>>>>>>> thread >>>>>>>> >> > creation >>>>>>>> >> > down to only the context or whatever other threads you have, >>>>>>>> and then >>>>>>>> >> > specify that the thread creation and any other thread apis go >>>>>>>> through >>>>>>>> >> > the >>>>>>>> >> > pin interface. If they don't then you potentially have a >>>>>>>> recursive >>>>>>>> >> > instrument and execute condition, where a thread you want to >>>>>>>> work for >>>>>>>> >> > the >>>>>>>> >> > instrumentation/analysis side is also being targeted for >>>>>>>> >> > instrumentation. >>>>>>>> >> > >>>>>>>> >> > My changes to zeromq consist of some minor alterations to >>>>>>>> thread.hpp and >>>>>>>> >> > mutex.hpp. You can see what I've changed on my forked >>>>>>>> repository over >>>>>>>> >> > here-all I do is replace the threading primitives that each >>>>>>>> class is >>>>>>>> >> > abstracting from with an instance for pin's. >>>>>>>> >> > >>>>>>>> >> > I know for certain that my changes are at taking effect, >>>>>>>> because what I >>>>>>>> >> > can >>>>>>>> >> > see is that inter thread communication for just ZMQ_PULL and >>>>>>>> ZMQ_PUSH >>>>>>>> >> > works >>>>>>>> >> > just fine. But when I go to create dealer's and router's and >>>>>>>> link them >>>>>>>> >> > with >>>>>>>> >> > a proxy I start getting errors with code that came straight >>>>>>>> from the >>>>>>>> >> > manual, >>>>>>>> >> > which is really not good. I would prefer that my hack continue >>>>>>>> to work, >>>>>>>> >> > and >>>>>>>> >> > I have confidence that with some help I can make it work. >>>>>>>> >> > >>>>>>>> >> > custom libzmq implementation: >>>>>>>> >> > https://github.com/KennethAdamMiller/libzmq >>>>>>>> >> > >>>>>>>> >> > ./autogen.sh >>>>>>>> >> > >>>>>>>> >> > (for 64 bit) >>>>>>>> >> > ./configure CXXFLAGS="-DBIGARRAY_MULTIPLIER=1 -Wall -Werror >>>>>>>> >> > -Wno-unknown-pragmas -fno-stack-protector -DTARGET_IA32E >>>>>>>> -DHOST_IA32E >>>>>>>> >> > -fPIC >>>>>>>> >> > -DTARGET_LINUX -I$HOME/pin-2.13/source/include/pin >>>>>>>> >> > -I$HOME/pin-2.13/source/include/pin/gen >>>>>>>> >> > -I$HOME/pin-2.13/extras/components/include >>>>>>>> >> > -I$HOME/pin-2.13/extras/xed2-intel64/include >>>>>>>> >> > -I$HOME/pin-2.13/source/tools/InstLib -O3 -fomit-frame-pointer >>>>>>>> >> > -fno-strict-aliasing -g -std=c++11 -L$HOMEpin-2.13/intel64/lib >>>>>>>> >> > -L$HOME/pin-2.13/intel64/lib-ext >>>>>>>> -L$HOME/pin-2.13/intel64/runtime/glibc >>>>>>>> >> > -L$HOME/pin-2.13/extras/xed2-intel64/lib -lpin -lxed -ldwarf >>>>>>>> -lelf -ldl >>>>>>>> >> > " >>>>>>>> >> > >>>>>>>> >> > sudo make install >>>>>>>> >> > >>>>>>>> >> > repeatable pintool error: >>>>>>>> >> > https://github.com/KennethAdamMiller/PinTool-ErrorReplication >>>>>>>> >> > Building this pintool is easy: >>>>>>>> >> > download a pin kit and extract to some location, like >>>>>>>> ~/pin-2.13 >>>>>>>> >> > >>>>>>>> >> > export PIN_ROOT=location >>>>>>>> >> > export TOOLS_ROOT=$PIN_ROOT/source/tools >>>>>>>> >> > export CONFIG_ROOT=$TOOLS_ROOT/Config >>>>>>>> >> > >>>>>>>> >> > then just cd to where you checked out my tool and type make. >>>>>>>> You'll see >>>>>>>> >> > that >>>>>>>> >> > you get socket operation on non-socket. I was hoping that >>>>>>>> someone else >>>>>>>> >> > could >>>>>>>> >> > help me find out why it is getting that error, and then may >>>>>>>> suggest what >>>>>>>> >> > I >>>>>>>> >> > change in libzmq. >>>>>>>> >> > >>>>>>>> >> > _______________________________________________ >>>>>>>> >> > zeromq-dev mailing list >>>>>>>> >> > [email protected] >>>>>>>> >> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>>>>>>> >> > >>>>>>>> >> _______________________________________________ >>>>>>>> >> zeromq-dev mailing list >>>>>>>> >> [email protected] >>>>>>>> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > _______________________________________________ >>>>>>>> > zeromq-dev mailing list >>>>>>>> > [email protected] >>>>>>>> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>>>>>>> > >>>>>>>> _______________________________________________ >>>>>>>> zeromq-dev mailing list >>>>>>>> [email protected] >>>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> zeromq-dev mailing list >>>>>>> [email protected] >>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>>>>>> >>>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> zeromq-dev mailing list >>>>>> [email protected] >>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>>>>> >>>>>> >>>>> >>>>> _______________________________________________ >>>>> zeromq-dev mailing list >>>>> [email protected] >>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> zeromq-dev mailing list >>>> [email protected] >>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>>> >>>> >>> >> >> _______________________________________________ >> zeromq-dev mailing list >> [email protected] >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> >> > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > >
_______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
