Looking at this post<http://mail-archives.apache.org/mod_mbox/incubator-thrift-user/200911.mbox/%[email protected]%3e>it looks like Thrift expects thread safety to be implemented in the application layer. So if you protect the client calls with a mutex that would work too and shouldn't have any more of a performance hit than if the locks were pushed into the generated thrift code.
-Sanjit On Thu, Jul 1, 2010 at 7:26 AM, Sanjit Jhala <[email protected]> wrote: > You're right, it looks like the thrift client isn't threadsafe. I'll take a > look at whether theres a thrift knob to generate threadsafe client code. > There shouldn't be any other reason that the Hypertable thrift client > shouldn't be thread safe. > > -Sanjit > > > > On Thu, Jul 1, 2010 at 2:48 AM, Sreejith K <[email protected]> wrote: > >> Hi Sanjit, >> >> So, the only way to use ThriftClient in a multithreaded application is to >> instantiate the client in all the threads. Any plans to make it thread-safe >> ? Just curious to know why it is behaving like this. >> >> -- >> Regards, >> >> Sreejith K >> >> >> >> On Thu, Jul 1, 2010 at 9:52 AM, Sreejith K <[email protected]> wrote: >> >>> Thanks for the reply Sanjit. That seems to have solved the issue. >>> >>> -- >>> Regards, >>> >>> Sreejith K >>> >>> >>> >>> On Thu, Jul 1, 2010 at 12:29 AM, Sanjit Jhala <[email protected]> wrote: >>> >>>> Hi Sreejith, >>>> >>>> Instead of sharing the ThriftClient across all the threads, try creating >>>> one ThriftClient per thread (ie 'push "client = ThriftClient( >>>> 'localhost', 38080)" ' into the 'Get' method in your program), >>>> >>>> -Sanjit >>>> >>>> >>>> >>>> On Wed, Jun 30, 2010 at 12:45 AM, Sreejith K <[email protected]>wrote: >>>> >>>>> Hi, >>>>> >>>>> Actually the failed open_scanner/next_cells calls are being called from >>>>> newly spawned threads. I wrote a sample program to reproduce this. See the >>>>> program and its output. >>>>> >>>>> Hypertable content: http://pastie.org/1024681 >>>>> Program: http://pastie.org/1024685 >>>>> Output: http://pastie.org/1024684 >>>>> >>>>> >>>>> -- >>>>> Regards, >>>>> >>>>> Sreejith K >>>>> >>>>> >>>>> >>>>> On Wed, Jun 30, 2010 at 8:05 AM, Sanjit Jhala <[email protected]>wrote: >>>>> >>>>>> And, I assume you're running the ThriftBroker from this 0.9.3.3 build >>>>>> ? Do you see this error on every open_scanner/next_cells call or some >>>>>> specific one ? >>>>>> >>>>>> -Sanjit >>>>>> >>>>>> >>>>>> On Tue, Jun 29, 2010 at 6:32 AM, Sreejith K <[email protected]>wrote: >>>>>> >>>>>>> Hi Sanjit, >>>>>>> >>>>>>> I used thrift-0.3.0-rc4 ( >>>>>>> http://people.apache.org/~bryanduxbury/thrift-0.3.0-rc4.tar.gz<http://people.apache.org/%7Ebryanduxbury/thrift-0.3.0-rc4.tar.gz>) >>>>>>> for building the binary packages from hypertable 0.9.3.3 source. I >>>>>>> installed >>>>>>> thrift python bindings from thrift-0.3.0-rc4 to >>>>>>> /usr/local/lib/python2.6/dist-packages/thrift. The cmake output >>>>>>> confirmed >>>>>>> the thrift version as 0.3.0 as shown below >>>>>>> >>>>>>> s...@codedesk:~/build/hypertable$ cmake -DPACKAGE_THRIFTBROKER= >>>>>>> -DBUILD_SHARED_LIBS=ON -DCMAKE_BUILD_TYPE=Release ~/src/hypertable >>>>>>> -DJAVA_INCLUDE_PATH=/usr/lib/jvm/java-6-sun-1.6.0.20/include >>>>>>> -DJAVA_INCLUDE_PATH2=/usr/lib/jvm/java-6-sun-1.6.0.20/include/linux >>>>>>> -DHADOOP_INCLUDE_PATH=/home/sree/src/hadoop-0.20.2/src/c++/install/include >>>>>>> -DHADOOP_LIB_PATH=/home/sree/src/hadoop-0.20.2/src/c++/install/lib >>>>>>> -DPHPTHRIFT_ROOT=/home/sree/src/thrift-0.3.0/lib/php/src >>>>>>> -- Use thread library: -lpthread >>>>>>> -- Looking for required boost libraries... >>>>>>> -- GCC version: 4.4.1 >>>>>>> -- Boost version: 1_38 >>>>>>> -- Boost thread lib: /usr/lib/libboost_thread-mt.so >>>>>>> -- Boost program options lib: /usr/lib/libboost_program_options-mt.so >>>>>>> -- Boost filesystem lib: /usr/lib/libboost_filesystem-mt.so >>>>>>> -- Boost iostreams lib: /usr/lib/libboost_iostreams-mt.so >>>>>>> -- Boost python lib: /usr/lib/libboost_python-mt.so >>>>>>> -- Boost system lib: /usr/lib/libboost_system-mt.so >>>>>>> -- Boost lib dir: /usr/lib >>>>>>> -- Found BerkeleyDB: /usr/local/lib/libdb_cxx.so >>>>>>> -- Berkeley DB version: 4.8.26 >>>>>>> -- Found RRDtool: /usr/lib/librrd.so >>>>>>> -- RRDtool version: 1.3.1 >>>>>>> -- Found Readline libraries: >>>>>>> /usr/lib/libreadline.so;/usr/lib/libncurses.so >>>>>>> -- Found SIGAR: /usr/local/lib/libsigar-x86-linux.so;dl >>>>>>> -- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) >>>>>>> -- Found Tcmalloc: /usr/local/lib/libtcmalloc_minimal.so >>>>>>> -- version: 1.5 >>>>>>> -- Not Found Hoard: Hoard_LIBRARY-NOTFOUND >>>>>>> -- Did not find ceph libraries >>>>>>> -- Found Ant: Apache Ant version 1.7.1 compiled on October 19 2009 >>>>>>> -- Javac: javac 1.6.0_20 >>>>>>> -- Found libevent: /usr/lib/libevent.so >>>>>>> -- Found thrift: >>>>>>> /usr/local/lib/libthrift.so;/usr/local/lib/libthriftnb.so >>>>>>> -- compiler: Thrift version 0.3.0 >>>>>>> -- Thrift for ruby not found. ThriftBroker support for ruby will be >>>>>>> disabled >>>>>>> -- Found thrift for php: /home/sree/src/thrift-0.3.0/lib/php/src >>>>>>> -- Found thrift for perl >>>>>>> -- Found thrift for python >>>>>>> -- Java headers found at: /usr/lib/jvm/java-6-sun-1.6.0.20/include >>>>>>> -- Hadoop includes located at: >>>>>>> /home/sree/src/hadoop-0.20.2/src/c++/install/include >>>>>>> -- Hadoop libraries located at: >>>>>>> /home/sree/src/hadoop-0.20.2/src/c++/install/lib >>>>>>> -- Configuring done >>>>>>> -- Generating done >>>>>>> -- Build files have been written to: /home/sree/build/hypertable >>>>>>> >>>>>>> The only strange thing I noticed was the version for thrift bindings >>>>>>> for python. It says 0.1 >>>>>>> >>>>>>> s...@codedesk:~/build/hypertable$ cat >>>>>>> /usr/local/lib/python2.6/dist-packages/Thrift-0.1.egg-info >>>>>>> Metadata-Version: 1.0 >>>>>>> Name: Thrift >>>>>>> Version: 0.1 >>>>>>> Summary: Thrift Python Libraries >>>>>>> Home-page: http://incubator.apache.org/thrift/ >>>>>>> Author: ['Thrift Developers'] >>>>>>> Author-email: ['[email protected]'] >>>>>>> License: Apache License 2.0 >>>>>>> Description: UNKNOWN >>>>>>> Platform: UNKNOWN >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Regards, >>>>>>> >>>>>>> Sreejith K >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, Jun 29, 2010 at 12:19 AM, Sanjit Jhala <[email protected]>wrote: >>>>>>> >>>>>>>> Are the ThriftBroker and ThriftClient code generated by the same >>>>>>>> version of the code and Thrift? From the error message it sounds like >>>>>>>> theres >>>>>>>> a mismatch in the protocol. >>>>>>>> >>>>>>>> -Sanjit >>>>>>>> >>>>>>>> On Mon, Jun 28, 2010 at 1:06 AM, Sreejith K >>>>>>>> <[email protected]>wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I'm using latest Hypertable thriftclient for Python from version >>>>>>>>> 0.9.3.3 (built dynamically with Thrift package from >>>>>>>>> http://people.apache.org/~bryanduxbury/thrift-0.3.0-rc4.tar.gz<http://people.apache.org/%7Ebryanduxbury/thrift-0.3.0-rc4.tar.gz>). >>>>>>>>> But when using open_scanner and next_cells api calls its raising >>>>>>>>> TApplicationException and sometimes EOFError. >>>>>>>>> >>>>>>>>> TApplicationException: next_cells failed: unknown result >>>>>>>>> TApplicationException: open_scanner failed: unknown result >>>>>>>>> >>>>>>>>> Unfortunately I'm unable to reproduce this error using a sample >>>>>>>>> script. What could be the reason for thriftclient to raise such >>>>>>>>> errors ? >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Regards, >>>>>>>>> >>>>>>>>> Sreejith K >>>>>>>>> >>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "Hypertable Development" group. >>>>>>>>> To post to this group, send email to >>>>>>>>> [email protected]. >>>>>>>>> To unsubscribe from this group, send email to >>>>>>>>> [email protected]<hypertable-dev%[email protected]> >>>>>>>>> . >>>>>>>>> For more options, visit this group at >>>>>>>>> http://groups.google.com/group/hypertable-dev?hl=en. >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "Hypertable Development" group. >>>>>>>> To post to this group, send email to >>>>>>>> [email protected]. >>>>>>>> To unsubscribe from this group, send email to >>>>>>>> [email protected]<hypertable-dev%[email protected]> >>>>>>>> . >>>>>>>> For more options, visit this group at >>>>>>>> http://groups.google.com/group/hypertable-dev?hl=en. >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Hypertable Development" group. >>>>>>> To post to this group, send email to [email protected] >>>>>>> . >>>>>>> To unsubscribe from this group, send email to >>>>>>> [email protected]<hypertable-dev%[email protected]> >>>>>>> . >>>>>>> For more options, visit this group at >>>>>>> http://groups.google.com/group/hypertable-dev?hl=en. >>>>>>> >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Hypertable Development" group. >>>>>> To post to this group, send email to [email protected]. >>>>>> To unsubscribe from this group, send email to >>>>>> [email protected]<hypertable-dev%[email protected]> >>>>>> . >>>>>> For more options, visit this group at >>>>>> http://groups.google.com/group/hypertable-dev?hl=en. >>>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Hypertable Development" group. >>>>> To post to this group, send email to [email protected]. >>>>> To unsubscribe from this group, send email to >>>>> [email protected]<hypertable-dev%[email protected]> >>>>> . >>>>> For more options, visit this group at >>>>> http://groups.google.com/group/hypertable-dev?hl=en. >>>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Hypertable Development" group. >>>> To post to this group, send email to [email protected]. >>>> To unsubscribe from this group, send email to >>>> [email protected]<hypertable-dev%[email protected]> >>>> . >>>> For more options, visit this group at >>>> http://groups.google.com/group/hypertable-dev?hl=en. >>>> >>> >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "Hypertable Development" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]<hypertable-dev%[email protected]> >> . >> For more options, visit this group at >> http://groups.google.com/group/hypertable-dev?hl=en. >> > > -- You received this message because you are subscribed to the Google Groups "Hypertable Development" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/hypertable-dev?hl=en.
