Thanks for the suggestion.
I tried it. I passed it as a string. I get the following error:
File "client.py", line 27, in <module>
proc_res = stub.SquareProc(req)
File "/home/jatin/.local/lib/python3.8/site-packages/grpc/_channel.py",
line 826, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/home/jatin/.local/lib/python3.8/site-packages/grpc/_channel.py",
line 729, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated
with:
status = StatusCode.UNKNOWN
details = "Exception calling application: invalid syntax (<string>, line 1)"
debug_error_string =
"{"created":"@1601270472.165351495","description":"Error received from peer
ipv6:[::1]:8080","file":"src/core/lib/surface/call.cc","file_line":1061,"grpc_message":"Exception
calling application: invalid syntax (<string>, line 1)","grpc_status":2}"
And when I passed it as bytes after changing it to pickle format, I got the
following error:
Traceback (most recent call last):
File "client.py", line 19, in <module>
req.class_str = code
TypeError: "class Square_Loc():\n def proc(self,df):\n
print('This is the client function being sent has type str, but expected
one of: bytes
On Fri, Sep 25, 2020 at 1:24 PM Russell Wu <[email protected]> wrote:
> Generally, allowing users to run arbitrary code on the server is a huge
> security risk. But if you really wanna do it, in python it's relatively
> easy.
>
> What you need to send to the server is the code itself. To do that, you
> first get the code of the function/class, and send it to the server. On the
> server, you use eval to run it with a dict as local scope, then your method
> will be available in that dict/scope. Pseudo code as below
>
> ```
> # in client.py
> import inspect
> from .loc import SquareLoc
>
> code = inspect.getsource(SquareLoc)
> req = myrequest_pb2.Request()
> req.code = code
> stub.Get_Data(req)
>
> # in server.py handler code
> ctx = {}
> eval(req.code, {}, ctx)
> SquareLoc = ctx['SquareLoc']
> ...
> ```
>
>
> On Fri, Sep 25, 2020 at 3:02 PM Jatin Sharma <[email protected]> wrote:
>
>> Hi Russell,
>> It worked.
>> However, I believe this may not be the solution that I am looking for.
>> The idea that I'm trying to apply is that when the server contains a
>> database/pandas and the client sends a request to the server. The request
>> includes the function that needs to operate on the database.
>>
>> In the suggested solution, I have to make the class available on the
>> server. If I have to make the class available on the server, it beats the
>> purpose of using a client to send the pickled procedure. Is there a
>> workaround such that I will not have to make the class available on the
>> server?
>>
>> On Fri, Sep 25, 2020 at 11:41 AM Russell Wu <[email protected]> wrote:
>>
>>> You will need to put your Square_Loc in a shared module, whose import
>>> path must be identical to your client and server, for example:
>>>
>>> ```
>>> # in client.py
>>> from .loc import Square_Loc
>>>
>>> # in server.py
>>> from .loc import Square_Loc
>>> ```
>>>
>>>
>>>
>>> On Fri, Sep 25, 2020 at 2:00 PM Jatin Sharma <[email protected]>
>>> wrote:
>>>
>>>> I rewrote the program to pass a class. Now I'm getting the following
>>>> error:
>>>>
>>>> Traceback (most recent call last):
>>>> File "client.py", line 25, in <module>
>>>> proc_res = stub.SquareProc(serverdata_pb2.client_class(class_str =
>>>> pick_sq))
>>>> File
>>>> "/home/jatin/.local/lib/python3.8/site-packages/grpc/_channel.py", line
>>>> 826, in __call__
>>>> return _end_unary_response_blocking(state, call, False, None)
>>>> File
>>>> "/home/jatin/.local/lib/python3.8/site-packages/grpc/_channel.py", line
>>>> 729, in _end_unary_response_blocking
>>>> raise _InactiveRpcError(state)
>>>> grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that
>>>> terminated with:
>>>> status = StatusCode.UNKNOWN
>>>> details = "Exception calling application: Can't get attribute
>>>> 'Square_Loc' on <module '__main__' from 'server.py'>"
>>>> debug_error_string =
>>>> "{"created":"@1601012690.097357367","description":"Error received from peer
>>>> ipv6:[::1]:50051","file":"src/core/lib/surface/call.cc","file_line":1061,"grpc_message":"Exception
>>>> calling application: Can't get attribute 'Square_Loc' on <module '__main__'
>>>> from 'server.py'>","grpc_status":2}"
>>>>
>>>> So, Square_Loc is the class that I have defined in the client and sent
>>>> to the server. Can you suggest to me how I can debug this? Thanks
>>>>
>>>> On Thu, Sep 24, 2020 at 12:17 PM Russell Wu <[email protected]> wrote:
>>>>
>>>>> It seems to me that on the server when the `loads` trys to restore the
>>>>> Python object Get_Hash it cannot find it. Because for classes, pickle
>>>>> doesn't serialize its implementation but its import path (in this case
>>>>> `main.Get_Hash` if you run client.py directly). I suggest you put a
>>>>> try...except clause around the logic in Get_Data handler and print it
>>>>>
>>>>> On Thu, Sep 24, 2020 at 2:00 PM Jatin Sharma <[email protected]>
>>>>> wrote:
>>>>>
>>>>>> Hi Russell,
>>>>>> Please find below the code for server.
>>>>>>
>>>>>> import grpc
>>>>>> from concurrent import futures
>>>>>> import time
>>>>>> import pickle
>>>>>>
>>>>>> import pandas as pd
>>>>>> import serverdata_pb2
>>>>>> import serverdata_pb2_grpc
>>>>>>
>>>>>> def get_df():
>>>>>> cars = {'Brand': ['Honda Civic','Toyota Corolla','Ford
>>>>>> Focus','Audi A4'],
>>>>>> 'Price': [22000,25000,27000,35000]
>>>>>> }
>>>>>>
>>>>>> df = pd.DataFrame(cars, columns = ['Brand', 'Price'])
>>>>>> return df
>>>>>>
>>>>>> class DataProviderServicer(serverdata_pb2_grpc.DataProviderServicer):
>>>>>> def Get_Data(self,request,context):
>>>>>> get_func = pickle.loads(request.client_class)
>>>>>> a = get_func()
>>>>>> df = get_df()
>>>>>> ans = a.get_hash(df)
>>>>>> response = serverdata_pb2.result()
>>>>>> response.result = ans
>>>>>> return response
>>>>>>
>>>>>> server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
>>>>>>
>>>>>> serverdata_pb2_grpc.add_DataProviderServicer_to_server(
>>>>>> DataProviderServicer(), server)
>>>>>>
>>>>>> print('Starting server. Listening on port 50051.')
>>>>>> server.add_insecure_port('[::]:50051')
>>>>>> server.start()
>>>>>>
>>>>>> try:
>>>>>> while True:
>>>>>> time.sleep(86400)
>>>>>> except KeyboardInterrupt:
>>>>>> server.stop(0)
>>>>>>
>>>>>> On Thu, Sep 24, 2020 at 9:04 AM Russell Wu <[email protected]> wrote:
>>>>>>
>>>>>>> Please also post the log on the server side. It does look like some
>>>>>>> error happened in Get_Data handler on the server. Maybe mismatch of
>>>>>>> generated client_class code between client and server?
>>>>>>>
>>>>>>> On Thu, Sep 24, 2020 at 11:11 AM Jatin Sharma <[email protected]>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I don't understand the handle terminology. Can you please explain
>>>>>>>> what you mean by handle. Thanks.
>>>>>>>>
>>>>>>>> On Wednesday, September 23, 2020 at 11:08:35 PM UTC+5:30
>>>>>>>> [email protected] wrote:
>>>>>>>>
>>>>>>>>> I would expect there to be more to the error message than just
>>>>>>>>> "client_class". I'm assuming there's an indentation problem in your
>>>>>>>>> original post and that the instantiation of the "client_class"
>>>>>>>>> message is
>>>>>>>>> happening within the Get_Data handler. What happens if you try to
>>>>>>>>> instantiate a "client_class" outside of the handler, on the main
>>>>>>>>> thread. Do
>>>>>>>>> you get a more illuminating error message?
>>>>>>>>> On Wednesday, September 23, 2020 at 9:20:45 AM UTC-4 Jatin Sharma
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> I want to send a class to the grpc server. So, I am pickling the
>>>>>>>>>> class and sharing as a bytes format in the message.
>>>>>>>>>> The .proto file(serverdata.proto) looks like below:
>>>>>>>>>>
>>>>>>>>>> syntax = "proto3";
>>>>>>>>>>
>>>>>>>>>> service DataProvider{
>>>>>>>>>> rpc Get_Data(client_class) returns (result);
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> message client_class{
>>>>>>>>>> bytes class_str = 1;
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> message result{
>>>>>>>>>> int64 res = 1;
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> client.py file looks like below:
>>>>>>>>>>
>>>>>>>>>> import grpc
>>>>>>>>>> import serverdata_pb2
>>>>>>>>>> import serverdata_pb2_grpc
>>>>>>>>>> import pickle
>>>>>>>>>> import pandas as pd
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> class Get_Hash():
>>>>>>>>>> def get_hash(self,df):
>>>>>>>>>> return pd.util.hash_pandas_object(df).sum()
>>>>>>>>>>
>>>>>>>>>> a = serverdata_pb2.client_class()
>>>>>>>>>> a.class_str = pickle.dumps(Get_Hash)
>>>>>>>>>>
>>>>>>>>>> channel = grpc.insecure_channel('localhost:50051')
>>>>>>>>>> # create a stub (client)
>>>>>>>>>> stub = serverdata_pb2_grpc.DataProviderStub(channel)
>>>>>>>>>> response = stub.Get_Data(a)
>>>>>>>>>>
>>>>>>>>>> print(response.res)
>>>>>>>>>>
>>>>>>>>>> On running this client, I'm getting the following error:
>>>>>>>>>>
>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>> File "client.py", line 18, in <module>
>>>>>>>>>> response = stub.Get_Data(a)
>>>>>>>>>> File
>>>>>>>>>> "/home/jatin/.local/lib/python3.8/site-packages/grpc/_channel.py",
>>>>>>>>>> line
>>>>>>>>>> 826, in __call__
>>>>>>>>>> return _end_unary_response_blocking(state, call, False, None)
>>>>>>>>>> File
>>>>>>>>>> "/home/jatin/.local/lib/python3.8/site-packages/grpc/_channel.py",
>>>>>>>>>> line
>>>>>>>>>> 729, in _end_unary_response_blocking
>>>>>>>>>> raise _InactiveRpcError(state)
>>>>>>>>>> grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that
>>>>>>>>>> terminated with:
>>>>>>>>>> status = StatusCode.UNKNOWN
>>>>>>>>>> details = "Exception calling application: client_class"
>>>>>>>>>> debug_error_string =
>>>>>>>>>> "{"created":"@1600867086.557068214","description":"Error received
>>>>>>>>>> from peer
>>>>>>>>>> ipv6:[::1]:50051","file":"src/core/lib/surface/call.cc","file_line":1061,"grpc_message":"Exception
>>>>>>>>>> calling application: client_class","grpc_status":2}"
>>>>>>>>>> >
>>>>>>>>>>
>>>>>>>>>> I'm unable to resolve this error. I checked the type of the
>>>>>>>>>> pickle file. It was 'bytes'. So, I changed the type in the .proto
>>>>>>>>>> file to
>>>>>>>>>> bytes. I'd be grateful if someone can help me resolve this error.
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Jatin
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>> Groups "grpc.io" group.
>>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>>> send an email to [email protected].
>>>>>>>> To view this discussion on the web visit
>>>>>>>> https://groups.google.com/d/msgid/grpc-io/65656b59-cc4d-40d7-a00f-eef9b8f785ecn%40googlegroups.com
>>>>>>>> <https://groups.google.com/d/msgid/grpc-io/65656b59-cc4d-40d7-a00f-eef9b8f785ecn%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>>>>> .
>>>>>>>>
>>>>>>>
--
You received this message because you are subscribed to the Google Groups
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/grpc-io/CAHsNn%2Bn9dW8s7vz5mW9PznkGiZuRz1_LZ_RZdmcLqxQTotWSaQ%40mail.gmail.com.
