Thanks Thomas, Thought like upload request needs to be bytes and for reply used int, that is the issue thanks a lot. Issue solved :)
Thanks Team, it is my mistake. On Tuesday, April 28, 2020 at 12:37:06 AM UTC+5:30, Thomas Mercier wrote: > > The maximum value of an int32 is 2,147,483,647. Why are you using a signed > type for length in the first place though? > > On Mon, Apr 27, 2020 at 12:01 PM VigneshDhanraj G <[email protected] > <javascript:>> wrote: > >> Thanks Michael, >> >> syntax = "proto3"; >> >> service FileServer { >> rpc upload(stream Chunk) returns (Reply) {} >> rpc download(Request) returns (stream Chunk) {} >> } >> >> message Chunk { >> bytes buffer = 1; >> } >> >> message Request { >> string name = 1; >> } >> >> message Reply { >> int32 length = 1; >> >> and my client.py >> >> >> def get_file_chunks(filename): >> with open(filename, 'rb') as f: >> while True: >> piece = f.read(CHUNK_SIZE); >> if len(piece) == 0: >> return >> yield chunk_pb2.Chunk(buffer=piece) >> >> >> def save_chunks_to_file(chunks, filename): >> with open(filename, 'wb') as f: >> for chunk in chunks: >> f.write(chunk.buffer) >> >> >> class FileClient: >> def __init__(self, address): >> channel = grpc.insecure_channel(address) >> self.stub = chunk_pb2_grpc.FileServerStub(channel) >> >> def upload(self, in_file_name): >> chunks_generator = get_file_chunks(in_file_name) >> response = self.stub.upload(chunks_generator) >> assert response.length == os.path.getsize(in_file_name) >> >> def download(self, target_name, out_file_name): >> response = self.stub.download(chunk_pb2.Request(name=target_name)) >> save_chunks_to_file(response, out_file_name) >> >> >> if __name__ == '__main__': >> client = FileClient('localhost:8888') >> >> # demo for file uploading >> in_file_name = sys.argv[1] >> client.upload(in_file_name) >> >> >> >> >> On Tuesday, April 28, 2020 at 12:13:47 AM UTC+5:30, Michael Webster wrote: >>> >>> Do you have any snippets you can share from your code? I use >>> python-grpc in another project for transferring files and I've never had an >>> issue with files > 2gb, with 1mb chunks (local network operations). >>> >>> On Monday, April 27, 2020 at 7:08:50 PM UTC+1, VigneshDhanraj G wrote: >>>> >>>> Thanks Mya, >>>> >>>> There is no any error on the server and i am getting error if i upload >>>> more than 2GB file, i am constanly uploading chunk on the storge. >>>> >>>> On Monday, April 27, 2020 at 8:42:50 PM UTC+5:30, Mya Pitzeruse wrote: >>>>> >>>>> Is there an associated error on the server? The error message suggests >>>>> a server error with an index out of bounds exception: 2^31. Only way I >>>>> could see something like that happening is if the file was being buffered >>>>> on the server side in memory. >>>>> >>>>> On Mon, Apr 27, 2020 at 6:56 AM VigneshDhanraj G <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Team, >>>>>> >>>>>> I am not able to upload large files more than 2GB even after chunk. I >>>>>> have found the file is uploaded still exeception raised? Please help me >>>>>> to >>>>>> understand the issue here and way to fix this problem. >>>>>> >>>>>> Traceback (most recent call last): >>>>>> File "demo_client.py", line 12, in <module> >>>>>> client.upload(in_file_name) >>>>>> File >>>>>> "/home/vigneshdhanraj/Project/grpc-upload/grpc-file-transfer/src/lib.py", >>>>>> >>>>>> line 34, in upload >>>>>> response = self.stub.upload(chunks_generator) >>>>>> File >>>>>> "/home/vigneshdhanraj/Project/grpc-upload/myenv/lib/python3.6/site-packages/grpc/_channel.py", >>>>>> >>>>>> line 1011, in __call__ >>>>>> return _end_unary_response_blocking(state, call, False, None) >>>>>> File >>>>>> "/home/vigneshdhanraj/Project/grpc-upload/myenv/lib/python3.6/site-packages/grpc/_channel.py", >>>>>> >>>>>> line 729, in _end_unary_response_blocking >>>>>> raise _InactiveRpcError(state) >>>>>> grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that >>>>>> terminated with: >>>>>> status = StatusCode.UNKNOWN >>>>>> details = "*Exception calling application: Value out of range: >>>>>> 2147483648*" >>>>>> debug_error_string = >>>>>> "{"created":"@1587649701.194973268","description":"Error received from >>>>>> peer >>>>>> ipv6:[::1]:8888","file":"src/core/lib/surface/call.cc","file_line":1056,"grpc_message":"Exception >>>>>> >>>>>> calling application: Value out of range: 2147483648","grpc_status":2}" >>>>>> >>>>>> Regards, >>>>>> VigneshDhanraj G >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "grpc.io" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/grpc-io/cbc945c2-6d3a-4d2b-a52f-2a6b59a9dfba%40googlegroups.com >>>>>> >>>>>> <https://groups.google.com/d/msgid/grpc-io/cbc945c2-6d3a-4d2b-a52f-2a6b59a9dfba%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Mya Pitzeruse >>>>> >>>>> Principal Software Engineer - Service Infrastructure >>>>> >>>>> Gender Pronouns: She, Her, Hers >>>>> >>>>> [email protected] >>>>> >>>>> >>>>> Indeed - We help people get jobs. >>>>> >>>>> Indeed.com <http://www.indeed.com/> >>>>> >>>>> Facebook <http://www.facebook.com/indeed> | Twitter >>>>> <http://www.twitter.com/indeed> | Instagram >>>>> <http://www.instagram.com/indeedworks> >>>>> >>>> -- >> You received this message because you are subscribed to the Google Groups >> "grpc.io" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/grpc-io/e3f6c515-71cd-40ca-bafb-b409788cb1fb%40googlegroups.com >> >> <https://groups.google.com/d/msgid/grpc-io/e3f6c515-71cd-40ca-bafb-b409788cb1fb%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/1b0d361a-5b03-4a42-91c8-28b3da906280%40googlegroups.com.
