motybz opened a new issue, #12974:
URL: https://github.com/apache/arrow/issues/12974
I made a do_get function that returned the GeneratorStream object.
```python
def time_it(func):
"""This decorator prints the execution time for the decorated
function."""
@wraps(func)
def wrapper(*args, **kwargs):
start = time.time()
result = func(*args, **kwargs)
end = time.time()
logging.debug("{} executed in {}s".format(func.__name__, round(end -
start, 2)))
return result
return wrapper
class FlightServer(pa.flight.FlightServerBase):
def __init__(self, host="localhost", location=None,filesystem="s3",
**kwargs):
super(FlightServer, self).__init__(location, **kwargs)
@time_it
def do_get(self, context, ticket):
....
scanner = dataset.scanner(
batch_size=1000*1000*10
)
return pa.flight.GeneratorStream(
scanner.projected_schema, scanner.to_batches()
)
```
The logged time is not for the complete data transfer between the server and
the client but for the initial connection.
How can I log the complete duration of the data streaming (from the
server-side...)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
