piby180 opened a new issue, #16356: URL: https://github.com/apache/pinot/issues/16356
Hi all, I am trying to enable cursor support for our Pinot cluster. From the docs [here](https://docs.pinot.apache.org/users/user-guide-query/query-using-cursors), I can see we can either store the query results in local storage or S3. For pinot helm chart, we do not have any option to deploy persistent volume for broker pods, so we use node's ephermal storage for broker pods. It could be that the local storage is not sufficient in size so I wanted to use S3. However it is not working with S3. It is storing the results at `/tmp/broker/responseStore/data` and `/tmp/broker/responseStore/temp` My broker configuration is ``` pinot.query.runner.port=8442 pinot.broker.enable.query.cancellation=true pinot.broker.adaptive.server.selector.enable.stats.collection=true pinot.broker.adaptive.server.selector.type=HYBRID # Turn on resource usage tracking in statistics framework. # Configuration has to be set in broker and server config files. pinot.broker.instance.enableThreadAllocatedBytesMeasurement=true pinot.server.instance.enableThreadAllocatedBytesMeasurement=true pinot.broker.instance.enableThreadCpuTimeMeasurement=true pinot.server.instance.enableThreadCpuTimeMeasurement=true pinot.query.scheduler.accounting.factory.name=org.apache.pinot.core.accounting.PerQueryCPUMemAccountantFactory pinot.query.scheduler.accounting.enable.thread.memory.sampling=true # Set in broker and server pinot.query.scheduler.accounting.oom.enable.killing.query=true pinot.query.scheduler.accounting.query.killed.metric.enabled=true pinot.query.scheduler.accounting.enable.thread.cpu.sampling=true pinot.query.scheduler.accounting.oom.enable.killing.query=true pinot.query.scheduler.accounting.publishing.jvm.heap.usage=true # Cursor Response store pinot.broker.cursor.result.store.type=file pinot.broker.storage.factory.s3.region=${AWS_S3_REGION} pinot.broker.storage.factory.class.s3=org.apache.pinot.plugin.filesystem.S3PinotFS pinot.broker.cursor.result.store.file.temp.dir=s3://mybucket/cursors/temp pinot.broker.cursor.result.store.file.data.dir=s3://mybucket/cursors/data pinot.broker.cursor.response.store.expiration=1h pinot.broker.cursor.fetch.rows=1000 controller.cluster.response.store.cleaner.frequencyPeriod=1h controller.cluster.response.store.cleaner.initialDelay=60 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
