2020-10-13 10:09:56 UTC - alex kurtser: Hello Naive question about Presto SQL and offloading to a S3. We are going to implement data offloading to the S3 and we also have Presto SQL which quires data from the pulsar
As I understand, the Presto quires data directly from bookkeepers. So have i also configure offloading settings in the Presto settings ? I see these settings commented out in the pulsar.properties. Do we really need to configure them if we want to offload a data from bookkeeepers to S3? Does it mean that presto SQL quires the offloaded data directly from S3 but not via bookkeepers ? ---- 2020-10-13 13:38:30 UTC - David Tinker: Anything in particular I need to watch out for upgrading from 2.3.1 to 2.6.1? Seems the bk upgrade (4.9.0 -> 4.10.0) is ok and thats the most risky part right? ---- 2020-10-13 14:10:06 UTC - Sijie Guo: There are no incompatible changes between these versions. So you don’t need to take any special steps here +1 : David Tinker ---- 2020-10-13 14:11:46 UTC - Sijie Guo: If the data is offloaded to s3, presto will query the data from s3 directly. ---- 2020-10-13 16:10:58 UTC - Rattanjot Singh: bin/bookkeeper shell bookiesanity ```09:07:43.510 [main-SendThread(10.80.124.56:2281)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server 10.80.124.56/10.80.124.56:2281. Will not attempt to authenticate using SASL (unknown error) 09:07:43.515 [main-SendThread(10.80.124.56:2281)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established, initiating session, client: /10.80.124.10:41036, server: 10.80.124.56/10.80.124.56:2281``` Getting this when i run this. Any config that I might be missing. ---- 2020-10-13 16:21:36 UTC - Addison Higham: what issue are you seeing there? are you trying to connect with SASL auth? ---- 2020-10-13 16:26:19 UTC - Rattanjot Singh: ```EnabledProtocols = TLSv1.2 # TLS Provider (JDK or OpenSSL). tlsProvider=OpenSSL``` using tls1.2 for Tls settings ---- 2020-10-13 16:27:31 UTC - Addison Higham: does it fail? SASL != TLS, do you have your bookkeeper configured to present a tls cert? ---- 2020-10-13 18:23:32 UTC - Alexandre DUVAL: Considering <https://pastebin.com/XFQ3XvwF>, why the backlogSize is that size? :open_mouth: ---- 2020-10-13 19:21:15 UTC - alex kurtser: Thanks ---- 2020-10-13 19:22:33 UTC - alex kurtser: and if we are talking about regular pulsar consumer ? in this case , u guess the consumer reads the data from bookkeeper and not directly from S3 ? ---- 2020-10-13 19:40:59 UTC - Ryan: Hello, we're waiting Transactions and related shared subscription Large Message Support in 2.7.x release. Any thoughts on when 2.7.0 is planned for release? ---- 2020-10-13 20:03:19 UTC - Addison Higham: There is some final work to finish up transactions to get them in, I think that has held up the release a bit, but you should still expect an RC in the next couple of weeks (last I knew) ---- 2020-10-13 20:06:11 UTC - Nathan Mills: Are there any pulsar permissions that can be granted to a role to allow listing of topics in a namespace without granting admin? ---- 2020-10-13 20:07:27 UTC - Frank Kelly: Hey @Addison Higham let me know if you have any questions about the issue above - again not urgent in the least ---- 2020-10-13 20:10:02 UTC - Addison Higham: not with the built-in ACL framework. Since 2.6.0, the ACL framework has been improved so that checks can be much more granular but that isn't yet exposed in the default implementation ---- 2020-10-13 20:10:33 UTC - Nathan Mills: :thumbsup: thanks ---- 2020-10-13 20:21:53 UTC - Ryan: Great, we'll be on the lookout! ----
