apotheque opened a new pull request, #7729: URL: https://github.com/apache/ozone/pull/7729
## What changes were proposed in this pull request? I tried running Trino together with Ozone over OFS but encountered a Protobuf shading incompatibility between the [Trino Hadoop library](https://github.com/trinodb/trino-hadoop-apache/tree/master) and the Ozone Hadoop client: ``` 2024-12-27T09:09:30.178Z DEBUG hive-hive-0 org.apache.hadoop.fs.ozone.BasicRootedOzoneFileSystem Ozone URI for OFS initialization is ofs://ozone-om-1 2024-12-27T09:09:30.183Z WARN hive-hive-0 org.apache.hadoop.ozone.om.helpers.OzoneFSUtils Ignoring ozone.fs.hsync.enabled = true because HBase enhancements are disallowed. To enable it, set ozone.client.hbase.enhancements.allowed = true as well. 2024-12-27T09:09:30.184Z DEBUG hive-hive-0 org.apache.hadoop.fs.ozone.BasicRootedOzoneFileSystem hsyncEnabled = false 2024-12-27T09:09:30.473Z INFO dispatcher-query-3 io.trino.event.QueryMonitor TIMELINE: Query 20241227_090929_00003_x4ukx :: FINISHED :: elapsed 480ms :: planning 134ms :: waiting 23ms :: scheduling 202ms :: running 128ms :: finishing 16ms :: begin 2024-12-27T09:09:29.959Z :: end 2024-12-27T09:09:30.641Z INFO hive-hive-0 org.apache.hadoop.hdds.utils.LeakDetector Starting leak detector thread OzoneClientObject0. 2024-12-27T09:09:30.816Z DEBUG hive-hive-0 org.apache.hadoop.ozone.om.protocolPB.OmTransportFactory Loading OM transport implementation org.apache.hadoop.ozone.om.protocolPB.Hadoop3OmTransportFactory as specified by configuration. 2024-12-27T09:09:31.241Z DEBUG hive-hive-0 org.apache.hadoop.ozone.om.ha.OMFailoverProxyProviderBase RetryProxy: OM om1: class org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos$OMRequest cannot be cast to class io.trino.hadoop.$internal.com.google.protobuf.Message (org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos$OMRequest and io.trino.hadoop.$internal.com.google.protobuf.Message are in unnamed module of loader io.trino.filesystem.manager.HdfsClassLoader @43fffec2) 2024-12-27T09:09:31.243Z DEBUG hive-hive-0 org.apache.hadoop.ozone.om.ha.OMFailoverProxyProviderBase Incrementing OM proxy index to 0, nodeId: om1 2024-12-27T09:09:33.246Z DEBUG hive-hive-0 org.apache.hadoop.ozone.om.ha.OMFailoverProxyProviderBase Failing over OM from om1:0 to om1:0 2024-12-27T09:09:33.246Z DEBUG hive-hive-0 org.apache.hadoop.ozone.om.ha.OMFailoverProxyProviderBase RetryProxy: OM om1: null ``` The simplest way to resolve this issue is to adjust the `ozonefs-hadoop3-client` module, which already shades Protobuf, but currently uses a fixed prefix. I suggest making the prefix configurable as a separate property to allow customization if needed. With the proposed changes, running `mvn clean package -Dproto.shaded.prefix='io.trino.hadoop.\$internal'` inside the `ozonefs-hadoop3-client` module will properly shade Protobuf and allow the output JAR to be used with Trino. Related Trino issue: https://github.com/trinodb/trino/discussions/18026 ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-12116 ## How was this patch tested? Manually tested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ozone.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@ozone.apache.org For additional commands, e-mail: issues-h...@ozone.apache.org
