MonkeyCanCode commented on PR #426: URL: https://github.com/apache/polaris/pull/426#issuecomment-2464988064
> > Hello @fivetran-arunsuri , > > That is expected as Polaris is not enable eclipse-link by default. It is possible to enable it as part of the build. See following for detail (same can be found in https://polaris.apache.org/in-dev/unreleased/metastores/): > > ``` > > Apache Polaris supports the following optional build options: > > > > -PeclipseLink=true – Enables the EclipseLink extension. > > -PeclipseLinkDeps=[groupId]:[artifactId]:[version],... – Specifies one or more additional dependencies for EclipseLink (e.g., JDBC drivers) separated by commas. > > ``` > > > > > > > > > > > > > > > > > > > > > > > > This is archived by the same file you put the PR with following snippet: > > ``` > > if (project.properties.get("eclipseLink") == "true") { > > dependencies { implementation(project(":polaris-eclipselink")) } > > } > > ``` > > > > > > > > > > > > > > > > > > > > > > > > Reasoning for this is by default eclipselink is configured to use h2 database which is not very useful for prod graded deployment. Due to licensing concern as well as various diff backends end-users may preferred, none of the other database drivers are included as of now. There is a purpose to use quarkus to close this gap (#392), however, that is not yet completed as of today. > > Thanks, Yong Zheng > > Hey Yong Zheng, Thank you for the information. I agree that EclipseLink is currently configured to use the H2 database by default. However, I noticed that the steps to include EclipseLink as a build option are missing from the production documentation [here](https://polaris.io/#tag/Configuring-Polaris-for-Production). This also leads to issues because the configuration requires setting up the metastoremanager like this: metaStoreManager: > > ``` > type: eclipse-link > conf-file: META-INF/persistence.xml > persistence-unit: polaris > ``` > > in polaris-server.yml, which currently pulls EclipseLink from the build options as you mentioned, creating confusion. > > Adding the dependency directly in the PR, as I’ve done, should prevent any errors mentioned in the documentation and keep things straightforward. We could also update the documentation to specify that in case additional backends require dependencies for eg postgres here that can be added to the build file, as our team is currently handling it now. Since we’ll be modifying persistence.xml anyway, this approach seems practical. > > I'm unclear on why making EclipseLink optional is necessary. Regardless of the database backend used, this property would still be required for production cases and not the in memory obe, wouldn’t it? Let me know your thoughts. > > Thanks, Arun Hello, So my first PR to Polaris is also making that a must: https://github.com/apache/polaris/pull/47/files. Later on this change got accidently rolled off due to PR https://github.com/apache/polaris/pull/114. Then back to your question, I think it is more likes a preference thing regarding if EclipseLink should not enabled or not by default. IMO, is it not very useful when we can't use persistent backend for a catalog other than some simple testing etc. I took a quick look at the production doc you mentioned above, it does stated the following on how to enable it: ``` To use EclipseLink for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an EclipseLink `persistence.xml` file. This file, local to the Polaris service, contains details of the database used for metastore management and the connection settings. For more information, refer to the [metastore documentation]({{% ref "metastores" %}}). > [!IMPORTANT] > EclipseLink requires > 1. Building the JAR for the EclipseLink extension > 2. Setting the `eclipseLink` gradle property to `true`. > > This can be achieved by setting `eclipseLink=true` in the `gradle.properties` file, or by passing the property explicitly while building all JARs, e.g.: `./gradlew -PeclipseLink=true clean assemble` ``` Then for sure we will need persistent backend. How to add additional libs for persistent backend is document in the main READ.me instead of production doc. I think there may be some documentation improvement needed (feel free to PR those or I can take care it later this weekend). It may be better to raise a discussion with community on weather to enable this by default. Based on my use case, it is part of build on my pipeline. So this is always included along with additional jar dependencies for the db client. Thanks, Yong -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
