[jira] [Comment Edited] (FLINK-24427) Hide Scala in flink-table-planner from API
[ https://issues.apache.org/jira/browse/FLINK-24427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17439238#comment-17439238 ] Chesnay Schepler edited comment on FLINK-24427 at 11/5/21, 12:50 PM: - What is currently blocking you in this investigation? Conceptually I would think we can use the exact same pattern we used for flink-rpc-akka; bundle scala and flink-table-planner in a separate jar, extract it at runtime, wrap all APIs in a wrapper that sets the context class loader on access. Sure, it would currently mean we bundle Scala twice, but that's a price I would be willing to pay for the time being. Are there any dependencies on flink-table-planner that actually require the code to be accessible for compilation? was (Author: zentol): What is currently blocking you in this investigation? Conceptually I would think we can use the exact same pattern we used for flink-rpc-akka; bundle scala and flink-table-planner in a separate jar, extract it at runtime, wrap all APIs in a wrapper that sets the context class loader on access. Sure, it would currently mean we bundle Scala twice, but that's a price I would be willing to pay for the time being. Are there any (actual!) compile dependencies on flink-table-planner? > Hide Scala in flink-table-planner from API > -- > > Key: FLINK-24427 > URL: https://issues.apache.org/jira/browse/FLINK-24427 > Project: Flink > Issue Type: Sub-task > Components: API / Scala, Table SQL / API, Table SQL / Planner >Reporter: Timo Walther >Priority: Major > > FLIP-32 has decoupled the planner from the API. However, the planner code > base is still the largest Scala code base we have in Flink that we cannot > port to Java easily. > In order to allow arbitrary Scala versions in the user API, we suggest to > hide the Scala version of the planner from the Scala version of the API. The > API is mostly developed in Java and contains only a couple of classes > implemented in Scala. Those should be easier to maintain for various Scala > versions. The planner is already discovered via Java SPI. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (FLINK-24427) Hide Scala in flink-table-planner from API
[ https://issues.apache.org/jira/browse/FLINK-24427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17439238#comment-17439238 ] Chesnay Schepler edited comment on FLINK-24427 at 11/5/21, 12:49 PM: - What is currently blocking you in this investigation? Conceptually I would think we can use the exact same pattern we used for flink-rpc-akka; bundled scala and flink-table-planner in a separate jar, extract it at runtime, wrap all APIs in a wrapper that sets the context class loader on access. Sure, it would currently mean we bundle Scala twice, but that's a price I would be willing to pay for the time being. Are there any (actual!) compile dependencies on flink-table-planner? was (Author: zentol): What is currently blocking you in this investigation? Conceptually I would think we can use the exact same pattern we used for flink-rpc-akka; bundled scala and flink-table-planner in a separate jar, extract it at runtime, wrap all APIs in a wrapper that sets the context class loader on access. Sure, it would currently mean we bundle Scala twice, but that's a price I would be willing to pay for the time being. Are the any compile dependencies on flink-table-planner? > Hide Scala in flink-table-planner from API > -- > > Key: FLINK-24427 > URL: https://issues.apache.org/jira/browse/FLINK-24427 > Project: Flink > Issue Type: Sub-task > Components: API / Scala, Table SQL / API, Table SQL / Planner >Reporter: Timo Walther >Priority: Major > > FLIP-32 has decoupled the planner from the API. However, the planner code > base is still the largest Scala code base we have in Flink that we cannot > port to Java easily. > In order to allow arbitrary Scala versions in the user API, we suggest to > hide the Scala version of the planner from the Scala version of the API. The > API is mostly developed in Java and contains only a couple of classes > implemented in Scala. Those should be easier to maintain for various Scala > versions. The planner is already discovered via Java SPI. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (FLINK-24427) Hide Scala in flink-table-planner from API
[ https://issues.apache.org/jira/browse/FLINK-24427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17439238#comment-17439238 ] Chesnay Schepler edited comment on FLINK-24427 at 11/5/21, 12:49 PM: - What is currently blocking you in this investigation? Conceptually I would think we can use the exact same pattern we used for flink-rpc-akka; bundle scala and flink-table-planner in a separate jar, extract it at runtime, wrap all APIs in a wrapper that sets the context class loader on access. Sure, it would currently mean we bundle Scala twice, but that's a price I would be willing to pay for the time being. Are there any (actual!) compile dependencies on flink-table-planner? was (Author: zentol): What is currently blocking you in this investigation? Conceptually I would think we can use the exact same pattern we used for flink-rpc-akka; bundled scala and flink-table-planner in a separate jar, extract it at runtime, wrap all APIs in a wrapper that sets the context class loader on access. Sure, it would currently mean we bundle Scala twice, but that's a price I would be willing to pay for the time being. Are there any (actual!) compile dependencies on flink-table-planner? > Hide Scala in flink-table-planner from API > -- > > Key: FLINK-24427 > URL: https://issues.apache.org/jira/browse/FLINK-24427 > Project: Flink > Issue Type: Sub-task > Components: API / Scala, Table SQL / API, Table SQL / Planner >Reporter: Timo Walther >Priority: Major > > FLIP-32 has decoupled the planner from the API. However, the planner code > base is still the largest Scala code base we have in Flink that we cannot > port to Java easily. > In order to allow arbitrary Scala versions in the user API, we suggest to > hide the Scala version of the planner from the Scala version of the API. The > API is mostly developed in Java and contains only a couple of classes > implemented in Scala. Those should be easier to maintain for various Scala > versions. The planner is already discovered via Java SPI. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (FLINK-24427) Hide Scala in flink-table-planner from API
[ https://issues.apache.org/jira/browse/FLINK-24427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17423039#comment-17423039 ] Seth Wiesman edited comment on FLINK-24427 at 9/30/21, 11:01 PM: - Hi [~eloisant], The goal is to hide the planner from the user code classloader so that user code can use any scala version without the community having to cross build the planner. There are a few ways to achieve this in practice; and I'm not sure what specifically Timo has in mind. One option is to use Flink's plugin mechanism, which is used to load file systems and metric reporters, and hides all dependencies in dedicated classloaders. Another is what we recently did to hide akka from flink-runtime, making it effectively scala free FLINK-18783 was (Author: sjwiesman): Hi [~eloisant], The goal is to hide the planner from the user code classloader so that user code can use any scala version without the community having to cross build the planner. There are a few ways to achieve this in practice; and I'm not sure what specifically Timo has in mind. One option is to use Flink's plugin mechanism, which is used to load file systems and metric reporters, and hides all dependencies in a dedicated classloader. Another is what we recently did to hide akka from flink-runtime, making it effectively scala free FLINK-18783 > Hide Scala in flink-table-planner from API > -- > > Key: FLINK-24427 > URL: https://issues.apache.org/jira/browse/FLINK-24427 > Project: Flink > Issue Type: Sub-task > Components: API / Scala, Table SQL / API, Table SQL / Planner >Reporter: Timo Walther >Priority: Major > > FLIP-32 has decoupled the planner from the API. However, the planner code > base is still the largest Scala code base we have in Flink that we cannot > port to Java easily. > In order to allow arbitrary Scala versions in the user API, we suggest to > hide the Scala version of the planner from the Scala version of the API. The > API is mostly developed in Java and contains only a couple of classes > implemented in Scala. Those should be easier to maintain for various Scala > versions. The planner is already discovered via Java SPI. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (FLINK-24427) Hide Scala in flink-table-planner from API
[ https://issues.apache.org/jira/browse/FLINK-24427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17422979#comment-17422979 ] Erwan Loisant edited comment on FLINK-24427 at 9/30/21, 7:08 PM: - I'm not sure what's the proposed solution here, I don't think hiding the version would really make it possible to use code compiled for Scala 2.12 in 2.13. So I guess either we cross-compile to multiple version (like it was done for 2.11 and 2.12 until now), or we migrate all the code to Java? was (Author: eloisant): I'm not sure what's the proposed solution here, I don't think hiding the version would really make it possible to use code compiled for Scala 2.12 in 2.13. So I guess either we cross-compile to multiple version (like it was done for 2.11 and 2.12 until now), or we migrate all the code to Java? > Hide Scala in flink-table-planner from API > -- > > Key: FLINK-24427 > URL: https://issues.apache.org/jira/browse/FLINK-24427 > Project: Flink > Issue Type: Sub-task > Components: API / Scala, Table SQL / API, Table SQL / Planner >Reporter: Timo Walther >Priority: Major > > FLIP-32 has decoupled the planner from the API. However, the planner code > base is still the largest Scala code base we have in Flink that we cannot > port to Java easily. > In order to allow arbitrary Scala versions in the user API, we suggest to > hide the Scala version of the planner from the Scala version of the API. The > API is mostly developed in Java and contains only a couple of classes > implemented in Scala. Those should be easier to maintain for various Scala > versions. The planner is already discovered via Java SPI. -- This message was sent by Atlassian Jira (v8.3.4#803005)
