Re: [I] [Spark] Move datasources under org.apache.spark package [incubator-graphar]
acezen closed issue #493: [Spark] Move datasources under org.apache.spark package URL: https://github.com/apache/incubator-graphar/issues/493 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [I] [Spark] Move datasources under org.apache.spark package [incubator-graphar]
acezen commented on issue #493: URL: https://github.com/apache/incubator-graphar/issues/493#issuecomment-2128817133 > > > > The graphar package relys on datasource, I don't know how can we release graphar without release the datasource package? > > > > > > > > > It will be a compile time dependency. For example, like in delta-spark: https://github.com/delta-io/delta/tree/master/spark/src/main/scala/org/apache/spark/sql > > > > > > It seems that as a compile time dependency, we need to put datasource back to maven-projects/graphar? Or do you know is there a way that datasource still be separated from graphar and can be a compile time dependency too? That would be perfect for this issue. > > I can experiment with it. It seems to me that yes, it should be possible. Great, thanks Sem, Can I assign this issue to you? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [I] [Spark] Move datasources under org.apache.spark package [incubator-graphar]
SemyonSinchenko commented on issue #493: URL: https://github.com/apache/incubator-graphar/issues/493#issuecomment-2128778602 > > > The graphar package relys on datasource, I don't know how can we release graphar without release the datasource package? > > > > It will be a compile time dependency. For example, like in delta-spark: https://github.com/delta-io/delta/tree/master/spark/src/main/scala/org/apache/spark/sql > > It seems that as a compile time dependency, we need to put datasource back to maven-projects/graphar? Or do you know is there a way that datasource still be separated from graphar and can be a compile time dependency too? That would be perfect for this issue. I can experiment with it. It seems to me that yes, it should be possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [I] [Spark] Move datasources under org.apache.spark package [incubator-graphar]
acezen commented on issue #493: URL: https://github.com/apache/incubator-graphar/issues/493#issuecomment-2128314523 > > The graphar package relys on datasource, I don't know how can we release graphar without release the datasource package? > > It will be a compile time dependency. For example, like in delta-spark: https://github.com/delta-io/delta/tree/master/spark/src/main/scala/org/apache/spark/sql It seems that as a compile time dependency, we need to put datasource back to graphar? Or do you know is there a way that datasource still be separated from graphar and can be a compile time dependency? That would be perfect for this issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [I] [Spark] Move datasources under org.apache.spark package [incubator-graphar]
SemyonSinchenko commented on issue #493: URL: https://github.com/apache/incubator-graphar/issues/493#issuecomment-2126937259 > The graphar package relys on datasource, I don't know how can we release graphar without release the datasource package? It wild be a compile time dependency. For example, like in delta-spark: https://github.com/delta-io/delta/tree/master/spark/src/main/scala/org/apache/spark/sql -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [I] [Spark] Move datasources under org.apache.spark package [incubator-graphar]
acezen commented on issue #493: URL: https://github.com/apache/incubator-graphar/issues/493#issuecomment-2126578853 > > There is a problem if we move to org.apache.spark is that we can not release the datasource maven packages to the org.apache.spark in the future. > > But do we need it? We may release graphar itself and we may release PyPI package that already include JARs from datasources. Should datasources be released separately? The graphar package relys on datasource, I don't know how can we release graphar without release the datasource package? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [I] [Spark] Move datasources under org.apache.spark package [incubator-graphar]
SemyonSinchenko commented on issue #493: URL: https://github.com/apache/incubator-graphar/issues/493#issuecomment-2126567158 > There is a problem if we move to org.apache.spark is that we can not release the datasource maven packages to the org.apache.spark in the future. But do we need it? We may release graphar itself and we may release PyPI package that already include JARs from datasources. Should datasources be released separately? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [I] [Spark] Move datasources under org.apache.spark package [incubator-graphar]
acezen commented on issue #493: URL: https://github.com/apache/incubator-graphar/issues/493#issuecomment-2126563955 There is a problem if we move to org.apache.spark is that we can not release the datasource maven package to the org.apache.spark in the future. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [I] [Spark] Move datasources under org.apache.spark package [incubator-graphar]
SemyonSinchenko commented on issue #493: URL: https://github.com/apache/incubator-graphar/issues/493#issuecomment-2126563976 It may also simplify the full support of `v2` datasources. Last time when I tried to switch to `v2` I faced a problem that few important methods are `private[sql]` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
