Re: [I] [CI][Integration] Spark jobs are failing with problem on org.apache.arrow.flatbuf [arrow]
kiszk commented on issue #41571: URL: https://github.com/apache/arrow/issues/41571#issuecomment-2134390450 FYI: [This](https://github.com/apache/arrow/pull/41832) is my trial. In the current configuration, we cannot upgrade 3.3 or later. It is consistent with @laurentgo 's observation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [CI][Integration] Spark jobs are failing with problem on org.apache.arrow.flatbuf [arrow]
lidavidm commented on issue #41571: URL: https://github.com/apache/arrow/issues/41571#issuecomment-2123690470 Hmm, the POM should reflect the main file. The shaded version might need to be split into a separate artifact... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [CI][Integration] Spark jobs are failing with problem on org.apache.arrow.flatbuf [arrow]
laurentgo commented on issue #41571: URL: https://github.com/apache/arrow/issues/41571#issuecomment-2123679316 The issue seems to be because `maven-shade-plugin` generates and attach a new pom file containing the following dependency: ``` org.apache.arrow arrow-format 17.0.0-SNAPSHOT provided ``` which is exactly what the plugin [is supposed to be doing](https://github.com/apache/arrow/blob/main/java/vector/pom.xml#L233) I have the same resulting pom by using `maven-shade-plugin` 3.3.0, and going through changes, I found [this change](https://github.com/apache/maven-shade-plugin/commit/6ea8543b8b380bd4ebc0e876970957426bb46808) which is a bugfix to the plugin for this [issue](https://issues.apache.org/jira/browse/MSHADE-321) As to how to address this within the Arrow project, I'd need some perspective. The main issue is that a Maven module has a unique `pom.xml` definition for all the artifacts attached to it. But when attaching shaded artifacts the list of dependencies is not quite the same, and this is definitively conflicting. I have no idea which project is actually using the shaded version of `arrow-vector`, but Spark isn't and so if we agree that the pom.xml should reflect the dependencies for the main artifact, then we should tell `maven-shade-plugin` to not generate a new `pom.xml` file. If we need both dependencies, then we are quite in a predicament I'd guess (and same would apply to the other plugins using `maven-shade-plugin`) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [CI][Integration] Spark jobs are failing with problem on org.apache.arrow.flatbuf [arrow]
lidavidm commented on issue #41571: URL: https://github.com/apache/arrow/issues/41571#issuecomment-2123625150 Yes, I've asked Vibhatha if we can get more of those tests into proper CI... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [CI][Integration] Spark jobs are failing with problem on org.apache.arrow.flatbuf [arrow]
laurentgo commented on issue #41571: URL: https://github.com/apache/arrow/issues/41571#issuecomment-2123624129 I guess my bad but it's quite hard to work on improving the build and also track tests which are not part of the CI. I will repost the change though with hopefully a fix, although nobody really understands what's the exact issue? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [CI][Integration] Spark jobs are failing with problem on org.apache.arrow.flatbuf [arrow]
lidavidm commented on issue #41571: URL: https://github.com/apache/arrow/issues/41571#issuecomment-2122292856 Issue resolved by pull request 41628 https://github.com/apache/arrow/pull/41628 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [CI][Integration] Spark jobs are failing with problem on org.apache.arrow.flatbuf [arrow]
lidavidm commented on issue #41571: URL: https://github.com/apache/arrow/issues/41571#issuecomment-2106188593 It would be good to know _why_ maven-shade-plugin is causing issues, though -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [CI][Integration] Spark jobs are failing with problem on org.apache.arrow.flatbuf [arrow]
lidavidm commented on issue #41571: URL: https://github.com/apache/arrow/issues/41571#issuecomment-2106188425 We can revert. Sorry about that @kiszk -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [CI][Integration] Spark jobs are failing with problem on org.apache.arrow.flatbuf [arrow]
kiszk commented on issue #41571: URL: https://github.com/apache/arrow/issues/41571#issuecomment-2106144955 I am just surprised [this comment](https://github.com/apache/arrow/pull/41309/files#diff-48b02078fbd28d81ac2b82d219d24be49fc3c5a21d8384c5772cc531a713dd9dL154-L160) is completely ignored. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [CI][Integration] Spark jobs are failing with problem on org.apache.arrow.flatbuf [arrow]
kiszk commented on issue #41571: URL: https://github.com/apache/arrow/issues/41571#issuecomment-2106144227 https://github.com/apache/arrow/pull/41309#issuecomment-2106123625 causes this failure. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [CI][Integration] Spark jobs are failing with problem on org.apache.arrow.flatbuf [arrow]
raulcd commented on issue #41571: URL: https://github.com/apache/arrow/issues/41571#issuecomment-2106144152 @danepitkin @lidavidm what should we do with this one? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [CI][Integration] Spark jobs are failing with problem on org.apache.arrow.flatbuf [arrow]
kiszk commented on issue #41571: URL: https://github.com/apache/arrow/issues/41571#issuecomment-2106123760 In the previous issue, [this PR](https://github.com/apache/arrow/pull/40462) causes the issue. > MINOR: [Java] Bump org.apache.maven.plugins:maven-shade-plugin from 3.2.4 to 3.5.2 in /java [This PR](https://github.com/apache/arrow/pull/41309), which was merged on April 29, include the following change. > Update maven-shade-plugin to 3.5.1 (via org.apache:parent) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [CI][Integration] Spark jobs are failing with problem on org.apache.arrow.flatbuf [arrow]
kiszk commented on issue #41571: URL: https://github.com/apache/arrow/issues/41571#issuecomment-2101838863 2024-05-07?, failure [test-conda-python-3.10-spark-v3.5.0](https://github.com/ursacomputing/crossbow/actions/runs/8977656814/job/24656792706) 2024-05-09, failure [test-conda-python-3.10-spark-v3.5.0](https://github.com/ursacomputing/crossbow/actions/runs/9010366090/job/24756302671#step:7:15559) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [CI][Integration] Spark jobs are failing with problem on org.apache.arrow.flatbuf [arrow]
kiszk commented on issue #41571: URL: https://github.com/apache/arrow/issues/41571#issuecomment-2099016715 The error message is the same as https://github.com/apache/arrow/issues/40549. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [CI][Integration] Spark jobs are failing with problem on org.apache.arrow.flatbuf [arrow]
raulcd commented on issue #41571: URL: https://github.com/apache/arrow/issues/41571#issuecomment-2098362012 This is also reproducible for 16.1.0 but I don't think this is related to any change on our side and doesn't seem to be a blocker. @danepitkin @vibhatha @kiszk any thoughts? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
