davidzollo commented on PR #11648: URL: https://github.com/apache/seatunnel/pull/11648#issuecomment-5371214674
Hi @loustler, Good job. Thank you very much for your contribution. I would like to clarify that seatunnel-shade module has been separated into a separate repository. SeaTunnel introduces the seatunnel-shade dependency through the jar method. You can refer to the documentation: https://github.com/apache/seatunnel/blob/dev/docs/en/developer/shade-guide.md By the way, https://github.com/apache/seatunnel/pull/9993 has been merged now. so you can update your PR. Please ignore the following message if you have joined in the Slack~ ``` To facilitate focused communication among contributors, we’ve created a brand-new dedicated #contributors channel in official Slack workspace. Slack workspace link: https://s.apache.org/seatunnel-slack This channel is designed for technical discussions, feature design brainstorming, roadmap sync and direct communication with project core Contributors. Everyone who contributes code, docs, test cases or community ideas is warmly invited to join, share thoughts and collaborate together under The Apache Way. Welcome to join 😄 > @hawk9821 @davidzollo — following up on #11851. Before I put more work into #11648 I'd like to check where it should go. > > #11851 reverted #11673 because `seatunnel-shade` has moved to apache/seatunnel-shade, so changes made to it here no longer take effect. #11648 touches the same module, so presumably the same applies. > > Before splitting it up, though, I want to be sure I'm reading the current state right. On `dev` as it stands: > > * the root pom still lists `<module>seatunnel-shade</module>` > * `seatunnel-dist` still pulls `seatunnel-hadoop3-3.1.4-uber` and `seatunnel-hadoop-aws` at `${project.version}`, i.e. from the reactor, not the released standalone artifacts (those are on Central at `3.1.4-3.0.0`) > * [[Feature][shade]Refactor the seatunnel-shade module. #9993](https://github.com/apache/seatunnel/pull/9993) is still open > > So an in-repo shade change does still end up in the build output today. Is the intent that these sources are already frozen and #9993 is the last step? That would make the revert make sense to me — I just don't want to assume it. > > The other thing is that #11648 can't move wholesale. Of its 88 files only 3 are shade: > > * `seatunnel-shade/pom.xml` > * `seatunnel-shade/seatunnel-hadoop-aws/pom.xml` — AWS SDK v2 bundle filtering and the `org/slf4j/impl/**` exclusion > * `seatunnel-hadoop3-3.1.4-uber/pom.xml` → `seatunnel-hadoop3-3.4.3-uber/pom.xml` > > The rest is connector-file-s3, connector-file-oss, `seatunnel-dist`, `seatunnel-e2e`, checkpoint-storage-hdfs, imap-storage-file and docs. That has to stay here, and it only works once a shade build with Hadoop 3.4.3 and AWS SDK v2 exists. > > So what I'd do, if you agree: take those two poms to apache/seatunnel-shade (bump `hadoop3.version` / `hadoop-aws.version` to 3.4.3, carry over the SDK v2 bundle allowlist that keeps the jar around 25 MB instead of 658 MB), get a release out so `3.4.3-<shade-version>` is on Central, then rebase #11648 onto that for the remaining 85 files. > > Moving it actually removes a problem. Over there the Hadoop version sits in the artifact version (`${hadoop3.version}-${seatunnel.shade.version}`) instead of the artifactId, so there's no module to rename. The rename here is the one real breaking change in #11648 and the reason it needs an `incompatible-changes.md` entry. > > So: > > 1. Is my reading right — shade sources here already frozen, [[Feature][shade]Refactor the seatunnel-shade module. #9993](https://github.com/apache/seatunnel/pull/9993) the last step? > 2. Split it as above, or hold everything until [[Feature][shade]Refactor the seatunnel-shade module. #9993](https://github.com/apache/seatunnel/pull/9993) lands? > 3. If I open a shade PR, roughly how long until a release? Step three is blocked on it, so it matters whether that's days or a release cycle. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
