This is an automated email from the ASF dual-hosted git repository.
chengpan pushed a commit to branch branch-0.3
in repository https://gitbox.apache.org/repos/asf/incubator-celeborn.git
The following commit(s) were added to refs/heads/branch-0.3 by this push:
new a16d8af18 [CELEBORN-1003] Correct the LICENSE and NOTICE for shaded
client jars
a16d8af18 is described below
commit a16d8af18adb4029b5348ad0ca8d2ee608fd256e
Author: Cheng Pan <[email protected]>
AuthorDate: Thu Sep 28 19:23:54 2023 +0800
[CELEBORN-1003] Correct the LICENSE and NOTICE for shaded client jars
Correct the `LICENSE` and `NOTICE` for the following shaded client jars
- `celeborn-client-flink-1.14-shaded_2.12-<version>.jar`
- `celeborn-client-flink-1.15-shaded_2.12-<version>.jar`
- `celeborn-client-flink-1.17-shaded_2.12-<version>.jar`
- `celeborn-client-mr-shaded_2.12-<version>.jar`
- `celeborn-client-spark-2-shaded_2.11-<version>.jar`
- `celeborn-client-spark-3-shaded_2.12-<version>.jar`
The `LICENSE` and `NOTICE` shipped in a jar should match the content of the
jar, for shaded jars, it should acknowledge all the third-party classes that
are bundled.
See more discussion at
https://lists.apache.org/thread/8v4wy5o132rpsjync6465zztgjlf6h5p
For how to determine which third-party jars are bundled, take
`celeborn-client-spark-3-shaded_2.12-<version>.jar` as an example, the
following command performs the packaging, and we can find them out by looking
at logs like `Including ... in the shaded jar`
```
build/mvn clean package -DskipTests -pl
:celeborn-client-spark-3-shaded_2.12 -am -Pspark-3.3
```
```
[INFO] --- maven-shade-plugin:3.4.0:shade (default)
celeborn-client-spark-3-shaded_2.12 ---
[INFO] Including
org.apache.celeborn:celeborn-client-spark-3_2.12:jar:0.4.0-SNAPSHOT in the
shaded jar.
[INFO] Including
org.apache.celeborn:celeborn-common_2.12:jar:0.4.0-SNAPSHOT in the shaded jar.
[INFO] Including org.apache.commons:commons-lang3:jar:3.12.0 in the shaded
jar.
[INFO] Including io.netty:netty-all:jar:4.1.93.Final in the shaded jar.
[INFO] Including io.netty:netty-buffer:jar:4.1.93.Final in the shaded jar.
...
[INFO] Excluding org.apache.ratis:ratis-common:jar:2.5.1 from the shaded
jar.
[INFO] Excluding org.apache.ratis:ratis-thirdparty-misc:jar:1.0.4 from the
shaded jar.
[INFO] Excluding org.apache.ratis:ratis-proto:jar:2.5.1 from the shaded jar.
...
```
No.
Manually.
Closes #1933 from pan3793/CELEBORN-1003.
Authored-by: Cheng Pan <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
(cherry picked from commit 78553f141822dbe415481e074296684948168cd9)
Signed-off-by: Cheng Pan <[email protected]>
---
LICENSE | 2 +-
LICENSE-binary | 8 ++--
NOTICE-binary | 17 +++++++++
client-flink/flink-1.14-shaded/pom.xml | 8 ++++
.../src/main/resources/META-INF/LICENSE | 44 +++++++++++++++++++---
.../src/main/resources/META-INF/NOTICE | 13 +++++++
.../META-INF/licenses/LICENSE-protobuf.txt | 42 +++++++++++++++++++++
client-flink/flink-1.15-shaded/pom.xml | 8 ++++
.../src/main/resources/META-INF/LICENSE | 44 +++++++++++++++++++---
.../src/main/resources/META-INF/NOTICE | 13 +++++++
.../META-INF/licenses/LICENSE-protobuf.txt | 42 +++++++++++++++++++++
client-flink/flink-1.17-shaded/pom.xml | 8 ++++
.../src/main/resources/META-INF/LICENSE | 44 +++++++++++++++++++---
.../src/main/resources/META-INF/NOTICE | 13 +++++++
.../META-INF/licenses/LICENSE-protobuf.txt | 42 +++++++++++++++++++++
client-spark/spark-2-shaded/pom.xml | 11 ++++++
.../src/main/resources/META-INF/LICENSE | 44 +++++++++++++++++++---
.../src/main/resources/META-INF/NOTICE | 17 +++++++++
client-spark/spark-3-shaded/pom.xml | 7 ++++
.../src/main/resources/META-INF/LICENSE | 44 +++++++++++++++++++---
.../src/main/resources/META-INF/NOTICE | 17 +++++++++
licenses-binary/LICENSE-BSD-3.txt | 11 ------
licenses-binary/LICENSE-MIT.txt | 21 -----------
23 files changed, 458 insertions(+), 62 deletions(-)
diff --git a/LICENSE b/LICENSE
index 07b27d9bf..2d2b2597f 100644
--- a/LICENSE
+++ b/LICENSE
@@ -207,7 +207,7 @@ This section summarizes those components and their
licenses. See licenses/
for text of these licenses.
-Apache Software Foundation License 2.0
+Apache License 2.0
--------------------------------------
common/src/main/java/org/apache/celeborn/common/network/util/LimitedInputStream.java
diff --git a/LICENSE-binary b/LICENSE-binary
index 8f081384d..ef60263a1 100644
--- a/LICENSE-binary
+++ b/LICENSE-binary
@@ -201,7 +201,7 @@
limitations under the License.
------------------------------------------------------------------------------------
-This project bundles the following dependencies under the Apache Software
License 2.0 (http://www.apache.org/licenses/LICENSE-2.0.txt):
+This project bundles the following dependencies under the Apache License 2.0
(http://www.apache.org/licenses/LICENSE-2.0.txt):
com.google.code.findbugs:jsr305
com.google.guava:guava
@@ -277,20 +277,20 @@ for text of these licenses.
BSD 3-clause
-See license/LICENSE-BSD-3.txt for details.
------------
See license/LICENSE-protobuf.txt for details.
com.google.protobuf:protobuf-java
See license/LICENSE-leveldbjni.txt for details.
org.fusesource.leveldbjni:leveldbjni-all
+
Common Development and Distribution License (CDDL) 1.0
-See license/LICENSE-CDDL.txt for details.
------------
+See license/LICENSE-CDDL.txt for details.
javax.servlet:javax.servlet-api
+
MIT License
-See license/LICENSE-MIT.txt for details.
------------
See license/LICENSE-slf4j.txt for details.
org.slf4j:jul-to-slf4j
diff --git a/NOTICE-binary b/NOTICE-binary
index 709ec77d7..8034aeadf 100644
--- a/NOTICE-binary
+++ b/NOTICE-binary
@@ -74,3 +74,20 @@ Copyright 2005-2006 Tim Fennell
Apache Flink
Copyright 2014-2022 The Apache Software Foundation
+
+Scala
+Copyright (c) 2002-2023 EPFL
+Copyright (c) 2011-2023 Lightbend, Inc.
+
+Scala includes software developed at
+LAMP/EPFL (https://lamp.epfl.ch/) and
+Lightbend, Inc. (https://www.lightbend.com/).
+
+Licensed under the Apache License, Version 2.0 (the "License").
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+This software includes projects with other licenses -- see `doc/LICENSE.md`.
diff --git a/client-flink/flink-1.14-shaded/pom.xml
b/client-flink/flink-1.14-shaded/pom.xml
index eb5a6999d..9f3b52308 100644
--- a/client-flink/flink-1.14-shaded/pom.xml
+++ b/client-flink/flink-1.14-shaded/pom.xml
@@ -78,13 +78,21 @@
<filter>
<artifact>*:*</artifact>
<excludes>
+ <exclude>**/*.proto</exclude>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
<exclude>**/log4j.properties</exclude>
+ <exclude>META-INF/LICENSE.txt</exclude>
+ <exclude>META-INF/NOTICE.txt</exclude>
+ <exclude>LICENSE.txt</exclude>
+ <exclude>NOTICE.txt</exclude>
</excludes>
</filter>
</filters>
+ <transformers>
+ <transformer
implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"></transformer>
+ </transformers>
</configuration>
</plugin>
<plugin>
diff --git a/LICENSE
b/client-flink/flink-1.14-shaded/src/main/resources/META-INF/LICENSE
similarity index 90%
copy from LICENSE
copy to client-flink/flink-1.14-shaded/src/main/resources/META-INF/LICENSE
index 07b27d9bf..0441af2ac 100644
--- a/LICENSE
+++ b/client-flink/flink-1.14-shaded/src/main/resources/META-INF/LICENSE
@@ -202,12 +202,46 @@
------------------------------------------------------------------------------------
-This product bundles various third-party components under other open source
licenses.
-This section summarizes those components and their licenses. See licenses/
-for text of these licenses.
+This project bundles the following dependencies under the Apache License 2.0
(http://www.apache.org/licenses/LICENSE-2.0.txt):
-Apache Software Foundation License 2.0
+Apache License 2.0
--------------------------------------
-common/src/main/java/org/apache/celeborn/common/network/util/LimitedInputStream.java
+org.apache.commons:commons-lang3
+io.netty:netty-all
+io.netty:netty-buffer
+io.netty:netty-codec
+io.netty:netty-codec-dns
+io.netty:netty-codec-haproxy
+io.netty:netty-codec-http
+io.netty:netty-codec-http2
+io.netty:netty-codec-memcache
+io.netty:netty-codec-mqtt
+io.netty:netty-codec-redis
+io.netty:netty-codec-smtp
+io.netty:netty-codec-socks
+io.netty:netty-codec-stomp
+io.netty:netty-codec-xml
+io.netty:netty-common
+io.netty:netty-handler
+io.netty:netty-transport-native-unix-common
+io.netty:netty-handler-proxy
+io.netty:netty-resolver
+io.netty:netty-resolver-dns
+io.netty:netty-transport
+io.netty:netty-transport-rxtx
+io.netty:netty-transport-sctp
+io.netty:netty-transport-udt
+io.netty:netty-transport-classes-epoll
+io.netty:netty-transport-classes-kqueue
+io.netty:netty-transport-native-epoll
+io.netty:netty-transport-native-kqueue
+com.google.guava:guava
+org.roaringbitmap:RoaringBitmap
+
+
+BSD 3-clause
+------------
+See license/LICENSE-protobuf.txt for details.
+com.google.protobuf:protobuf-java
diff --git a/client-flink/flink-1.14-shaded/src/main/resources/META-INF/NOTICE
b/client-flink/flink-1.14-shaded/src/main/resources/META-INF/NOTICE
new file mode 100644
index 000000000..a4001a2ce
--- /dev/null
+++ b/client-flink/flink-1.14-shaded/src/main/resources/META-INF/NOTICE
@@ -0,0 +1,13 @@
+
+Apache Celeborn (Incubating)
+Copyright 2022-2023 The Apache Software Foundation.
+
+This product includes software developed at
+The Apache Software Foundation (https://www.apache.org/).
+
+=============================================================================
+= NOTICE file corresponding to section 4d of the Apache License Version 2.0 =
+=============================================================================
+
+Apache Commons Lang
+Copyright 2001-2021 The Apache Software Foundation
diff --git
a/client-flink/flink-1.14-shaded/src/main/resources/META-INF/licenses/LICENSE-protobuf.txt
b/client-flink/flink-1.14-shaded/src/main/resources/META-INF/licenses/LICENSE-protobuf.txt
new file mode 100644
index 000000000..b4350ec83
--- /dev/null
+++
b/client-flink/flink-1.14-shaded/src/main/resources/META-INF/licenses/LICENSE-protobuf.txt
@@ -0,0 +1,42 @@
+This license applies to all parts of Protocol Buffers except the following:
+
+ - Atomicops support for generic gcc, located in
+ src/google/protobuf/stubs/atomicops_internals_generic_gcc.h.
+ This file is copyrighted by Red Hat Inc.
+
+ - Atomicops support for AIX/POWER, located in
+ src/google/protobuf/stubs/atomicops_internals_aix.h.
+ This file is copyrighted by Bloomberg Finance LP.
+
+Copyright 2014, Google Inc. All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are
+met:
+
+ * Redistributions of source code must retain the above copyright
+notice, this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above
+copyright notice, this list of conditions and the following disclaimer
+in the documentation and/or other materials provided with the
+distribution.
+ * Neither the name of Google Inc. nor the names of its
+contributors may be used to endorse or promote products derived from
+this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+Code generated by the Protocol Buffer compiler is owned by the owner
+of the input file used when generating it. This code is not
+standalone and requires a support library to be linked with it. This
+support library is itself covered by the above license.
\ No newline at end of file
diff --git a/client-flink/flink-1.15-shaded/pom.xml
b/client-flink/flink-1.15-shaded/pom.xml
index bc3c8c7fe..a4597d64b 100644
--- a/client-flink/flink-1.15-shaded/pom.xml
+++ b/client-flink/flink-1.15-shaded/pom.xml
@@ -78,13 +78,21 @@
<filter>
<artifact>*:*</artifact>
<excludes>
+ <exclude>**/*.proto</exclude>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
<exclude>**/log4j.properties</exclude>
+ <exclude>META-INF/LICENSE.txt</exclude>
+ <exclude>META-INF/NOTICE.txt</exclude>
+ <exclude>LICENSE.txt</exclude>
+ <exclude>NOTICE.txt</exclude>
</excludes>
</filter>
</filters>
+ <transformers>
+ <transformer
implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"></transformer>
+ </transformers>
</configuration>
</plugin>
<plugin>
diff --git a/LICENSE
b/client-flink/flink-1.15-shaded/src/main/resources/META-INF/LICENSE
similarity index 90%
copy from LICENSE
copy to client-flink/flink-1.15-shaded/src/main/resources/META-INF/LICENSE
index 07b27d9bf..0441af2ac 100644
--- a/LICENSE
+++ b/client-flink/flink-1.15-shaded/src/main/resources/META-INF/LICENSE
@@ -202,12 +202,46 @@
------------------------------------------------------------------------------------
-This product bundles various third-party components under other open source
licenses.
-This section summarizes those components and their licenses. See licenses/
-for text of these licenses.
+This project bundles the following dependencies under the Apache License 2.0
(http://www.apache.org/licenses/LICENSE-2.0.txt):
-Apache Software Foundation License 2.0
+Apache License 2.0
--------------------------------------
-common/src/main/java/org/apache/celeborn/common/network/util/LimitedInputStream.java
+org.apache.commons:commons-lang3
+io.netty:netty-all
+io.netty:netty-buffer
+io.netty:netty-codec
+io.netty:netty-codec-dns
+io.netty:netty-codec-haproxy
+io.netty:netty-codec-http
+io.netty:netty-codec-http2
+io.netty:netty-codec-memcache
+io.netty:netty-codec-mqtt
+io.netty:netty-codec-redis
+io.netty:netty-codec-smtp
+io.netty:netty-codec-socks
+io.netty:netty-codec-stomp
+io.netty:netty-codec-xml
+io.netty:netty-common
+io.netty:netty-handler
+io.netty:netty-transport-native-unix-common
+io.netty:netty-handler-proxy
+io.netty:netty-resolver
+io.netty:netty-resolver-dns
+io.netty:netty-transport
+io.netty:netty-transport-rxtx
+io.netty:netty-transport-sctp
+io.netty:netty-transport-udt
+io.netty:netty-transport-classes-epoll
+io.netty:netty-transport-classes-kqueue
+io.netty:netty-transport-native-epoll
+io.netty:netty-transport-native-kqueue
+com.google.guava:guava
+org.roaringbitmap:RoaringBitmap
+
+
+BSD 3-clause
+------------
+See license/LICENSE-protobuf.txt for details.
+com.google.protobuf:protobuf-java
diff --git a/client-flink/flink-1.15-shaded/src/main/resources/META-INF/NOTICE
b/client-flink/flink-1.15-shaded/src/main/resources/META-INF/NOTICE
new file mode 100644
index 000000000..a4001a2ce
--- /dev/null
+++ b/client-flink/flink-1.15-shaded/src/main/resources/META-INF/NOTICE
@@ -0,0 +1,13 @@
+
+Apache Celeborn (Incubating)
+Copyright 2022-2023 The Apache Software Foundation.
+
+This product includes software developed at
+The Apache Software Foundation (https://www.apache.org/).
+
+=============================================================================
+= NOTICE file corresponding to section 4d of the Apache License Version 2.0 =
+=============================================================================
+
+Apache Commons Lang
+Copyright 2001-2021 The Apache Software Foundation
diff --git
a/client-flink/flink-1.15-shaded/src/main/resources/META-INF/licenses/LICENSE-protobuf.txt
b/client-flink/flink-1.15-shaded/src/main/resources/META-INF/licenses/LICENSE-protobuf.txt
new file mode 100644
index 000000000..b4350ec83
--- /dev/null
+++
b/client-flink/flink-1.15-shaded/src/main/resources/META-INF/licenses/LICENSE-protobuf.txt
@@ -0,0 +1,42 @@
+This license applies to all parts of Protocol Buffers except the following:
+
+ - Atomicops support for generic gcc, located in
+ src/google/protobuf/stubs/atomicops_internals_generic_gcc.h.
+ This file is copyrighted by Red Hat Inc.
+
+ - Atomicops support for AIX/POWER, located in
+ src/google/protobuf/stubs/atomicops_internals_aix.h.
+ This file is copyrighted by Bloomberg Finance LP.
+
+Copyright 2014, Google Inc. All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are
+met:
+
+ * Redistributions of source code must retain the above copyright
+notice, this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above
+copyright notice, this list of conditions and the following disclaimer
+in the documentation and/or other materials provided with the
+distribution.
+ * Neither the name of Google Inc. nor the names of its
+contributors may be used to endorse or promote products derived from
+this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+Code generated by the Protocol Buffer compiler is owned by the owner
+of the input file used when generating it. This code is not
+standalone and requires a support library to be linked with it. This
+support library is itself covered by the above license.
\ No newline at end of file
diff --git a/client-flink/flink-1.17-shaded/pom.xml
b/client-flink/flink-1.17-shaded/pom.xml
index 88ce11929..32754de00 100644
--- a/client-flink/flink-1.17-shaded/pom.xml
+++ b/client-flink/flink-1.17-shaded/pom.xml
@@ -78,13 +78,21 @@
<filter>
<artifact>*:*</artifact>
<excludes>
+ <exclude>**/*.proto</exclude>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
<exclude>**/log4j.properties</exclude>
+ <exclude>META-INF/LICENSE.txt</exclude>
+ <exclude>META-INF/NOTICE.txt</exclude>
+ <exclude>LICENSE.txt</exclude>
+ <exclude>NOTICE.txt</exclude>
</excludes>
</filter>
</filters>
+ <transformers>
+ <transformer
implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"></transformer>
+ </transformers>
</configuration>
</plugin>
<plugin>
diff --git a/LICENSE
b/client-flink/flink-1.17-shaded/src/main/resources/META-INF/LICENSE
similarity index 90%
copy from LICENSE
copy to client-flink/flink-1.17-shaded/src/main/resources/META-INF/LICENSE
index 07b27d9bf..0441af2ac 100644
--- a/LICENSE
+++ b/client-flink/flink-1.17-shaded/src/main/resources/META-INF/LICENSE
@@ -202,12 +202,46 @@
------------------------------------------------------------------------------------
-This product bundles various third-party components under other open source
licenses.
-This section summarizes those components and their licenses. See licenses/
-for text of these licenses.
+This project bundles the following dependencies under the Apache License 2.0
(http://www.apache.org/licenses/LICENSE-2.0.txt):
-Apache Software Foundation License 2.0
+Apache License 2.0
--------------------------------------
-common/src/main/java/org/apache/celeborn/common/network/util/LimitedInputStream.java
+org.apache.commons:commons-lang3
+io.netty:netty-all
+io.netty:netty-buffer
+io.netty:netty-codec
+io.netty:netty-codec-dns
+io.netty:netty-codec-haproxy
+io.netty:netty-codec-http
+io.netty:netty-codec-http2
+io.netty:netty-codec-memcache
+io.netty:netty-codec-mqtt
+io.netty:netty-codec-redis
+io.netty:netty-codec-smtp
+io.netty:netty-codec-socks
+io.netty:netty-codec-stomp
+io.netty:netty-codec-xml
+io.netty:netty-common
+io.netty:netty-handler
+io.netty:netty-transport-native-unix-common
+io.netty:netty-handler-proxy
+io.netty:netty-resolver
+io.netty:netty-resolver-dns
+io.netty:netty-transport
+io.netty:netty-transport-rxtx
+io.netty:netty-transport-sctp
+io.netty:netty-transport-udt
+io.netty:netty-transport-classes-epoll
+io.netty:netty-transport-classes-kqueue
+io.netty:netty-transport-native-epoll
+io.netty:netty-transport-native-kqueue
+com.google.guava:guava
+org.roaringbitmap:RoaringBitmap
+
+
+BSD 3-clause
+------------
+See license/LICENSE-protobuf.txt for details.
+com.google.protobuf:protobuf-java
diff --git a/client-flink/flink-1.17-shaded/src/main/resources/META-INF/NOTICE
b/client-flink/flink-1.17-shaded/src/main/resources/META-INF/NOTICE
new file mode 100644
index 000000000..a4001a2ce
--- /dev/null
+++ b/client-flink/flink-1.17-shaded/src/main/resources/META-INF/NOTICE
@@ -0,0 +1,13 @@
+
+Apache Celeborn (Incubating)
+Copyright 2022-2023 The Apache Software Foundation.
+
+This product includes software developed at
+The Apache Software Foundation (https://www.apache.org/).
+
+=============================================================================
+= NOTICE file corresponding to section 4d of the Apache License Version 2.0 =
+=============================================================================
+
+Apache Commons Lang
+Copyright 2001-2021 The Apache Software Foundation
diff --git
a/client-flink/flink-1.17-shaded/src/main/resources/META-INF/licenses/LICENSE-protobuf.txt
b/client-flink/flink-1.17-shaded/src/main/resources/META-INF/licenses/LICENSE-protobuf.txt
new file mode 100644
index 000000000..b4350ec83
--- /dev/null
+++
b/client-flink/flink-1.17-shaded/src/main/resources/META-INF/licenses/LICENSE-protobuf.txt
@@ -0,0 +1,42 @@
+This license applies to all parts of Protocol Buffers except the following:
+
+ - Atomicops support for generic gcc, located in
+ src/google/protobuf/stubs/atomicops_internals_generic_gcc.h.
+ This file is copyrighted by Red Hat Inc.
+
+ - Atomicops support for AIX/POWER, located in
+ src/google/protobuf/stubs/atomicops_internals_aix.h.
+ This file is copyrighted by Bloomberg Finance LP.
+
+Copyright 2014, Google Inc. All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are
+met:
+
+ * Redistributions of source code must retain the above copyright
+notice, this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above
+copyright notice, this list of conditions and the following disclaimer
+in the documentation and/or other materials provided with the
+distribution.
+ * Neither the name of Google Inc. nor the names of its
+contributors may be used to endorse or promote products derived from
+this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+Code generated by the Protocol Buffer compiler is owned by the owner
+of the input file used when generating it. This code is not
+standalone and requires a support library to be linked with it. This
+support library is itself covered by the above license.
\ No newline at end of file
diff --git a/client-spark/spark-2-shaded/pom.xml
b/client-spark/spark-2-shaded/pom.xml
index ffded4760..111511f05 100644
--- a/client-spark/spark-2-shaded/pom.xml
+++ b/client-spark/spark-2-shaded/pom.xml
@@ -73,13 +73,24 @@
<filter>
<artifact>*:*</artifact>
<excludes>
+ <exclude>**/*.proto</exclude>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
<exclude>**/log4j.properties</exclude>
+<<<<<<< HEAD
+=======
+ <exclude>META-INF/LICENSE.txt</exclude>
+ <exclude>META-INF/NOTICE.txt</exclude>
+ <exclude>LICENSE.txt</exclude>
+ <exclude>NOTICE.txt</exclude>
+>>>>>>> 78553f14 ([CELEBORN-1003] Correct the LICENSE and NOTICE for shaded
client jars)
</excludes>
</filter>
</filters>
+ <transformers>
+ <transformer
implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"></transformer>
+ </transformers>
</configuration>
</plugin>
<plugin>
diff --git a/LICENSE
b/client-spark/spark-2-shaded/src/main/resources/META-INF/LICENSE
similarity index 90%
copy from LICENSE
copy to client-spark/spark-2-shaded/src/main/resources/META-INF/LICENSE
index 07b27d9bf..0441af2ac 100644
--- a/LICENSE
+++ b/client-spark/spark-2-shaded/src/main/resources/META-INF/LICENSE
@@ -202,12 +202,46 @@
------------------------------------------------------------------------------------
-This product bundles various third-party components under other open source
licenses.
-This section summarizes those components and their licenses. See licenses/
-for text of these licenses.
+This project bundles the following dependencies under the Apache License 2.0
(http://www.apache.org/licenses/LICENSE-2.0.txt):
-Apache Software Foundation License 2.0
+Apache License 2.0
--------------------------------------
-common/src/main/java/org/apache/celeborn/common/network/util/LimitedInputStream.java
+org.apache.commons:commons-lang3
+io.netty:netty-all
+io.netty:netty-buffer
+io.netty:netty-codec
+io.netty:netty-codec-dns
+io.netty:netty-codec-haproxy
+io.netty:netty-codec-http
+io.netty:netty-codec-http2
+io.netty:netty-codec-memcache
+io.netty:netty-codec-mqtt
+io.netty:netty-codec-redis
+io.netty:netty-codec-smtp
+io.netty:netty-codec-socks
+io.netty:netty-codec-stomp
+io.netty:netty-codec-xml
+io.netty:netty-common
+io.netty:netty-handler
+io.netty:netty-transport-native-unix-common
+io.netty:netty-handler-proxy
+io.netty:netty-resolver
+io.netty:netty-resolver-dns
+io.netty:netty-transport
+io.netty:netty-transport-rxtx
+io.netty:netty-transport-sctp
+io.netty:netty-transport-udt
+io.netty:netty-transport-classes-epoll
+io.netty:netty-transport-classes-kqueue
+io.netty:netty-transport-native-epoll
+io.netty:netty-transport-native-kqueue
+com.google.guava:guava
+org.roaringbitmap:RoaringBitmap
+
+
+BSD 3-clause
+------------
+See license/LICENSE-protobuf.txt for details.
+com.google.protobuf:protobuf-java
diff --git a/client-spark/spark-2-shaded/src/main/resources/META-INF/NOTICE
b/client-spark/spark-2-shaded/src/main/resources/META-INF/NOTICE
new file mode 100644
index 000000000..b362f8abd
--- /dev/null
+++ b/client-spark/spark-2-shaded/src/main/resources/META-INF/NOTICE
@@ -0,0 +1,17 @@
+
+Apache Celeborn (Incubating)
+Copyright 2022-2023 The Apache Software Foundation.
+
+This product includes software developed at
+The Apache Software Foundation (https://www.apache.org/).
+
+=============================================================================
+= NOTICE file corresponding to section 4d of the Apache License Version 2.0 =
+=============================================================================
+
+Apache Spark
+Copyright 2014 and onwards The Apache Software Foundation
+
+Apache Commons Lang
+Copyright 2001-2021 The Apache Software Foundation
+
diff --git a/client-spark/spark-3-shaded/pom.xml
b/client-spark/spark-3-shaded/pom.xml
index 1a989d2d4..ef6f4d840 100644
--- a/client-spark/spark-3-shaded/pom.xml
+++ b/client-spark/spark-3-shaded/pom.xml
@@ -78,9 +78,16 @@
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
<exclude>**/log4j.properties</exclude>
+ <exclude>META-INF/LICENSE.txt</exclude>
+ <exclude>META-INF/NOTICE.txt</exclude>
+ <exclude>LICENSE.txt</exclude>
+ <exclude>NOTICE.txt</exclude>
</excludes>
</filter>
</filters>
+ <transformers>
+ <transformer
implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"></transformer>
+ </transformers>
</configuration>
</plugin>
<plugin>
diff --git a/LICENSE
b/client-spark/spark-3-shaded/src/main/resources/META-INF/LICENSE
similarity index 90%
copy from LICENSE
copy to client-spark/spark-3-shaded/src/main/resources/META-INF/LICENSE
index 07b27d9bf..a8c58e532 100644
--- a/LICENSE
+++ b/client-spark/spark-3-shaded/src/main/resources/META-INF/LICENSE
@@ -202,12 +202,46 @@
------------------------------------------------------------------------------------
-This product bundles various third-party components under other open source
licenses.
-This section summarizes those components and their licenses. See licenses/
-for text of these licenses.
+This project bundles the following dependencies under the Apache License 2.0
(http://www.apache.org/licenses/LICENSE-2.0.txt):
-Apache Software Foundation License 2.0
+Apache License 2.0
--------------------------------------
-common/src/main/java/org/apache/celeborn/common/network/util/LimitedInputStream.java
+org.apache.commons:commons-lang3
+io.netty:netty-all
+io.netty:netty-buffer
+io.netty:netty-codec
+io.netty:netty-codec-dns
+io.netty:netty-codec-haproxy
+io.netty:netty-codec-http
+io.netty:netty-codec-http2
+io.netty:netty-codec-memcache
+io.netty:netty-codec-mqtt
+io.netty:netty-codec-redis
+io.netty:netty-codec-smtp
+io.netty:netty-codec-socks
+io.netty:netty-codec-stomp
+io.netty:netty-codec-xml
+io.netty:netty-common
+io.netty:netty-handler
+io.netty:netty-transport-native-unix-common
+io.netty:netty-handler-proxy
+io.netty:netty-resolver
+io.netty:netty-resolver-dns
+io.netty:netty-transport
+io.netty:netty-transport-rxtx
+io.netty:netty-transport-sctp
+io.netty:netty-transport-udt
+io.netty:netty-transport-classes-epoll
+io.netty:netty-transport-classes-kqueue
+io.netty:netty-transport-native-epoll
+io.netty:netty-transport-native-kqueue
+com.google.guava:guava
+org.roaringbitmap:RoaringBitmap
+
+
+BSD 3-clause
+------------
+See license/LICENSE-protobuf.txt for details.
+com.google.protobuf:protobuf-java
\ No newline at end of file
diff --git a/client-spark/spark-3-shaded/src/main/resources/META-INF/NOTICE
b/client-spark/spark-3-shaded/src/main/resources/META-INF/NOTICE
new file mode 100644
index 000000000..b362f8abd
--- /dev/null
+++ b/client-spark/spark-3-shaded/src/main/resources/META-INF/NOTICE
@@ -0,0 +1,17 @@
+
+Apache Celeborn (Incubating)
+Copyright 2022-2023 The Apache Software Foundation.
+
+This product includes software developed at
+The Apache Software Foundation (https://www.apache.org/).
+
+=============================================================================
+= NOTICE file corresponding to section 4d of the Apache License Version 2.0 =
+=============================================================================
+
+Apache Spark
+Copyright 2014 and onwards The Apache Software Foundation
+
+Apache Commons Lang
+Copyright 2001-2021 The Apache Software Foundation
+
diff --git a/licenses-binary/LICENSE-BSD-3.txt
b/licenses-binary/LICENSE-BSD-3.txt
deleted file mode 100644
index 5ed0e2846..000000000
--- a/licenses-binary/LICENSE-BSD-3.txt
+++ /dev/null
@@ -1,11 +0,0 @@
-The 3-Clause BSD License
-
-Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
-
-1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
-
-2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
-
-3. Neither the name of the copyright holder nor the names of its contributors
may be used to endorse or promote products derived from this software without
specific prior written permission.
-
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROF [...]
\ No newline at end of file
diff --git a/licenses-binary/LICENSE-MIT.txt b/licenses-binary/LICENSE-MIT.txt
deleted file mode 100644
index 63b4b681c..000000000
--- a/licenses-binary/LICENSE-MIT.txt
+++ /dev/null
@@ -1,21 +0,0 @@
-MIT License
-
-Copyright (c) [year] [fullname]
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
\ No newline at end of file
