[
https://issues.apache.org/jira/browse/HADOOP-19236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17915947#comment-17915947
]
ASF GitHub Bot commented on HADOOP-19236:
-----------------------------------------
hadoop-yetus commented on PR #7294:
URL: https://github.com/apache/hadoop/pull/7294#issuecomment-2606513095
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 57s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 2s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 6m 30s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 31m 11s | | trunk passed |
| +1 :green_heart: | compile | 16m 49s | | trunk passed with JDK
Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 15m 39s | | trunk passed with JDK
Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga |
| +1 :green_heart: | checkstyle | 4m 12s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 3s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 4s | | trunk passed with JDK
Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 1s | | trunk passed with JDK
Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga |
| +0 :ok: | spotbugs | 0m 44s | | branch/hadoop-project no spotbugs
output file (spotbugsXml.xml) |
| +0 :ok: | spotbugs | 0m 39s | |
branch/hadoop-cloud-storage-project/hadoop-cloud-storage no spotbugs output
file (spotbugsXml.xml) |
| +1 :green_heart: | shadedclient | 34m 42s | | branch has no errors
when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 37s | | Maven dependency ordering for patch |
| -1 :x: | mvninstall | 0m 28s |
[/patch-mvninstall-hadoop-cloud-storage-project_hadoop-tos.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7294/3/artifact/out/patch-mvninstall-hadoop-cloud-storage-project_hadoop-tos.txt)
| hadoop-tos in the patch failed. |
| -1 :x: | mvninstall | 0m 28s |
[/patch-mvninstall-hadoop-cloud-storage-project.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7294/3/artifact/out/patch-mvninstall-hadoop-cloud-storage-project.txt)
| hadoop-cloud-storage-project in the patch failed. |
| -1 :x: | mvninstall | 0m 14s |
[/patch-mvninstall-hadoop-cloud-storage-project_hadoop-cloud-storage.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7294/3/artifact/out/patch-mvninstall-hadoop-cloud-storage-project_hadoop-cloud-storage.txt)
| hadoop-cloud-storage in the patch failed. |
| +1 :green_heart: | compile | 16m 22s | | the patch passed with JDK
Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 16m 22s | | the patch passed |
| +1 :green_heart: | compile | 15m 27s | | the patch passed with JDK
Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga |
| +1 :green_heart: | javac | 15m 27s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7294/3/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<<patch_file>>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 4m 5s |
[/buildtool-patch-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7294/3/artifact/out/buildtool-patch-checkstyle-root.txt)
| The patch fails to run checkstyle in root |
| -1 :x: | mvnsite | 0m 36s |
[/patch-mvnsite-hadoop-cloud-storage-project_hadoop-cloud-storage.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7294/3/artifact/out/patch-mvnsite-hadoop-cloud-storage-project_hadoop-cloud-storage.txt)
| hadoop-cloud-storage in the patch failed. |
| -1 :x: | javadoc | 0m 47s |
[/results-javadoc-javadoc-hadoop-cloud-storage-project_hadoop-tos-jdkUbuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7294/3/artifact/out/results-javadoc-javadoc-hadoop-cloud-storage-project_hadoop-tos-jdkUbuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.txt)
|
hadoop-cloud-storage-project_hadoop-tos-jdkUbuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04
with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04 generated 18 new + 0
unchanged - 0 fixed = 18 total (was 0) |
| -1 :x: | javadoc | 0m 57s |
[/patch-javadoc-hadoop-cloud-storage-project-jdkUbuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7294/3/artifact/out/patch-javadoc-hadoop-cloud-storage-project-jdkUbuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.txt)
| hadoop-cloud-storage-project in the patch failed with JDK
Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04. |
| -1 :x: | javadoc | 0m 37s |
[/patch-javadoc-hadoop-cloud-storage-project_hadoop-cloud-storage-jdkUbuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7294/3/artifact/out/patch-javadoc-hadoop-cloud-storage-project_hadoop-cloud-storage-jdkUbuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.txt)
| hadoop-cloud-storage in the patch failed with JDK
Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04. |
| -1 :x: | javadoc | 0m 44s |
[/results-javadoc-javadoc-hadoop-cloud-storage-project_hadoop-tos-jdkPrivateBuild-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7294/3/artifact/out/results-javadoc-javadoc-hadoop-cloud-storage-project_hadoop-tos-jdkPrivateBuild-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga.txt)
|
hadoop-cloud-storage-project_hadoop-tos-jdkPrivateBuild-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga
with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga generated 18
new + 0 unchanged - 0 fixed = 18 total (was 0) |
| -1 :x: | javadoc | 0m 47s |
[/patch-javadoc-hadoop-cloud-storage-project-jdkPrivateBuild-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7294/3/artifact/out/patch-javadoc-hadoop-cloud-storage-project-jdkPrivateBuild-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga.txt)
| hadoop-cloud-storage-project in the patch failed with JDK Private
Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga. |
| -1 :x: | javadoc | 0m 36s |
[/patch-javadoc-hadoop-cloud-storage-project_hadoop-cloud-storage-jdkPrivateBuild-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7294/3/artifact/out/patch-javadoc-hadoop-cloud-storage-project_hadoop-cloud-storage-jdkPrivateBuild-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga.txt)
| hadoop-cloud-storage in the patch failed with JDK Private
Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga. |
| +0 :ok: | spotbugs | 0m 38s | | hadoop-project has no data from
spotbugs |
| -1 :x: | spotbugs | 1m 36s |
[/new-spotbugs-hadoop-cloud-storage-project.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7294/3/artifact/out/new-spotbugs-hadoop-cloud-storage-project.html)
| hadoop-cloud-storage-project generated 24 new + 0 unchanged - 0 fixed = 24
total (was 0) |
| -1 :x: | spotbugs | 0m 37s |
[/patch-spotbugs-hadoop-cloud-storage-project_hadoop-cloud-storage.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7294/3/artifact/out/patch-spotbugs-hadoop-cloud-storage-project_hadoop-cloud-storage.txt)
| hadoop-cloud-storage in the patch failed. |
| +1 :green_heart: | shadedclient | 36m 17s | | patch has no errors
when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 37s | | hadoop-project in the patch
passed. |
| +1 :green_heart: | unit | 0m 47s | | hadoop-tos in the patch passed.
|
| +1 :green_heart: | unit | 0m 49s | | hadoop-cloud-storage-project in
the patch passed. |
| -1 :x: | unit | 0m 37s |
[/patch-unit-hadoop-cloud-storage-project_hadoop-cloud-storage.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7294/3/artifact/out/patch-unit-hadoop-cloud-storage-project_hadoop-cloud-storage.txt)
| hadoop-cloud-storage in the patch failed. |
| +1 :green_heart: | asflicense | 1m 2s | | The patch does not
generate ASF License warnings. |
| | | 213m 36s | | |
| Reason | Tests |
|-------:|:------|
| SpotBugs | module:hadoop-cloud-storage-project |
| | org.apache.hadoop.fs.tosfs.RawFileStatus.checksum() may expose
internal representation by returning RawFileStatus.checksum At
RawFileStatus.java:by returning RawFileStatus.checksum At
RawFileStatus.java:[line 49] |
| | new org.apache.hadoop.fs.tosfs.RawFileStatus(long, boolean, long,
long, Path, String, byte[]) may expose internal representation by storing an
externally mutable object into RawFileStatus.checksum At
RawFileStatus.java:Path, String, byte[]) may expose internal representation by
storing an externally mutable object into RawFileStatus.checksum At
RawFileStatus.java:[line 45] |
| | org.apache.hadoop.fs.tosfs.RawFileStatus doesn't override
org.apache.hadoop.fs.FileStatus.equals(Object) At RawFileStatus.java:At
RawFileStatus.java:[line 1] |
| | org.apache.hadoop.fs.tosfs.TosChecksum.getBytes() may expose internal
representation by returning TosChecksum.checksum At TosChecksum.java:by
returning TosChecksum.checksum At TosChecksum.java:[line 47] |
| | new org.apache.hadoop.fs.tosfs.TosChecksum(String, byte[]) may expose
internal representation by storing an externally mutable object into
TosChecksum.checksum At TosChecksum.java:internal representation by storing an
externally mutable object into TosChecksum.checksum At TosChecksum.java:[line
32] |
| | Unsigned right shift cast to short/byte in
org.apache.hadoop.fs.tosfs.common.Bytes.toBytes(short) At
Bytes.java:short/byte in org.apache.hadoop.fs.tosfs.common.Bytes.toBytes(short)
At Bytes.java:[line 41] |
| | org.apache.hadoop.fs.tosfs.common.Tasks$Range$1.next() can't throw
NoSuchElementException At Tasks.java:At Tasks.java:[lines 535-537] |
| |
org.apache.hadoop.fs.tosfs.conf.TosKeys.FS_TOS_BATCH_DELETE_RETRY_CODES_DEFAULT
is a mutable array At TosKeys.java: At TosKeys.java:[line 196] |
| |
org.apache.hadoop.fs.tosfs.conf.TosKeys.FS_TOS_CUSTOM_CREDENTIAL_PROVIDER_CLASSES_DEFAULT
is a mutable array At TosKeys.java: At TosKeys.java:[line 80] |
| | org.apache.hadoop.fs.tosfs.object.Constants.MAGIC_CHECKSUM is a
mutable array At Constants.java: At Constants.java:[line 29] |
| | Exceptional return value of java.io.File.mkdirs() ignored in
org.apache.hadoop.fs.tosfs.object.FileStore.initialize(Configuration, String)
At FileStore.java:ignored in
org.apache.hadoop.fs.tosfs.object.FileStore.initialize(Configuration, String)
At FileStore.java:[line 120] |
| | org.apache.hadoop.fs.tosfs.object.ObjectContent.checksum() may expose
internal representation by returning ObjectContent.checksum At
ObjectContent.java:by returning ObjectContent.checksum At
ObjectContent.java:[line 50] |
| | new org.apache.hadoop.fs.tosfs.object.ObjectContent(byte[],
InputStream) may expose internal representation by storing an externally
mutable object into ObjectContent.checksum At ObjectContent.java:internal
representation by storing an externally mutable object into
ObjectContent.checksum At ObjectContent.java:[line 32] |
| | org.apache.hadoop.fs.tosfs.object.ObjectInfo.checksum() may expose
internal representation by returning ObjectInfo.checksum At ObjectInfo.java:by
returning ObjectInfo.checksum At ObjectInfo.java:[line 75] |
| | org.apache.hadoop.fs.tosfs.object.ObjectInfo.mtime() may expose
internal representation by returning ObjectInfo.mtime At ObjectInfo.java:by
returning ObjectInfo.mtime At ObjectInfo.java:[line 67] |
| | new org.apache.hadoop.fs.tosfs.object.ObjectInfo(String, long, Date,
byte[], boolean) may expose internal representation by storing an externally
mutable object into ObjectInfo.mtime At ObjectInfo.java:boolean) may expose
internal representation by storing an externally mutable object into
ObjectInfo.mtime At ObjectInfo.java:[line 47] |
| | new
org.apache.hadoop.fs.tosfs.object.ObjectMultiRangeInputStream(ExecutorService,
ObjectStorage, String, long, long, byte[]) may expose internal representation
by storing an externally mutable object into
ObjectMultiRangeInputStream.checksum At ObjectMultiRangeInputStream.java:long,
byte[]) may expose internal representation by storing an externally mutable
object into ObjectMultiRangeInputStream.checksum At
ObjectMultiRangeInputStream.java:[line 73] |
| | Inconsistent synchronization of
org.apache.hadoop.fs.tosfs.object.ObjectOutputStream.multipartUpload; locked
57% of time Unsynchronized access at ObjectOutputStream.java:57% of time
Unsynchronized access at ObjectOutputStream.java:[line 301] |
| | Possible null pointer dereference of dirs in
org.apache.hadoop.fs.tosfs.object.ObjectOutputStream.createStagingDirs(Configuration,
String) Dereferenced at ObjectOutputStream.java:dirs in
org.apache.hadoop.fs.tosfs.object.ObjectOutputStream.createStagingDirs(Configuration,
String) Dereferenced at ObjectOutputStream.java:[line 97] |
| | new
org.apache.hadoop.fs.tosfs.object.ObjectRangeInputStream(ObjectStorage, String,
Range, byte[]) may expose internal representation by storing an externally
mutable object into ObjectRangeInputStream.checksum At
ObjectRangeInputStream.java:may expose internal representation by storing an
externally mutable object into ObjectRangeInputStream.checksum At
ObjectRangeInputStream.java:[line 58] |
| | org.apache.hadoop.fs.tosfs.object.tos.ChainTOSInputStream.checksum()
may expose internal representation by returning ChainTOSInputStream.objChecksum
At ChainTOSInputStream.java:by returning ChainTOSInputStream.objChecksum At
ChainTOSInputStream.java:[line 135] |
| | org.apache.hadoop.fs.tosfs.object.tos.GetObjectOutput.checksum() may
expose internal representation by returning GetObjectOutput.checksum At
GetObjectOutput.java:by returning GetObjectOutput.checksum At
GetObjectOutput.java:[line 45] |
| | new
org.apache.hadoop.fs.tosfs.object.tos.GetObjectOutput(GetObjectV2Output,
byte[]) may expose internal representation by storing an externally mutable
object into GetObjectOutput.checksum At GetObjectOutput.java:internal
representation by storing an externally mutable object into
GetObjectOutput.checksum At GetObjectOutput.java:[line 37] |
| |
org.apache.hadoop.fs.tosfs.object.tos.TOSErrorCodes.FAST_FAILURE_CONFLICT_ERROR_CODES
isn't final but should be At TOSErrorCodes.java:be At
TOSErrorCodes.java:[line 52] |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.47 ServerAPI=1.47 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7294/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/7294 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
markdownlint |
| uname | Linux 5986c688eed4 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18
17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / a62d2ae1f9ee39e4331534ae5e84f2f2aba369e5 |
| Default Java | Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7294/3/testReport/ |
| Max. process+thread count | 546 (vs. ulimit of 5500) |
| modules | C: hadoop-project hadoop-cloud-storage-project/hadoop-tos
hadoop-cloud-storage-project hadoop-cloud-storage-project/hadoop-cloud-storage
U: . |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7294/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> Integration of Volcano Engine TOS in Hadoop.
> --------------------------------------------
>
> Key: HADOOP-19236
> URL: https://issues.apache.org/jira/browse/HADOOP-19236
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs, tools
> Affects Versions: 3.4.0
> Reporter: Jinglun
> Assignee: Zheng Hu
> Priority: Major
> Labels: pull-request-available
> Attachments: Integration of Volcano Engine TOS in Hadoop.pdf
>
>
> Volcano Engine is a fast growing cloud vendor launched by ByteDance, and TOS
> is the object storage service of Volcano Engine. A common way is to store
> data into TOS and run Hadoop/Spark/Flink applications to access TOS. But
> there is no original support for TOS in hadoop, thus it is not easy for users
> to build their Big Data System based on TOS.
>
> This work aims to integrate TOS with Hadoop to help users run their
> applications on TOS. Users only need to do some simple configuration, then
> their applications can read/write TOS without any code change. This work is
> similar to AWS S3, AzureBlob, AliyunOSS, Tencnet COS and HuaweiCloud Object
> Storage in Hadoop.
>
> Please see the attached document "Integration of Volcano Engine TOS in
> Hadoop" for more details.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
