[
https://issues.apache.org/jira/browse/TEZ-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16883936#comment-16883936
]
TezQA commented on TEZ-4075:
----------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 8m
43s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 12s{color} | {color:orange} tez-runtime-library: The patch generated 11 new
+ 28 unchanged - 0 fixed = 39 total (was 28) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
24s{color} | {color:green} tez-runtime-library in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=18.09.7 Server=18.09.7 Image:yetus/tez:d4a62de |
| JIRA Issue | TEZ-4075 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12974510/Tez-4075.3.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
checkstyle compile cc |
| uname | Linux bb8d1ccb1410 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | master / 4125639 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_212 |
| findbugs | v3.0.1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-TEZ-Build/148/artifact/out/diff-checkstyle-tez-runtime-library.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-TEZ-Build/148/testReport/ |
| Max. process+thread count | 155 (vs. ulimit of 10000) |
| modules | C: tez-runtime-library U: tez-runtime-library |
| Console output |
https://builds.apache.org/job/PreCommit-TEZ-Build/148/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Tez: Reimplement tez.runtime.transfer.data-via-events.enabled
> -------------------------------------------------------------
>
> Key: TEZ-4075
> URL: https://issues.apache.org/jira/browse/TEZ-4075
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Gopal V
> Priority: Major
> Attachments: Tez-4075.3.patch
>
>
> This was factored out by TEZ-2196, which does skip buffers for 1-partition
> data exchanges (therefore goes to disk directly).
> {code}
> if (shufflePayload.hasData()) {
> shuffleManager.addKnownInput(shufflePayload.getHost(),
> DataProto dataProto = shufflePayload.getData();
> shufflePayload.getPort(), srcAttemptIdentifier, srcIndex);
> FetchedInput fetchedInput =
> inputAllocator.allocate(dataProto.getRawLength(),
> dataProto.getCompressedLength(), srcAttemptIdentifier);
> moveDataToFetchedInput(dataProto, fetchedInput, hostIdentifier);
> shuffleManager.addCompletedInputWithData(srcAttemptIdentifier,
> fetchedInput);
> } else {
> shuffleManager.addKnownInput(shufflePayload.getHost(),
> shufflePayload.getPort(), srcAttemptIdentifier, srcIndex);
> }
> {code}
> got removed in
> https://github.com/apache/tez/commit/1ba1f927c16a1d7c273b6cd1a8553e5269d1541a
> It would be better to buffer up the 512Byte limit for the event size before
> writing to disk, since creating a new file always incurs disk traffic, even
> if the file is eventually being served out of the buffer cache.
> The total overhead of receiving an event, then firing an HTTP call to fetch
> the data etc adds approx 100-150ms to a query - the data xfer through the
> event will skip the disk entirely for this & also remove the extra IOPS
> incurred.
> This channel is not suitable for large-scale event transport, but
> specifically the workload here deals with 1-row control tables which consume
> more bandwidth with HTTP headers and hostnames than the 93 byte payload.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
