[
https://issues.apache.org/jira/browse/EAGLE-492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15435888#comment-15435888
]
ASF GitHub Bot commented on EAGLE-492:
--------------------------------------
Github user DadanielZ commented on the issue:
https://github.com/apache/incubator-eagle/pull/375
Patch apply failed, please fix it first.
error: patch failed:
eagle-jpm/eagle-jpm-entity/src/main/java/org/apache/eagle/jpm/spark/crawl/JHFSparkEventReader.java:21
error:
eagle-jpm/eagle-jpm-entity/src/main/java/org/apache/eagle/jpm/spark/crawl/JHFSparkEventReader.java:
patch does not apply
error: patch failed:
eagle-jpm/eagle-jpm-entity/src/main/java/org/apache/eagle/jpm/spark/crawl/JHFSparkParser.java:31
error:
eagle-jpm/eagle-jpm-entity/src/main/java/org/apache/eagle/jpm/spark/crawl/JHFSparkParser.java:
patch does not apply
error: patch failed:
eagle-jpm/eagle-jpm-entity/src/main/java/org/apache/eagle/jpm/spark/entity/SparkStage.java:36
error:
eagle-jpm/eagle-jpm-entity/src/main/java/org/apache/eagle/jpm/spark/entity/SparkStage.java:
patch does not apply
error: patch failed:
eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/Utils.java:68
error:
eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/Utils.java:
patch does not apply
error: patch failed:
eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/RMResourceFetcher.java:128
error:
eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/RMResourceFetcher.java:
patch does not apply
error: patch failed:
eagle-jpm/eagle-jpm-entity/src/main/java/org/apache/eagle/jpm/spark/crawl/JHFSparkParser.java:34
error:
eagle-jpm/eagle-jpm-entity/src/main/java/org/apache/eagle/jpm/spark/crawl/JHFSparkParser.java:
patch does not apply
error: patch failed:
eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/RMResourceFetcher.java:107
error:
eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/RMResourceFetcher.java:
patch does not apply
Das-MacBook-Pro:incubator-eagle dazhou$ git apply -v 375.patch
Checking patch
eagle-jpm/eagle-jpm-entity/src/main/java/org/apache/eagle/jpm/spark/crawl/JHFSparkEventReader.java...
error: while searching for:
import com.typesafe.config.ConfigFactory;
import org.apache.eagle.jpm.spark.entity.JobConfig;
import org.apache.eagle.jpm.spark.entity.*;
import org.apache.eagle.jpm.util.JSONUtil;
import org.apache.eagle.jpm.util.JobNameNormalization;
import org.apache.eagle.jpm.util.SparkEntityConstant;
import org.apache.eagle.jpm.util.SparkJobTagName;
import org.apache.eagle.log.base.taggedlog.TaggedLogAPIEntity;
import org.apache.eagle.service.client.impl.EagleServiceBaseClient;
import org.apache.eagle.service.client.impl.EagleServiceClientImpl;
error: patch failed:
eagle-jpm/eagle-jpm-entity/src/main/java/org/apache/eagle/jpm/spark/crawl/JHFSparkEventReader.java:21
error:
eagle-jpm/eagle-jpm-entity/src/main/java/org/apache/eagle/jpm/spark/crawl/JHFSparkEventReader.java:
patch does not apply
Checking patch
eagle-jpm/eagle-jpm-entity/src/main/java/org/apache/eagle/jpm/spark/crawl/JHFSparkParser.java...
error: while searching for:
private static final Logger logger =
LoggerFactory.getLogger(JHFSparkParser.class);
JHFSparkEventReader eventReader;
public JHFSparkParser(JHFSparkEventReader reader){
error: patch failed:
eagle-jpm/eagle-jpm-entity/src/main/java/org/apache/eagle/jpm/spark/crawl/JHFSparkParser.java:31
error:
eagle-jpm/eagle-jpm-entity/src/main/java/org/apache/eagle/jpm/spark/crawl/JHFSparkParser.java:
patch does not apply
Checking patch
eagle-jpm/eagle-jpm-entity/src/main/java/org/apache/eagle/jpm/spark/entity/SparkStage.java...
error: while searching for:
@Column("a")
private String status;
@Column("b")
private int numActiveTasks=0;
@Column("c")
private int numCompletedTasks=0;
@Column("d")
private int numFailedTasks=0;
@Column("e")
private long executorRunTime=0l;
@Column("f")
private long inputBytes=0l;
@Column("g")
private long inputRecords=0l;
@Column("h")
private long outputBytes=0l;
@Column("i")
private long outputRecords=0l;
@Column("j")
private long shuffleReadBytes=0l;
@Column("k")
private long shuffleReadRecords=0l;
@Column("l")
private long shuffleWriteBytes=0l;
@Column("m")
private long shuffleWriteRecords=0l;
@Column("n")
private long memoryBytesSpilled=0l;
@Column("o")
private long diskBytesSpilled=0l;
@Column("p")
private String name;
@Column("q")
error: patch failed:
eagle-jpm/eagle-jpm-entity/src/main/java/org/apache/eagle/jpm/spark/entity/SparkStage.java:36
error:
eagle-jpm/eagle-jpm-entity/src/main/java/org/apache/eagle/jpm/spark/entity/SparkStage.java:
patch does not apply
Checking patch
eagle-jpm/eagle-jpm-spark-history/src/main/java/org/apache/eagle/jpm/spark/history/storm/FinishedSparkJobSpout.java...
Hunk #1 succeeded at 76 (offset -2 lines).
Checking patch
eagle-jpm/eagle-jpm-spark-history/src/main/java/org/apache/eagle/jpm/spark/history/storm/SparkJobParseBolt.java...
Checking patch
eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/Utils.java...
error: while searching for:
public static long parseMemory(String memory) {
if (memory.endsWith("g") || memory.endsWith("G")) {
int executorGB = Integer.parseInt(memory.substring(0,
memory.length() - 1));
return 1024l * 1024 * 1024 * executorGB;
} else if (memory.endsWith("m") || memory.endsWith("M")) {
int executorMB = Integer.parseInt(memory.substring(0,
memory.length() - 1));
return 1024l * 1024 * executorMB;
} else if (memory.endsWith("k") || memory.endsWith("K")) {
int executorKB = Integer.parseInt(memory.substring(0,
memory.length() - 1));
return 1024l * executorKB;
} else if (memory.endsWith("t") || memory.endsWith("T")) {
int executorTB = Integer.parseInt(memory.substring(0,
memory.length() - 1));
return 1024l * 1024 * 1024 * 1024 * executorTB;
} else if (memory.endsWith("p") || memory.endsWith("P")) {
int executorPB = Integer.parseInt(memory.substring(0,
memory.length() - 1));
return 1024l * 1024 * 1024 * 1024 * 1024 * executorPB;
}
LOG.info("Cannot parse memory info " + memory);
return 0l;
}
}
error: patch failed:
eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/Utils.java:68
error:
eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/Utils.java:
patch does not apply
Checking patch
eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/RMResourceFetcher.java...
error: while searching for:
}
}
private List<AppInfo> getResource(Constants.ResourceType resoureType,
Constants.CompressionType compressionType, Object... parameter) throws
Exception {
switch (resoureType) {
case COMPLETE_SPARK_JOB:
final String urlString =
sparkCompleteJobServiceURLBuilder.build((String) parameter[0]);
return doFetchFinishApplicationsList(urlString,
compressionType);
case RUNNING_SPARK_JOB:
return
doFetchRunningApplicationsList(getSparkRunningJobURL(), compressionType);
error: patch failed:
eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/RMResourceFetcher.java:128
error:
eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/RMResourceFetcher.java:
patch does not apply
Checking patch
eagle-jpm/eagle-jpm-entity/src/main/java/org/apache/eagle/jpm/spark/crawl/JHFSparkParser.java...
error: while searching for:
private boolean isValidJson;
JHFSparkEventReader eventReader;
public JHFSparkParser(JHFSparkEventReader reader) {
this.eventReader = reader;
error: patch failed:
eagle-jpm/eagle-jpm-entity/src/main/java/org/apache/eagle/jpm/spark/crawl/JHFSparkParser.java:34
error:
eagle-jpm/eagle-jpm-entity/src/main/java/org/apache/eagle/jpm/spark/crawl/JHFSparkParser.java:
patch does not apply
Checking patch
eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/RMResourceFetcher.java...
error: while searching for:
private String getMRFinishedJobURL(String lastFinishedTime) {
String url = URLUtil.removeTrailingSlash(selector.getSelectedUrl());
StringBuilder sb = new StringBuilder();
sb.append(url).append("/").append(Constants.V2_APPS_URL);
sb.append("?applicationTypes=MAPREDUCE&state=FINISHED&finishedTimeBegin=");
sb.append(lastFinishedTime).append("&").append(Constants.ANONYMOUS_PARAMETER);
return sb.toString();
}
private List<AppInfo> doFetchRunningApplicationsList(String urlString,
Constants.CompressionType compressionType) throws Exception {
error: patch failed:
eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/RMResourceFetcher.java:107
error:
eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/RMResourceFetcher.java:
patch does not apply
Checking patch
eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/ha/HAURLSelectorImpl.java...
Checking patch
eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/url/SparkCompleteJobServiceURLBuilderImpl.java...
> Fix negative total execution time and make some code optimization for spark
> history job.
> ----------------------------------------------------------------------------------------
>
> Key: EAGLE-492
> URL: https://issues.apache.org/jira/browse/EAGLE-492
> Project: Eagle
> Issue Type: Bug
> Affects Versions: v0.5.0
> Reporter: Huizhi Lu
> Assignee: Huizhi Lu
> Fix For: v0.5.0
>
>
> 1) negative total execution time
> It is because the app was killed or terminated unexpectedly without
> completion. In the log file, it has only SparkListenerBlockManagerAdded
> timestamp. "Timestamp":1470768362299
> And the code initiated executor starttime with this timestamp.
> No executor or app endtime available, so executor endtime = 0L.
> Finally, totalExecutionTime = -1470768362299
> The code logic does not consider the situation when executor or app Endtime =
> 0L.
> Solution: If totalExecutionTime < 0L, set it = 0L. Stating the app is killed
> and not completed.
> 2) Url builder bug for completed jobs
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
