Ottomata has submitted this change and it was merged. (
https://gerrit.wikimedia.org/r/390227 )
Change subject: Correct bug in ClickstreamBuilder
......................................................................
Correct bug in ClickstreamBuilder
Incorrect output path was leftover from manual testing.
Bug: T175844
Change-Id: Id9653b342bc324731864681e751b7dd67eda484e
---
M
refinery-job/src/main/scala/org/wikimedia/analytics/refinery/job/ClickstreamBuilder.scala
1 file changed, 2 insertions(+), 2 deletions(-)
Approvals:
Ottomata: Looks good to me, approved
jenkins-bot: Verified
diff --git
a/refinery-job/src/main/scala/org/wikimedia/analytics/refinery/job/ClickstreamBuilder.scala
b/refinery-job/src/main/scala/org/wikimedia/analytics/refinery/job/ClickstreamBuilder.scala
index 30af964..5052636 100644
---
a/refinery-job/src/main/scala/org/wikimedia/analytics/refinery/job/ClickstreamBuilder.scala
+++
b/refinery-job/src/main/scala/org/wikimedia/analytics/refinery/job/ClickstreamBuilder.scala
@@ -495,8 +495,8 @@
})
val projectList = domainList.map(_.stripSuffix(".org"))
- val outputFolder = "/user/joal/wmf/data/archive/clickstream/" +
- f"${params.year}%04d-${params.month}%02d${params.day.map(d =>
f"-$d%02d").getOrElse("")}${params.hour.map(h => f"-$h%02d").getOrElse("")}"
+ val outputFolder = params.outputBasePath +
+ f"/${params.year}%04d-${params.month}%02d${params.day.map(d =>
f"-$d%02d").getOrElse("")}${params.hour.map(h => f"-$h%02d").getOrElse("")}"
// Reused RDDs
val pages = preparePages(sqlContext, params.pageTable, params.snapshot,
params.wikiList).cache()
--
To view, visit https://gerrit.wikimedia.org/r/390227
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Id9653b342bc324731864681e751b7dd67eda484e
Gerrit-PatchSet: 1
Gerrit-Project: analytics/refinery/source
Gerrit-Branch: master
Gerrit-Owner: Joal <[email protected]>
Gerrit-Reviewer: Ottomata <[email protected]>
Gerrit-Reviewer: jenkins-bot <>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
