Ottomata has submitted this change and it was merged. Change subject: Importing eventlogging via Camus into HDFS ......................................................................
Importing eventlogging via Camus into HDFS I've renamed camus.webrequest_properites -> camus.properties, as it now includes eventlogging data, which is not specific to webrequests. Change-Id: I3c39fe1db69430a44739384d784b0bf8b1bf2232 --- R kraken-etl/conf/camus.properties 1 file changed, 4 insertions(+), 4 deletions(-) Approvals: Ottomata: Verified; Looks good to me, approved diff --git a/kraken-etl/conf/camus.webrequest.properties b/kraken-etl/conf/camus.properties similarity index 95% rename from kraken-etl/conf/camus.webrequest.properties rename to kraken-etl/conf/camus.properties index 2091c63..02e2e4e 100644 --- a/kraken-etl/conf/camus.webrequest.properties +++ b/kraken-etl/conf/camus.properties @@ -31,9 +31,9 @@ etl.record.writer.provider.class=com.linkedin.camus.etl.kafka.common.SequenceFileRecordWriterProvider # Max hadoop tasks to use, each task can pull multiple topic partitions. -# Currently importing mobile and bits, each with 10 partitions. Setting -# map.tasks to 30. -mapred.map.tasks=30 +# Currently importing mobile, bits, and eventlogging each with 10 partitions. Setting +# map.tasks to 40. +mapred.map.tasks=40 # Connection parameters. kafka.brokers=analytics1021.eqiad.wmnet:9092,analytics1022.eqiad.wmnet:9092 @@ -56,7 +56,7 @@ kafka.blacklist.topics= # For now, we are only importing webrequest_mobile logs. -kafka.whitelist.topics=webrequest_mobile,webrequest_bits,webrequest_text +kafka.whitelist.topics=webrequest_mobile,webrequest_bits,webrequest_text,eventlogging-00 # Name of the client as seen by kafka kafka.client.name=camus-webrequest-01 -- To view, visit https://gerrit.wikimedia.org/r/130209 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I3c39fe1db69430a44739384d784b0bf8b1bf2232 Gerrit-PatchSet: 1 Gerrit-Project: analytics/kraken Gerrit-Branch: master Gerrit-Owner: Ottomata <[email protected]> Gerrit-Reviewer: Ottomata <[email protected]> _______________________________________________ MediaWiki-commits mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
