Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/232177
Change subject: Correct bug in pageview definition
..
Correct bug in pageview definition
prevent outreach.wikimedia.org and donate.wikipedia.org
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/232448
Change subject: Add CassandraXSVLoader to refinery-job
..
Add CassandraXSVLoader to refinery-job
The CassandraXSVLoader runs a map-reduce job in
Joal has submitted this change and it was merged.
Change subject: Upgrade camus to version wmf6
..
Upgrade camus to version wmf6
Bug: T109860
Change-Id: I7ac7f08b898f186115e087879b41896bc9235dff
---
M artifacts/camus-wmf.jar
D
Joal has submitted this change and it was merged.
Change subject: [WIP] Generate hourly aggregate statistics about webrequest
sequence stats
..
[WIP] Generate hourly aggregate statistics about webrequest sequence stats
This is
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/228010
Change subject: Add user_agent_map to intermediate pageview
..
Add user_agent_map to intermediate pageview
user_agent_map field added to table c
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/228011
Change subject: Remove empty projects from intermediate pageviews
..
Remove empty projects from intermediate pageviews
Change-Id: I5d24a669e3c59
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/231010
Change subject: Remove webrequest bits and add webrequest maps
..
Remove webrequest bits and add webrequest maps
Remove webrequest bits data (ca
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/231042
Change subject: Update changelog.md before deployment.
..
Update changelog.md before deployment.
Change-Id: I6583c2c5967554f1472028823275e0de458
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/231241
Change subject: Bump refinery-core and refinery-hive to 0.0.15
..
Bump refinery-core and refinery-hive to 0.0.15
Add jar files.
Bump version in
Joal has submitted this change and it was merged.
Change subject: Remove webrequest bits and add webrequest maps
..
Remove webrequest bits and add webrequest maps
Remove webrequest bits data (camus, partition loading, refinemen
Joal has submitted this change and it was merged.
Change subject: Bump refinery-core and refinery-hive to 0.0.15
..
Bump refinery-core and refinery-hive to 0.0.15
Add jar files.
Bump version in oozie refine job, as well as reco
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/231285
Change subject: Update dataset check dump script
..
Update dataset check dump script
Remove webrequest bits and legacy_tsv_5xx_bits
Add webreque
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/231545
Change subject: Correct pageview title normalization bug
..
Correct pageview title normalization bug
Pageview title normalization had a bug leav
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/202914
Change subject: Add access_method, client_type and is_zero fields to refined
webrequest table.
..
Add access_method, client_type and is_zero fie
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/216526
Change subject: Modify oozie lancher default parameters
..
Modify oozie lancher default parameters
Default parameters for oozie launcher were de
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/216930
Change subject: Modify oozie parameters at workflow level
..
Modify oozie parameters at workflow level
Remove queue_name for mark_data_set_done
Joal has submitted this change and it was merged.
Change subject: Modify oozie parameters at workflow level
..
Modify oozie parameters at workflow level
Remove queue_name for mark_data_set_done subworkflow (not a map-recude job
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/217226
Change subject: Correct UAParser not to fail for malformed wmf_app
..
Correct UAParser not to fail for malformed wmf_app
Previous commit had a f
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/217229
Change subject: Add 2 fields and 1 update to refine table
..
Add 2 fields and 1 update to refine table
Add pageview_info and normalized_host fie
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/217232
Change subject: Add oozie job for pageview_hourly aggregation
..
Add oozie job for pageview_hourly aggregation
Add hive creation table file.
Add
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/217322
Change subject: Rename project dialect to language_variant
..
Rename project dialect to language_variant
Change-Id: Id34da7bde5fd9ef69e5cf3682b3
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/236027
Change subject: Correct bug in refine oozie job definition
..
Correct bug in refine oozie job definition
Change-Id: I79a6b792eae7b499bb9a125003e
Joal has submitted this change and it was merged.
Change subject: Report RESTBase traffic metrics to Graphite
..
Report RESTBase traffic metrics to Graphite
This Spark Job aims to Run hourly and report restbase request counts
t
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/236224
Change subject: [WIP] Add cassandra load job for pageview API
..
[WIP] Add cassandra load job for pageview API
Hourly, daily and monthly data ag
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/236565
Change subject: Update changelog.md before deployment.
..
Update changelog.md before deployment.
Change-Id: I00314d2d7870549ba0b1ffa58e2234beb7d
Joal has submitted this change and it was merged.
Change subject: Add oozie job to schedule restbase metrics generation job
..
Add oozie job to schedule restbase metrics generation job
See also T109547 - ticket to track the Spa
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/236569
Change subject: Bump core and job jar versions to 0.0.18
..
Bump core and job jar versions to 0.0.18
Add refinery-core-0.0.18.jar and refinery-j
Joal has submitted this change and it was merged.
Change subject: Add aggregation across projects
..
Add aggregation across projects
With this change, if the flag --all-projects is used,
a new file 'all.csv' will be created for
Joal has submitted this change and it was merged.
Change subject: Add clarifying comment on --all-projects behavior
..
Add clarifying comment on --all-projects behavior
Bug: T95339
Change-Id: Ib4ef2a7d1910bc6a2ba0575de25c9f3222
Joal has submitted this change and it was merged.
Change subject: Add overall aggregate to project and language list
..
Add overall aggregate to project and language list
Bug: T95340
Change-Id: I29d83f4e9aaff6db33e12052ddcd82f6
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/233937
Change subject: Add json revisions sorted per page job
..
Add json revisions sorted per page job
Add new pom.xml and save previous as pom-hadoop
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/234351
Change subject: Amend changelog before deployment of 0.0.16
..
Amend changelog before deployment of 0.0.16
Change-Id: I4b4283f6a8110d53c86973ff3
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/234422
Change subject: Bump jars to 0.0.16 update refine record version
..
Bump jars to 0.0.16 update refine record version
Add refinery-core-0.0.16.ja
Joal has submitted this change and it was merged.
Change subject: Exclude the arbitration committee wikis from the pageviews
counts
..
Exclude the arbitration committee wikis from the pageviews counts
Arbcom maintains various
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/234983
Change subject: Update pageview_hourly oozie job for backfill
..
Update pageview_hourly oozie job for backfill
Reorder fields in pageview_hourly
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/234984
Change subject: Update changelog for v0.0.17 release.
..
Update changelog for v0.0.17 release.
Change-Id: Idebca852a9beecc187453ecf1dba94507e9ab
Joal has submitted this change and it was merged.
Change subject: Update changelog for v0.0.17 release.
..
Update changelog for v0.0.17 release.
Change-Id: Idebca852a9beecc187453ecf1dba94507e9abf3a
---
M changelog.md
1 file cha
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/234992
Change subject: Bump refinery core and hive version to 0.0.17
..
Bump refinery core and hive version to 0.0.17
Add jar files and update links.
M
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/222127
Change subject: Correct error in wikimetrics projectview simlink
..
Correct error in wikimetrics projectview simlink
Link was set to /srv/aggreg
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/244465
Change subject: Update bot regexp to catch http UA
..
Update bot regexp to catch http UA
Change-Id: Ib64061f2eb48c174d084219c627d54bc7dc00896
--
Joal has submitted this change and it was merged.
Change subject: Add refinery-camus module
..
Add refinery-camus module
In order to add analytics specific code without changing the existing
upstream camus repo, we are adding a
Joal has submitted this change and it was merged.
Change subject: Fix inconsistent mobile uniques reports due to partial job runs
..
Fix inconsistent mobile uniques reports due to partial job runs
Bug: T114406
Change-Id: I0fb8a
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/249368
Change subject: Correct CamusPartitionChecker bug
..
Correct CamusPartitionChecker bug
CamusPartitionChecker was flagging currently importing ho
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/249373
Change subject: Update oozie load job to use _IMPORTED flag
..
Update oozie load job to use _IMPORTED flag
Change-Id: I0b8bb9d40cc0d70836da87ad0
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/249378
Change subject: Correct pageview to dumps synchro
..
Correct pageview to dumps synchro
The path given for synchronization was not correct.
Chan
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/249407
Change subject: Update changelog.md for v0.0.22
..
Update changelog.md for v0.0.22
Change-Id: I167acaca897352d007a9ee3cf5e969337a0b3a6a
---
M ch
Joal has submitted this change and it was merged.
Change subject: Upgrade refinery-job.jar to v0.0.22
..
Upgrade refinery-job.jar to v0.0.22
Change-Id: If3580b0f988b1ff62ed919f204a393edc85ff2ed
---
A artifacts/org/wikimedia/ana
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/249425
Change subject: Upgrade refinery-job.jar to v0.0.22
..
Upgrade refinery-job.jar to v0.0.22
Change-Id: If3580b0f988b1ff62ed919f204a393edc85ff2ed
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/249698
Change subject: Compress output of pageview legacy files (gzip)
..
Compress output of pageview legacy files (gzip)
Change-Id: I90ef510d1a9eb03d7
Joal has submitted this change and it was merged.
Change subject: Compress output of pageview legacy files (gzip)
..
Compress output of pageview legacy files (gzip)
Change-Id: I90ef510d1a9eb03d7b64c3b02112facca8e1b040
---
M ooz
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/249715
Change subject: Correct bug in gzipped legacy pageview filename
..
Correct bug in gzipped legacy pageview filename
Change-Id: I1e08171c553e4d724
Joal has submitted this change and it was merged.
Change subject: Correct bug in gzipped legacy pageview filename
..
Correct bug in gzipped legacy pageview filename
Change-Id: I1e08171c553e4d7243eac9dfeadb9bf25ee38764
---
M ooz
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/228247
Change subject: Remove empty projects from intermediate pageviews
..
Remove empty projects from intermediate pageviews
Change-Id: I54b117c3240ed
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/229369
Change subject: Correct Pageview host regexps and tests
..
Correct Pageview host regexps and tests
Pageview host regexps were incorrect, as well
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/229372
Change subject: Prevent failure in case of wrong line format
..
Prevent failure in case of wrong line format
Instead of raising a RuntimeError,
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/240868
Change subject: [WIP] Add camus offsets reader to refinery-core
..
[WIP] Add camus offsets reader to refinery-core
Change-Id: I7e93a1f857d76dc63
Joal has submitted this change and it was merged.
Change subject: Include avro schema in refinery-camus jar
..
Include avro schema in refinery-camus jar
Bug: T117885
Change-Id: I0f47b586f380f10d8c68b5dd0f45fb5a17aae974
---
M re
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/251311
Change subject: Improve sorted-json job
..
Improve sorted-json job
Revamp the InputFormat to be parameterized by WikiMediaObjectFactory
Add para
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/251497
Change subject: Update mediawiki_CirrusSearchREquestSet camus cron
..
Update mediawiki_CirrusSearchREquestSet camus cron
Add --check parameter t
Joal has submitted this change and it was merged.
Change subject: Archive hourly pageviews in legacy format
..
Archive hourly pageviews in legacy format
Legacy format, also called webstatcollector format in the past, is
used by
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/247847
Change subject: Correct camus-partition-checker to use hdfs conf
..
Correct camus-partition-checker to use hdfs conf
Change-Id: I3e4ada59211d1d3
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/248041
Change subject: Update oozie diagram to reflect current status
..
Update oozie diagram to reflect current status
Change-Id: Ie026b1d750dc39928d6
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/248048
Change subject: Update bin/camus to include CamusPartitionChecker
..
Update bin/camus to include CamusPartitionChecker
Bug:T113252
Change-Id: Ia
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/248991
Change subject: Correct two bugs in refinery oozie jobs
..
Correct two bugs in refinery oozie jobs
send_error_email workflow file was incorrectl
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/249088
Change subject: Correct bug in parameter definition
..
Correct bug in parameter definition
output-projectviews parameter was not in the parsed p
Joal has submitted this change and it was merged.
Change subject: Correct bug in parameter definition
..
Correct bug in parameter definition
output-projectviews parameter was not in the parsed parameters list, generating
an er
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/249100
Change subject: Update camus runs
..
Update camus runs
Add run parameter to match script update
Add check parameter to webrequest job
Bug: T113
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/249140
Change subject: Update changelog.md preparing for release 0.0.21
..
Update changelog.md preparing for release 0.0.21
Change-Id: I36fb983b09b1132
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/249180
Change subject: Upgrade refinery-camus and refinery-job to 0.0.21
..
Upgrade refinery-camus and refinery-job to 0.0.21
Change-Id: I80fbec7013718
Joal has submitted this change and it was merged.
Change subject: Add oozie job to compute browser usage reports
..
Add oozie job to compute browser usage reports
It computes 2 reports: mobile-web and desktop+mobile-web.
As bot
Joal has submitted this change and it was merged.
Change subject: Add percent loss to refinery-dump-status script
..
Add percent loss to refinery-dump-status script
Adds --percent-loss argument. When used, instead of showing an
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/237392
Change subject: Update bot filtering for webrequests.
..
Update bot filtering for webrequests.
Rename is_crawler to isSpider to more coherent wi
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/237419
Change subject: [WIP] Update agent_type in webrequest
..
[WIP] Update agent_type in webrequest
Use new function isSpider.
Add a new agent_type '
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/238139
Change subject: Update code and separated repos scheme
..
Update code and separated repos scheme
Remove existing code.
Add up-to-date uap-core a
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/238302
Change subject: Update ua-parser version to 1.3.0-wmf2 and tests
..
Update ua-parser version to 1.3.0-wmf2 and tests
Bug: T106134
Change-Id: Ie2
Joal has submitted this change and it was merged.
Change subject: Make pageview definition aware of preview parameter
..
Make pageview definition aware of preview parameter
If x-analytics header includes tag preview
the request
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/238814
Change subject: Update changelog for version v0.0.19
..
Update changelog for version v0.0.19
Change-Id: I0d2a9f7eb590a796b187b131fe0f7bf87245ade
Joal has submitted this change and it was merged.
Change subject: Update changelog for version v0.0.19
..
Update changelog for version v0.0.19
Change-Id: I0d2a9f7eb590a796b187b131fe0f7bf87245aded
---
M changelog.md
1 file chang
Joal has submitted this change and it was merged.
Change subject: Bump core and hive jar versions
..
Bump core and hive jar versions
Bump refinery-core and refinery-hive jars to be 0.0.19 and update symbolic
links.
Rename isCr
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/239360
Change subject: Correct bug in refine
..
Correct bug in refine
New function was not correctly used
Change-Id: Ib0eb8dd7ef931e54c37bf09736fd9252
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/240094
Change subject: Add oozie email sending subworkflow wrapper
..
Add oozie email sending subworkflow wrapper
Change-Id: I0ef57ec914c7b13e896ff0836
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/240095
Change subject: Add email sending on error in webrequest-load
..
Add email sending on error in webrequest-load
Send email only in case of error,
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/240099
Change subject: Add pageview quality check to pageview_hourly
..
Add pageview quality check to pageview_hourly
Add pageview_whitelist and pagevi
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/312538
Change subject: Update hive-site.xml references
..
Update hive-site.xml references
hive-site.xml is needed on hdfs for oozie to run hive queries
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/308532
Change subject: Make camus partition checker more resilient
..
Make camus partition checker more resilient
In previous version camus partition c
Joal has submitted this change and it was merged.
Change subject: Update changelog for 0.0.35 release
..
Update changelog for 0.0.35 release
Change-Id: I2e45751b75124146a31bc18ba8f40331582f8ec9
---
M changelog.md
1 file changed
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/309323
Change subject: Update camus jar version
..
Update camus jar version
Camus jar has been update (patch
https://gerrit.wikimedia.org/r/#/c/308532/
Joal has submitted this change and it was merged.
Change subject: [search] Add support for generator api requests
..
[search] Add support for generator api requests
I think we underestimate api usage for some kind of searches:
Joal has submitted this change and it was merged.
Change subject: Adding several wikis to Pageview whitelist
..
Adding several wikis to Pageview whitelist
Linked to these changes:
https://gerrit.wikimedia.org/r/#/c/316845/
Bug
Joal has submitted this change and it was merged.
Change subject: Update cassandra load jobs to match new aqs needs
..
Update cassandra load jobs to match new aqs needs
Old aqs nodes are being removed, new cassandra loading
con
Joal has submitted this change and it was merged.
Change subject: Changes for refinery source v 0.0.36
..
Changes for refinery source v 0.0.36
Change-Id: I081e3ea660d9f5cb3113d0e083ab92bc5cfad01f
---
M changelog.md
1 file chang
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/319084
Change subject: Update jar version for webrequest load job
..
Update jar version for webrequest load job
refinery jars v0.0.36 contain updated P
Joal has submitted this change and it was merged.
Change subject: Include pageviews for all wikis in whitelist
..
Include pageviews for all wikis in whitelist
Bug: T130249
Change-Id: I3a652900ea540816d2fa2e9acd1679093aa1568a
--
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/319374
Change subject: Update PageviewDefinition fixing iOS bug
..
Update PageviewDefinition fixing iOS bug
Bug: T148663
Change-Id: I6231b050ec87255ee
Joal has submitted this change and it was merged.
Change subject: Update PageviewDefinition to better identify iOS Pageviews
..
Update PageviewDefinition to better identify iOS Pageviews
We look for iOS or Phone in user agent a
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/320376
Change subject: Add ec.wikimedia to pageview whitelist
..
Add ec.wikimedia to pageview whitelist
Change-Id: I9efe408321b18a43957fd8c07bbc185a4a8
Joal has submitted this change and it was merged.
Change subject: Add ec.wikimedia to pageview whitelist
..
Add ec.wikimedia to pageview whitelist
Change-Id: I9efe408321b18a43957fd8c07bbc185a4a83a2d9
---
M static_data/pageview/
Joal has submitted this change and it was merged.
Change subject: Update the src submodule's SHA to 2b19082
..
Update the src submodule's SHA to 2b19082
In T146389 the pivot src repository was updated with the build
directory,
Joal has uploaded a new change for review.
https://gerrit.wikimedia.org/r/315241
Change subject: Update cassandra load jobs to match new aqs needs
..
Update cassandra load jobs to match new aqs needs
Old aqs nodes are being r
Joal has submitted this change and it was merged.
Change subject: Add Oozie job to extract data for WDQS research
..
Add Oozie job to extract data for WDQS research
Purging of the sensitive data is tracked in T146915.
This job
1 - 100 of 639 matches
Mail list logo
