Following up on this, data is been re-ingested into druid so turnilo should not show up any holes related to this event.
Thanks, Nuria On Thu, Nov 15, 2018 at 8:45 AM Marcel Ruiz Forns <[email protected]> wrote: > Not all data sources are populated at the same time, the data on Druid is >> ingested twice, once per hour and once daily looking 4 days back. Data >> should appear once daily job runs for the "holes" missing. > > +1 The EL2Druid daily loading job will cover up the holes for the 12th > and 13th in 1 or 2 days. > > On Thu, Nov 15, 2018 at 5:03 PM Nuria Ruiz <[email protected]> wrote: > >> Hello, >> >> Not all data sources are populated at the same time, the data on Druid is >> ingested twice, once per hour and once daily looking 4 days back. Data >> should appear once daily job runs for the "holes" missing. >> >> Thanks, >> >> Nuria >> >> On Thu, Nov 15, 2018 at 7:49 AM Andrew Otto <[email protected]> wrote: >> >>> > Does "fixed" mean that the missing data already been backfilled? I’m >>> seeing gaps (zero events) in Turnilo for Druid-ingested EL data, for the >>> timespans between around 6am-16pm on November 13, and 7am-10am on November >>> 12. >>> >>> Hm. Fixed means the data has been refined to Hive. I didn’t check on >>> Druid EL data. Marcel, can you check up on that? This sounds related, we >>> might need to figure out how to re run the Druid ingestion if/when base >>> data partitions change. >>> >>> >>> >>> On Thu, Nov 15, 2018 at 10:17 AM Tilman Bayer <[email protected]> >>> wrote: >>> >>>> Hi Andrew, >>>> >>>> PS, offtopic: reminder that [email protected] has been >>>> deprecated/retired a while ago; our team can be reached at >>>> [email protected] (CCed) >>>> >>>> On Thu, Nov 15, 2018 at 7:13 AM Tilman Bayer <[email protected]> >>>> wrote: >>>> >>>>> Does "fixed" mean that the missing data already been backfilled? I'm >>>>> seeing gaps (zero events) in Turnilo for Druid-ingested EL data, for the >>>>> timespans between around 6am-16pm on November 13, and 7am-10am on November >>>>> 12. >>>>> >>>>> On Thu, Nov 15, 2018 at 6:51 AM Andrew Otto <[email protected]> >>>>> wrote: >>>>> >>>>>> OH I’m sorry! There is a Phab task, and this is fixed. >>>>>> https://phabricator.wikimedia.org/T209407 >>>>>> >>>>>> Very sorry, I should have updated and linked the task. >>>>>> >>>>>> On Thu, Nov 15, 2018 at 9:50 AM Gilles Dubuc <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Is this issue still ongoing? Is there a corresponding Phabricator >>>>>>> task to follow? >>>>>>> >>>>>>> On Tue, Nov 13, 2018 at 6:27 PM Andrew Otto <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi all, >>>>>>>> >>>>>>>> Yesterday we upgraded the Hadoop cluster to a newer version. It >>>>>>>> seems that along the way the job that imports EventLogging data into >>>>>>>> Hive >>>>>>>> has started failing for some EventLogging datasets. I’m still >>>>>>>> investigating, but it seems that tables (I believe ones that need to >>>>>>>> have >>>>>>>> their schema modified during refinement) are failing refinement. The >>>>>>>> list >>>>>>>> of affected tables at this moment is large, and will likely only grow >>>>>>>> as >>>>>>>> time goes on. >>>>>>>> >>>>>>>> We’re looking into this and will try to have things fixed ASAP. >>>>>>>> For now you might expect Hive EventLogging data to be delayed. >>>>>>>> >>>>>>>> Sorry for the trouble, and we’ll keep you updated! >>>>>>>> -Andrew Otto >>>>>>>> Systems Engineer, WMF >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> The list of currently affected tables is: >>>>>>>> >>>>>>>> AdvancedSearchRequest >>>>>>>> CentralAuth >>>>>>>> CentralNoticeBannerHistory >>>>>>>> CentralNoticeImpression >>>>>>>> CentralNoticeTiming >>>>>>>> ChangesListFilterGrouping >>>>>>>> ChangesListFilters >>>>>>>> CitationUsage >>>>>>>> CitationUsagePageLoad >>>>>>>> ContentTranslation >>>>>>>> ContentTranslationAbuseFilter >>>>>>>> ContentTranslationCTA >>>>>>>> ContentTranslationError >>>>>>>> ContentTranslationSuggestion >>>>>>>> CpuBenchmark >>>>>>>> EchoInteraction >>>>>>>> EchoMail >>>>>>>> EditAttemptStep >>>>>>>> EditConflict >>>>>>>> EditorActivation >>>>>>>> EUCCVisit >>>>>>>> EventError >>>>>>>> FlowReplies >>>>>>>> GettingStartedRedirectImpression >>>>>>>> GuidedTourButtonClick >>>>>>>> GuidedTourExited >>>>>>>> GuidedTourExternalLinkActivation >>>>>>>> GuidedTourGuiderHidden >>>>>>>> GuidedTourGuiderImpression >>>>>>>> LandingPageImpression >>>>>>>> MediaViewer >>>>>>>> MediaWikiPingback >>>>>>>> MobileAppCategorizationAttempts >>>>>>>> MobileAppUploadAttempts >>>>>>>> MobileWebMainMenuClickTracking >>>>>>>> MobileWebSearch >>>>>>>> MobileWebUIClickTracking >>>>>>>> MobileWikiAppAppearanceSettings >>>>>>>> MobileWikiAppArticleSuggestions >>>>>>>> MobileWikiAppCreateAccount >>>>>>>> MobileWikiAppDailyStats >>>>>>>> MobileWikiAppEdit >>>>>>>> MobileWikiAppFeed >>>>>>>> MobileWikiAppFeedConfigure >>>>>>>> MobileWikiAppFindInPage >>>>>>>> MobileWikiAppInstallReferrer >>>>>>>> MobileWikiAppIntents >>>>>>>> MobileWikiAppiOSFeed >>>>>>>> MobileWikiAppiOSLoginAction >>>>>>>> MobileWikiAppiOSReadingLists >>>>>>>> MobileWikiAppiOSSearch >>>>>>>> MobileWikiAppiOSSessions >>>>>>>> MobileWikiAppiOSSettingAction >>>>>>>> MobileWikiAppiOSUserHistory >>>>>>>> MobileWikiAppLangSelect >>>>>>>> MobileWikiAppLanguageSearching >>>>>>>> MobileWikiAppLanguageSettings >>>>>>>> MobileWikiAppLinkPreview >>>>>>>> MobileWikiAppLogin >>>>>>>> MobileWikiAppMediaGallery >>>>>>>> MobileWikiAppNavMenu >>>>>>>> MobileWikiAppNotificationInteraction >>>>>>>> MobileWikiAppNotificationPreferences >>>>>>>> MobileWikiAppOfflineLibrary >>>>>>>> MobileWikiAppOnboarding >>>>>>>> MobileWikiAppOnThisDay >>>>>>>> MobileWikiAppPageScroll >>>>>>>> MobileWikiAppProtectedEditAttempt >>>>>>>> MobileWikiAppRandomizer >>>>>>>> MobileWikiAppReadingLists >>>>>>>> MobileWikiAppSavedPages >>>>>>>> MobileWikiAppSearch >>>>>>>> MobileWikiAppSessions >>>>>>>> MobileWikiAppShareAFact >>>>>>>> MobileWikiAppStuffHappens >>>>>>>> MobileWikiAppTabs >>>>>>>> MobileWikiAppToCInteraction >>>>>>>> MobileWikiAppWidgets >>>>>>>> MobileWikiAppWiktionaryPopup >>>>>>>> MultimediaViewerAttribution >>>>>>>> MultimediaViewerDimensions >>>>>>>> MultimediaViewerDuration >>>>>>>> MultimediaViewerNetworkPerformance >>>>>>>> NavigationTiming >>>>>>>> PageIssues >>>>>>>> Popups >>>>>>>> PrefUpdate >>>>>>>> Print >>>>>>>> QuickSurveyInitiation >>>>>>>> QuickSurveysResponses >>>>>>>> ReadingDepth >>>>>>>> ResourceTiming >>>>>>>> SaveTiming >>>>>>>> SearchSatisfaction >>>>>>>> SearchSatisfactionErrors >>>>>>>> ServerSideAccountCreation >>>>>>>> TemplateWizard >>>>>>>> TestSearchSatisfaction2 >>>>>>>> TranslationRecommendationAPIRequests >>>>>>>> TranslationRecommendationUIRequests >>>>>>>> TranslationRecommendationUserAction >>>>>>>> TwoColConflictConflict >>>>>>>> UniversalLanguageSelector >>>>>>>> UploadWizardErrorFlowEvent >>>>>>>> UploadWizardExceptionFlowEvent >>>>>>>> UploadWizardFlowEvent >>>>>>>> UploadWizardStep >>>>>>>> UploadWizardTutorialActions >>>>>>>> UploadWizardUploadFlowEvent >>>>>>>> VirtualPageView >>>>>>>> WikidataCompletionSearchClicks >>>>>>>> WikimediaBlogVisit >>>>>>>> WikipediaPortal >>>>>>>> WikipediaZeroUsage >>>>>>>> WMDEBannerEvents >>>>>>>> WMDEBannerSizeIssue >>>>>>>> _______________________________________________ >>>>>>>> Analytics mailing list >>>>>>>> [email protected] >>>>>>>> https://lists.wikimedia.org/mailman/listinfo/analytics >>>>>>>> >>>>>>> _______________________________________________ >>>>>>> Analytics mailing list >>>>>>> [email protected] >>>>>>> https://lists.wikimedia.org/mailman/listinfo/analytics >>>>>>> >>>>>> >>>>> >>>>> -- >>>>> Tilman Bayer >>>>> Senior Analyst >>>>> Wikimedia Foundation >>>>> IRC (Freenode): HaeB >>>>> >>>> >>>> >>>> -- >>>> Tilman Bayer >>>> Senior Analyst >>>> Wikimedia Foundation >>>> IRC (Freenode): HaeB >>>> >>> _______________________________________________ >>> Analytics mailing list >>> [email protected] >>> https://lists.wikimedia.org/mailman/listinfo/analytics >>> >> _______________________________________________ >> Analytics mailing list >> [email protected] >> https://lists.wikimedia.org/mailman/listinfo/analytics >> > > > -- > *Marcel Ruiz Forns* > Analytics Developer > Wikimedia Foundation > _______________________________________________ > Analytics mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/analytics >
_______________________________________________ Analytics mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/analytics
