Hello, Not all data sources are populated at the same time, the data on Druid is ingested twice, once per hour and once daily looking 4 days back. Data should appear once daily job runs for the "holes" missing.
Thanks, Nuria On Thu, Nov 15, 2018 at 7:49 AM Andrew Otto <[email protected]> wrote: > > Does "fixed" mean that the missing data already been backfilled? I’m > seeing gaps (zero events) in Turnilo for Druid-ingested EL data, for the > timespans between around 6am-16pm on November 13, and 7am-10am on November > 12. > > Hm. Fixed means the data has been refined to Hive. I didn’t check on > Druid EL data. Marcel, can you check up on that? This sounds related, we > might need to figure out how to re run the Druid ingestion if/when base > data partitions change. > > > > On Thu, Nov 15, 2018 at 10:17 AM Tilman Bayer <[email protected]> > wrote: > >> Hi Andrew, >> >> PS, offtopic: reminder that [email protected] has been >> deprecated/retired a while ago; our team can be reached at >> [email protected] (CCed) >> >> On Thu, Nov 15, 2018 at 7:13 AM Tilman Bayer <[email protected]> >> wrote: >> >>> Does "fixed" mean that the missing data already been backfilled? I'm >>> seeing gaps (zero events) in Turnilo for Druid-ingested EL data, for the >>> timespans between around 6am-16pm on November 13, and 7am-10am on November >>> 12. >>> >>> On Thu, Nov 15, 2018 at 6:51 AM Andrew Otto <[email protected]> wrote: >>> >>>> OH I’m sorry! There is a Phab task, and this is fixed. >>>> https://phabricator.wikimedia.org/T209407 >>>> >>>> Very sorry, I should have updated and linked the task. >>>> >>>> On Thu, Nov 15, 2018 at 9:50 AM Gilles Dubuc <[email protected]> >>>> wrote: >>>> >>>>> Is this issue still ongoing? Is there a corresponding Phabricator task >>>>> to follow? >>>>> >>>>> On Tue, Nov 13, 2018 at 6:27 PM Andrew Otto <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi all, >>>>>> >>>>>> Yesterday we upgraded the Hadoop cluster to a newer version. It >>>>>> seems that along the way the job that imports EventLogging data into Hive >>>>>> has started failing for some EventLogging datasets. I’m still >>>>>> investigating, but it seems that tables (I believe ones that need to have >>>>>> their schema modified during refinement) are failing refinement. The >>>>>> list >>>>>> of affected tables at this moment is large, and will likely only grow as >>>>>> time goes on. >>>>>> >>>>>> We’re looking into this and will try to have things fixed ASAP. For >>>>>> now you might expect Hive EventLogging data to be delayed. >>>>>> >>>>>> Sorry for the trouble, and we’ll keep you updated! >>>>>> -Andrew Otto >>>>>> Systems Engineer, WMF >>>>>> >>>>>> >>>>>> >>>>>> The list of currently affected tables is: >>>>>> >>>>>> AdvancedSearchRequest >>>>>> CentralAuth >>>>>> CentralNoticeBannerHistory >>>>>> CentralNoticeImpression >>>>>> CentralNoticeTiming >>>>>> ChangesListFilterGrouping >>>>>> ChangesListFilters >>>>>> CitationUsage >>>>>> CitationUsagePageLoad >>>>>> ContentTranslation >>>>>> ContentTranslationAbuseFilter >>>>>> ContentTranslationCTA >>>>>> ContentTranslationError >>>>>> ContentTranslationSuggestion >>>>>> CpuBenchmark >>>>>> EchoInteraction >>>>>> EchoMail >>>>>> EditAttemptStep >>>>>> EditConflict >>>>>> EditorActivation >>>>>> EUCCVisit >>>>>> EventError >>>>>> FlowReplies >>>>>> GettingStartedRedirectImpression >>>>>> GuidedTourButtonClick >>>>>> GuidedTourExited >>>>>> GuidedTourExternalLinkActivation >>>>>> GuidedTourGuiderHidden >>>>>> GuidedTourGuiderImpression >>>>>> LandingPageImpression >>>>>> MediaViewer >>>>>> MediaWikiPingback >>>>>> MobileAppCategorizationAttempts >>>>>> MobileAppUploadAttempts >>>>>> MobileWebMainMenuClickTracking >>>>>> MobileWebSearch >>>>>> MobileWebUIClickTracking >>>>>> MobileWikiAppAppearanceSettings >>>>>> MobileWikiAppArticleSuggestions >>>>>> MobileWikiAppCreateAccount >>>>>> MobileWikiAppDailyStats >>>>>> MobileWikiAppEdit >>>>>> MobileWikiAppFeed >>>>>> MobileWikiAppFeedConfigure >>>>>> MobileWikiAppFindInPage >>>>>> MobileWikiAppInstallReferrer >>>>>> MobileWikiAppIntents >>>>>> MobileWikiAppiOSFeed >>>>>> MobileWikiAppiOSLoginAction >>>>>> MobileWikiAppiOSReadingLists >>>>>> MobileWikiAppiOSSearch >>>>>> MobileWikiAppiOSSessions >>>>>> MobileWikiAppiOSSettingAction >>>>>> MobileWikiAppiOSUserHistory >>>>>> MobileWikiAppLangSelect >>>>>> MobileWikiAppLanguageSearching >>>>>> MobileWikiAppLanguageSettings >>>>>> MobileWikiAppLinkPreview >>>>>> MobileWikiAppLogin >>>>>> MobileWikiAppMediaGallery >>>>>> MobileWikiAppNavMenu >>>>>> MobileWikiAppNotificationInteraction >>>>>> MobileWikiAppNotificationPreferences >>>>>> MobileWikiAppOfflineLibrary >>>>>> MobileWikiAppOnboarding >>>>>> MobileWikiAppOnThisDay >>>>>> MobileWikiAppPageScroll >>>>>> MobileWikiAppProtectedEditAttempt >>>>>> MobileWikiAppRandomizer >>>>>> MobileWikiAppReadingLists >>>>>> MobileWikiAppSavedPages >>>>>> MobileWikiAppSearch >>>>>> MobileWikiAppSessions >>>>>> MobileWikiAppShareAFact >>>>>> MobileWikiAppStuffHappens >>>>>> MobileWikiAppTabs >>>>>> MobileWikiAppToCInteraction >>>>>> MobileWikiAppWidgets >>>>>> MobileWikiAppWiktionaryPopup >>>>>> MultimediaViewerAttribution >>>>>> MultimediaViewerDimensions >>>>>> MultimediaViewerDuration >>>>>> MultimediaViewerNetworkPerformance >>>>>> NavigationTiming >>>>>> PageIssues >>>>>> Popups >>>>>> PrefUpdate >>>>>> Print >>>>>> QuickSurveyInitiation >>>>>> QuickSurveysResponses >>>>>> ReadingDepth >>>>>> ResourceTiming >>>>>> SaveTiming >>>>>> SearchSatisfaction >>>>>> SearchSatisfactionErrors >>>>>> ServerSideAccountCreation >>>>>> TemplateWizard >>>>>> TestSearchSatisfaction2 >>>>>> TranslationRecommendationAPIRequests >>>>>> TranslationRecommendationUIRequests >>>>>> TranslationRecommendationUserAction >>>>>> TwoColConflictConflict >>>>>> UniversalLanguageSelector >>>>>> UploadWizardErrorFlowEvent >>>>>> UploadWizardExceptionFlowEvent >>>>>> UploadWizardFlowEvent >>>>>> UploadWizardStep >>>>>> UploadWizardTutorialActions >>>>>> UploadWizardUploadFlowEvent >>>>>> VirtualPageView >>>>>> WikidataCompletionSearchClicks >>>>>> WikimediaBlogVisit >>>>>> WikipediaPortal >>>>>> WikipediaZeroUsage >>>>>> WMDEBannerEvents >>>>>> WMDEBannerSizeIssue >>>>>> _______________________________________________ >>>>>> Analytics mailing list >>>>>> [email protected] >>>>>> https://lists.wikimedia.org/mailman/listinfo/analytics >>>>>> >>>>> _______________________________________________ >>>>> Analytics mailing list >>>>> [email protected] >>>>> https://lists.wikimedia.org/mailman/listinfo/analytics >>>>> >>>> >>> >>> -- >>> Tilman Bayer >>> Senior Analyst >>> Wikimedia Foundation >>> IRC (Freenode): HaeB >>> >> >> >> -- >> Tilman Bayer >> Senior Analyst >> Wikimedia Foundation >> IRC (Freenode): HaeB >> > _______________________________________________ > Analytics mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/analytics >
_______________________________________________ Analytics mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/analytics
