Bernd, No, I can't give you the original sixze, the earliest register of the logs is already truncated. But you're right, the problem is the multi-byte code set, which when URI-decoded gets really long.
On Fri, May 15, 2015 at 11:38 PM, Bernd Sitzmann <[email protected]> wrote: > On Android, we restrict logging of the text field to the first 99 > characters. But, of course, there are many other fields. If there is a long > title, maybe the text is in a multi-byte code set, then it could be that > the total message gets too long. > Ideally, we would look at the total message length, but that's much harder > to implement. > > Marcel, do you know by how much we are over the limit? > > Bernd > > On Fri, May 15, 2015 at 11:11 PM, Marcel Ruiz Forns <[email protected]> > wrote: > >> Since today 2015-05-15 at 08:25h the validation errors for >> MobileWikiAppShareAFact >> schema are distributed like: >> >> 553 iPhone >> 261 Android >> >> On Fri, May 15, 2015 at 10:51 PM, Corey Floyd <[email protected]> >> wrote: >> >>> Just to be clear… not looking for exact percentages, just a note of >>> “iOS”, Android” or “both". >>> >>> On Fri, May 15, 2015 at 4:49 PM, Corey Floyd <[email protected]> >>> wrote: >>> >>>> Marcel, do you have breakdown per platform? Not sure if each of these >>>> issues is shared between iOS and Android or specific to one or the other. >>>> Would be nice to know as we check things out. >>>> >>>> On Fri, May 15, 2015 at 4:38 PM, Adam Baso <[email protected]> wrote: >>>> >>>>> Marcel, thanks for the links and Dmitry thanks for coordinating this. >>>>> >>>>> A couple notes from my memory: >>>>> >>>>> iOS: >>>>> EventLogging.h >>>>> <https://git.wikimedia.org/blob/apps%2Fios%2Fwikipedia.git/HEAD/Wikipedia%2FEventLogging%2FEventLoggingFunnel.h#L12> >>>>> defines WMFEventLoggingMaxStringLength_Snippet >>>>> and WMFEventLoggingMaxStringLength_General, which are only currently used >>>>> for Share a Fact in WMFShareFunnel.m >>>>> <https://git.wikimedia.org/blob/apps%2Fios%2Fwikipedia.git/HEAD/Wikipedia%2FView%20Controllers%2FShareCard%2FWMFShareFunnel.m> >>>>> . >>>>> Not sure if URL-encoding on the request path or actual database bytes >>>>> (Unicode?) occupied causes overflows despite this. >>>>> >>>>> As I recall, Android does something similar on Share a Fact ( >>>>> ShareAFactFunnel.java >>>>> <https://git.wikimedia.org/blob/apps%2Fandroid%2Fwikipedia.git/HEAD/wikipedia%2Fsrc%2Fmain%2Fjava%2Forg%2Fwikipedia%2Fanalytics%2FShareAFactFunnel.java#L19>). >>>>> Same thing here with URL-encoding and actual database bytes (Unicode?). >>>>> >>>>> -Adam >>>>> >>>>> >>>>> On Fri, May 15, 2015 at 1:21 PM, Marcel Ruiz Forns < >>>>> [email protected]> wrote: >>>>> >>>>>> Dmitry, >>>>>> >>>>>> you're totally right, it was in the email, but it should have been in >>>>>> the wikis. >>>>>> I added some documentation on it here: >>>>>> https://wikitech.wikimedia.org/wiki/EventLogging#Log_size_limit >>>>>> >>>>>> Thanks! >>>>>> >>>>>> On Fri, May 15, 2015 at 10:00 PM, Dmitry Brant <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> [brain fart] it's in your email. :( Thanks! >>>>>>> >>>>>>> On Fri, May 15, 2015 at 3:59 PM, Dmitry Brant <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Thanks for reporting this, Marcel! >>>>>>>> I've created a task for us to correct the behavior of our EL >>>>>>>> funnels: https://phabricator.wikimedia.org/T99276 >>>>>>>> >>>>>>>> Is the actual character limit of EL messages specified somewhere? >>>>>>>> >>>>>>>> >>>>>>>> -Dmitry >>>>>>>> >>>>>>>> >>>>>>>> On Fri, May 15, 2015 at 3:41 PM, Marcel Ruiz Forns < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi Mobile, >>>>>>>>> >>>>>>>>> Analyzing EventLogging logs we percieved that a significant share >>>>>>>>> of MobileWikiAppSavedPages, MobileWikiAppArticleSuggestions and >>>>>>>>> MobileWikiAppShareAFact events are failing validation. >>>>>>>>> >>>>>>>>> *1) MobileWikiAppShareAFact: 1.5% not validating* >>>>>>>>> In this schema, the field "text" stores long fractions of text >>>>>>>>> sometimes. >>>>>>>>> This exceeds the size limitation of EL, specially when the text >>>>>>>>> contains special characters, like chinese, greek, etc. >>>>>>>>> >>>>>>>>> *2) MobileWikiAppArticleSuggestions: 1% not validating* >>>>>>>>> In this case, it's the field "readMoreList" that is sometimes very >>>>>>>>> long, >>>>>>>>> specially when it contains special characters. >>>>>>>>> This, again, exceeds the log size limit. >>>>>>>>> >>>>>>>>> *3) MobileWikiAppSavedPages: 1% not validating* >>>>>>>>> Some events do not contain the required field "appInstallID". >>>>>>>>> >>>>>>>>> In cases 1) and 2) the percentage is not big overall, but it can >>>>>>>>> be that for a given language, a lot of events are lost. >>>>>>>>> >>>>>>>>> EventLogging performance is not compromised by these validation >>>>>>>>> errors, but we are receiving monitoring alerts, and would like to >>>>>>>>> maintain >>>>>>>>> the validation rate close to 100%. >>>>>>>>> >>>>>>>>> Is it possible for you to somehow reduce the size of the logs of >>>>>>>>> 1) and 2)? >>>>>>>>> If so, have in mind that the log size limit is 1k, and that the >>>>>>>>> highest priority for us would be 2). >>>>>>>>> >>>>>>>>> Thank you! >>>>>>>>> >>>>>>>>> Marcel >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Mobile-l mailing list >>>>>>>>> [email protected] >>>>>>>>> https://lists.wikimedia.org/mailman/listinfo/mobile-l >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Mobile-l mailing list >>>>>> [email protected] >>>>>> https://lists.wikimedia.org/mailman/listinfo/mobile-l >>>>>> >>>>>> >>>>> >>>>> _______________________________________________ >>>>> Mobile-l mailing list >>>>> [email protected] >>>>> https://lists.wikimedia.org/mailman/listinfo/mobile-l >>>>> >>>>> >>>> >>>> >>>> -- >>>> Corey Floyd >>>> Software Engineer >>>> Mobile Apps / iOS >>>> Wikimedia Foundation >>>> >>> >>> >>> >>> -- >>> Corey Floyd >>> Software Engineer >>> Mobile Apps / iOS >>> Wikimedia Foundation >>> >> >> >> _______________________________________________ >> Mobile-l mailing list >> [email protected] >> https://lists.wikimedia.org/mailman/listinfo/mobile-l >> >> >
_______________________________________________ Mobile-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/mobile-l
