I think there's another issue. The shortened ID is used as a Permalink, i.e. it is passed to thread.html/thread.lua
However the thread.lua code does not know how to unshorten the ID. Also the name of the function is misleading, as the parameter is an ID not a URL. On 13 September 2016 at 10:29, sebb <seb...@gmail.com> wrote: > On 13 September 2016 at 10:13, Daniel Gruno <humbed...@apache.org> wrote: >> On 09/13/2016 11:07 AM, sebb wrote: >>> On 13 September 2016 at 09:53, Daniel Gruno <humbed...@apache.org> wrote: >>>> On 09/13/2016 10:50 AM, sebb wrote: >>>>> On 13 September 2016 at 09:01, Daniel Gruno <humbed...@apache.org> wrote: >>>>>> On 09/12/2016 12:48 PM, sebb wrote: >>>>>>> On 8 September 2016 at 07:03, <humbed...@apache.org> wrote: >>>>>>>> Repository: incubator-ponymail >>>>>>>> Updated Branches: >>>>>>>> refs/heads/coffee-and-cake d15294548 -> ae791c1a6 >>>>>>>> >>>>>>>> >>>>>>>> add URL shortening >>>>>>>> >>>>>>>> this is a bit longer than the old format, and only shortens >>>>>>>> if the generator is medium or full, to avoid duplicates. >>>>>>>> >>>>>>>> >>>>>>>> Project: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/repo >>>>>>>> Commit: >>>>>>>> http://git-wip-us.apache.org/repos/asf/incubator-ponymail/commit/ae791c1a >>>>>>>> Tree: >>>>>>>> http://git-wip-us.apache.org/repos/asf/incubator-ponymail/tree/ae791c1a >>>>>>>> Diff: >>>>>>>> http://git-wip-us.apache.org/repos/asf/incubator-ponymail/diff/ae791c1a >>>>>>>> >>>>>>>> Branch: refs/heads/coffee-and-cake >>>>>>>> Commit: ae791c1a619fc965cca72f1f62edb87b07a088b9 >>>>>>>> Parents: d152945 >>>>>>>> Author: Daniel Gruno <humbed...@apache.org> >>>>>>>> Authored: Thu Sep 8 08:02:50 2016 +0200 >>>>>>>> Committer: Daniel Gruno <humbed...@apache.org> >>>>>>>> Committed: Thu Sep 8 08:02:50 2016 +0200 >>>>>>>> >>>>>>>> ---------------------------------------------------------------------- >>>>>>>> site/api/thread.lua | 6 +++++ >>>>>>>> site/js/coffee/email_display_basic.coffee | 3 ++- >>>>>>>> site/js/coffee/listview_basic.coffee | 3 ++- >>>>>>>> site/js/coffee/listview_single_email.coffee | 4 ++- >>>>>>>> site/js/coffee/misc.coffee | 31 >>>>>>>> ++++++++++++++++++++++++ >>>>>>>> site/js/coffee/scaffolding.coffee | 2 +- >>>>>>>> 6 files changed, 45 insertions(+), 4 deletions(-) >>>>>>>> ---------------------------------------------------------------------- >>>>>>>> >>>>>>>> >>>>>>>> http://git-wip-us.apache.org/repos/asf/incubator-ponymail/blob/ae791c1a/site/api/thread.lua >>>>>>>> ---------------------------------------------------------------------- >>>>>>>> diff --git a/site/api/thread.lua b/site/api/thread.lua >>>>>>>> index aaaa9da..46e93c1 100644 >>>>>>>> --- a/site/api/thread.lua >>>>>>>> +++ b/site/api/thread.lua >>>>>>>> @@ -126,6 +126,12 @@ function handle(r) >>>>>>>> if #docs == 1 then >>>>>>>> doc = docs[1] >>>>>>>> end >>>>>>>> + -- New shortening for medium/full generators >>>>>>>> + if #docs == 0 and #eid == 24 then >>>>>>>> + docs = elastic.find("mid:" .. r:escape(eid) .. "*", 1, >>>>>>>> "mbox") >>>>>>>> + >>>>>>>> + end >>>>>>>> + -- old shortening format, bleh >>>>>>>> if #docs == 0 and #eid == 18 then >>>>>>>> docs = elastic.find("mid:" .. r:escape(eid) .. "*", 1, >>>>>>>> "mbox") >>>>>>>> end >>>>>>>> >>>>>>>> http://git-wip-us.apache.org/repos/asf/incubator-ponymail/blob/ae791c1a/site/js/coffee/email_display_basic.coffee >>>>>>>> ---------------------------------------------------------------------- >>>>>>>> diff --git a/site/js/coffee/email_display_basic.coffee >>>>>>>> b/site/js/coffee/email_display_basic.coffee >>>>>>>> index 708c373..73a60f6 100644 >>>>>>>> --- a/site/js/coffee/email_display_basic.coffee >>>>>>>> +++ b/site/js/coffee/email_display_basic.coffee >>>>>>>> @@ -149,7 +149,8 @@ class BasicEmailDisplay >>>>>>>> ### Action buttons ### >>>>>>>> >>>>>>>> ### Permalink ### >>>>>>>> - pbutton = new HTML('a', { class: "label_yellow", href: >>>>>>>> "thread.html/#{json.mid}"}, "Permalink") >>>>>>>> + shortID = shortenURL(json.mid) >>>>>>>> + pbutton = new HTML('a', { class: "label_yellow", href: >>>>>>>> "thread.html/#{shortID}"}, "Permalink") >>>>>>>> >>>>>>>> ### Source ### >>>>>>>> sbutton = new HTML('a', { class: "label_red", href: >>>>>>>> "api/source.lua/#{json.mid}"}, "View source") >>>>>>>> >>>>>>>> http://git-wip-us.apache.org/repos/asf/incubator-ponymail/blob/ae791c1a/site/js/coffee/listview_basic.coffee >>>>>>>> ---------------------------------------------------------------------- >>>>>>>> diff --git a/site/js/coffee/listview_basic.coffee >>>>>>>> b/site/js/coffee/listview_basic.coffee >>>>>>>> index f9bfe47..bd97dbf 100644 >>>>>>>> --- a/site/js/coffee/listview_basic.coffee >>>>>>>> +++ b/site/js/coffee/listview_basic.coffee >>>>>>>> @@ -234,8 +234,9 @@ class BasicListView >>>>>>>> >>>>>>>> >>>>>>>> ### Subject, PLUS a bit of the body with a break before >>>>>>>> ### >>>>>>>> + sid = shortenURL(thread.tid) >>>>>>>> subject = new HTML('div', {}, [ >>>>>>>> - new HTML('a', { style: {fontWeight: readStyle}, href: >>>>>>>> "thread.html/#{thread.tid}", onclick: >>>>>>>> "readEmail(this.parentNode.parentNode.parentNode); >>>>>>>> this.style.fontWeight = 'normal'; return false;"}, original.subject), >>>>>>>> + new HTML('a', { style: {fontWeight: readStyle}, href: >>>>>>>> "thread.html/#sid}", onclick: >>>>>>>> "readEmail(this.parentNode.parentNode.parentNode); >>>>>>>> this.style.fontWeight = 'normal'; return false;"}, original.subject), >>>>>>>> new HTML('br'), >>>>>>>> new HTML('span', {class: "listview_item_body"}, >>>>>>>> thread.body) >>>>>>>> ]) >>>>>>>> >>>>>>>> http://git-wip-us.apache.org/repos/asf/incubator-ponymail/blob/ae791c1a/site/js/coffee/listview_single_email.coffee >>>>>>>> ---------------------------------------------------------------------- >>>>>>>> diff --git a/site/js/coffee/listview_single_email.coffee >>>>>>>> b/site/js/coffee/listview_single_email.coffee >>>>>>>> index 5aa3e07..b3baff8 100644 >>>>>>>> --- a/site/js/coffee/listview_single_email.coffee >>>>>>>> +++ b/site/js/coffee/listview_single_email.coffee >>>>>>>> @@ -102,9 +102,11 @@ class SingleListView extends BasicListView >>>>>>>> if hasRead(original.id) >>>>>>>> readStyle = "normal" >>>>>>>> >>>>>>>> + >>>>>>>> ### Subject, PLUS a bit of the body with a break before >>>>>>>> ### >>>>>>>> + sid = shortenURL(original.id) >>>>>>>> subject = new HTML('div', {}, [ >>>>>>>> - new HTML('a', { style: {fontWeight: readStyle}, href: >>>>>>>> "thread.html/#{original.id}", onclick: >>>>>>>> "readEmail(this.parentNode.parentNode.parentNode); >>>>>>>> this.style.fontWeight = 'normal'; return false;"}, original.subject), >>>>>>>> + new HTML('a', { style: {fontWeight: readStyle}, href: >>>>>>>> "thread.html/#{sid}", onclick: >>>>>>>> "readEmail(this.parentNode.parentNode.parentNode); >>>>>>>> this.style.fontWeight = 'normal'; return false;"}, original.subject), >>>>>>>> ]) >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> http://git-wip-us.apache.org/repos/asf/incubator-ponymail/blob/ae791c1a/site/js/coffee/misc.coffee >>>>>>>> ---------------------------------------------------------------------- >>>>>>>> diff --git a/site/js/coffee/misc.coffee b/site/js/coffee/misc.coffee >>>>>>>> index 131748a..2ac684c 100644 >>>>>>>> --- a/site/js/coffee/misc.coffee >>>>>>>> +++ b/site/js/coffee/misc.coffee >>>>>>>> @@ -93,3 +93,34 @@ ponymail_url_regex = new RegExp( >>>>>>>> ponymail_quote_regex = new RegExp( >>>>>>>> "((?:\r?\n)((on .+ wrote:[\r\n]+)|(sent from my .+)|(>+[ >>>>>>>> \t]*[^\r\n]*\r?\n[^\n]*\n*)+)+)+", "mi" >>>>>>>> ) >>>>>>>> + >>>>>>>> +shortBits = 3 >>>>>>>> + >>>>>>>> +shortenURL = (mid) -> >>>>>>>> + arr = mid.split("@") >>>>>>> >>>>>>> This function throws away the list id. >>>>>>> >>>>>>> However AFAIK the list id is essential to disambiguate messages which >>>>>>> are sent to multiple lists. >>>>>> >>>>>> When there are only 2 arr bits (iow one @), then the List ID is already >>>>>> hashed inside the initial first arr bit, and thus one does not need the >>>>>> list ID. This is a compromise and a change from the old shortening >>>>>> procedure where IDs were shortened regardless of which generator was >>>>>> used. The new shortener only shortens IF it can get away with it >>>>>> relatively painlessly, IOW if the list ID is part of the initial sha224 >>>>>> blob. This merely needs some documentation, which I will get around to >>>>>> now that I'm not on vacation :) >>>>> >>>>> In which case it's only necessary to drop the '@list' part. >>>>> >>>>> There is no need to encode/decode as base36. >>>>> The net effect of using shortenURL and unshortenURL does just that. >>>>> >>>>> The relevant change is the change to thread.lua which allows for >>>>> wild-card search if the mid length is 24. >>>> >>>> It's not an impressive compression, no, but it's still slightly shorter :) >>> >>> It's one character shorter. >>> >>> I don't see the point. >>> >>> If the original mid really is unique without the '@list' part, why not >>> just use that instead? >> >> Because it's too long for some people's taste? >> 56+ chars vs 21 chars. > > If it's true that the original mid is unique without the '@list' part, > then just drop the '@list' part to shorten the mid to 24 chars. > > So it's 24 chars versus 23 chars (remember the leading PZ). > >> Ideally, we'd probably change the ID generator to use base36 (or 48) or >> so instead, and avoid using a JS shortener altogether. But it's a moot >> point whether cutting off 3 bytes matter or not. It's a matter of taste. >> This shortens links for the current generators, while still retaining a >> 1 in 10^33 chance of not colliding. >> > >>> >>>>> >>>>>> With regards, >>>>>> Daniel. >>>>>> >>>>>>> >>>>>>>> + if arr.length == 2 and (pm_config and pm_config.shortenLinks) >>>>>>>> + out = "" >>>>>>>> + for i in [0..shortBits-1] >>>>>>>> + a = arr[0].substr(i*8,8) >>>>>>>> + num = parseInt(a, 16) >>>>>>>> + res = num.toString(36) >>>>>>>> + while res.length < 7 >>>>>>>> + res = '0' + res >>>>>>>> + out += res >>>>>>>> + return "PZ" + out >>>>>>>> + >>>>>>>> + return mid >>>>>>>> + >>>>>>>> +unshortenURL = (mid) -> >>>>>>>> + if mid.substr(0,2) == 'PZ' >>>>>>>> + out = "" >>>>>>>> + for i in [0..shortBits-1] >>>>>>>> + num = parseInt(mid.substr(2+(i*7), 7), 36) >>>>>>>> + res = num.toString(16) >>>>>>>> + while res.length < 8 >>>>>>>> + res = '0' + res >>>>>>>> + out += res >>>>>>>> + return out >>>>>>>> + else >>>>>>>> + return mid >>>>>>>> + >>>>>>> >>>>>>> The functions need some documentation. >>>>>>> >>>>>>>> >>>>>>>> http://git-wip-us.apache.org/repos/asf/incubator-ponymail/blob/ae791c1a/site/js/coffee/scaffolding.coffee >>>>>>>> ---------------------------------------------------------------------- >>>>>>>> diff --git a/site/js/coffee/scaffolding.coffee >>>>>>>> b/site/js/coffee/scaffolding.coffee >>>>>>>> index ec58fee..379b463 100644 >>>>>>>> --- a/site/js/coffee/scaffolding.coffee >>>>>>>> +++ b/site/js/coffee/scaffolding.coffee >>>>>>>> @@ -118,7 +118,7 @@ threadScaffolding = () -> >>>>>>>> >>>>>>>> ### Make an API call to the preferences script, have it call back >>>>>>>> to listView once done ### >>>>>>>> mid = location.href.match(/thread\.html\/(.+)/)[1] >>>>>>>> - r = new HTTPRequest("api/thread.lua?id=" + mid, { >>>>>>>> + r = new HTTPRequest("api/thread.lua?id=" + unshortenURL(mid), { >>>>>>>> callback: scaffoldingEmailCallback >>>>>>>> >>>>>>>> }) >>>>>>>> >>>>>> >>>> >>
