Hi Keith, Yes, various hash algorithms, including SHA-1 and -2, are part of the OpenSSL library we're using under the covers. We have had internal discussions about exposing them at some point, but not seriously enough to get the work scheduled into any particular release.
I'll note this as some interest on your part. Wayne. On Mon, 2010-03-29 at 12:04 -0700, Keith L. Breinholt wrote: > True, request IDs are not unique unto themselves. That was the reason for an > MD5 hash of the identifier/URI, host name, timestamp and request ID ... to > reduce the odds of a collision. > > But MD5 is not without collisions either. :( > > On that subject: Mike, Walter or Wayne; is there any chance of getting a > SHA-1 or SHA-2 function in the near future? The server must have one > available since the implementation of HTTPS/SSL, right? > > Keith L. Breinholt > [email protected] > > > -----Original Message----- > From: [email protected] > [mailto:[email protected]] On Behalf Of Geert Josten > Sent: Monday, March 29, 2010 12:26 PM > To: General Mark Logic Developer Discussion > Subject: RE: [MarkLogic Dev General] Unique key construction > > Hi Keith, > > Note that the request ID is not guaranteed to be unique either (just like > xdmp:random). It has been discussed previously on this mailinglist here > http://markmail.org/thread/v326vhjvbuhitckp and this lengthy thread here > http://marklogic.markmail.org/thread/bxklh6dwoctcsjcy (further down from > where Michael Blakeley joins the thread). The second thread does contain some > code that keeps track of a counter, if I recall correctly.. > > Kind regards, > Geert > > > -----Original Message----- > > From: [email protected] > > [mailto:[email protected]] On Behalf Of > > Keith L. Breinholt > > Sent: maandag 29 maart 2010 19:24 > > To: General Mark Logic Developer Discussion > > Subject: RE: [MarkLogic Dev General] Unique key construction > > > > Good point. > > > > > > > > So, to guarantee a UUID that is unique to the URI and the > > transaction we then change the function to use the request > > key in the hash like this: > > > > > > > > let $hash := xdmp:md5( concat( $uri, > > xs:string(current-dateTime()), $namespace, > > xs:string(xdmp:request()) ) ) > > > > > > > > If we need a unique UUID for every call of the function we > > then need to keep a count that increments with each call of > > the function and tracks the clock ticks between calls. > > > > > > > > Keith L. Breinholt > > > > [email protected] <mailto:[email protected]> > > > > > > > > From: [email protected] > > [mailto:[email protected]] On Behalf Of > > Walter Underwood > > Sent: Monday, March 29, 2010 10:55 AM > > To: General Mark Logic Developer Discussion > > Subject: Re: [MarkLogic Dev General] Unique key construction > > > > > > > > Using timestamps plus a random number is a dangerous way to > > build UUIDs. The birthday paradox makes it pretty easy to > > generate dupes inbetween clock ticks. I know, because my UUID > > generator did that about 15 years ago. > > > > > > > > The usual solution requires that you know the resolution of > > your clock. Fill the low-order bits of the timestamp with a > > sequence number that increments for each UUID generated. When > > that sequence field hits the limit, you busy-wait on the > > clock until it ticks. > > > > > > > > That's what the DCE code did, and my non-unique UUIDs went > > away after I did it too. > > > > > > > > wunder > > > > == > > > > Walter Underwood > > > > [email protected] > > > > > > > > On Mar 29, 2010, at 9:35 AM, Darin McBeath wrote: > > > > > > > > > > > > Yes, that would likely be more efficient ... forgot that his > > function had been added :-) > > > > > > > > > > > > ________________________________ > > > > From: Michael Blakeley <[email protected]> > > To: General Mark Logic Developer Discussion > > <[email protected]> > > Cc: "McBeath, Darin W (ELS-STL)" <[email protected]> > > Sent: Mon, March 29, 2010 12:18:26 PM > > Subject: Re: [MarkLogic Dev General] Unique key construction > > > > Wouldn't it be more efficient to add xdmp:elapsed-time()? > > > > -- Mike > > > > On 2010-03-29 08:19, McBeath, Darin W (ELS-STL) wrote: > > > I believe this could be fixed (if necessary) by wrapping > > the current-dateTime() call with a xdmp:eval. > > > > > > xdmp:eval("current-dateTime()") > > > > > > Darin. > > > > > > -----Original Message----- > > > From: [email protected] on behalf of > > Geert Josten > > > Sent: Mon 3/29/2010 10:00 AM > > > To: General Mark Logic Developer Discussion > > > Subject: RE: [MarkLogic Dev General] Unique key construction > > > > > > Note: this code will return only one unique value per > > transaction. current-datetime() returns the same value > > throughout the transaction as MarkLogic Server is based on > > point-in-time querying.. > > > > > > Kind regards, > > > Geert > > > > > > ________________________________ > > > From: [email protected] > > [mailto:[email protected]] On Behalf Of > > Keith L. Breinholt > > > Sent: maandag 29 maart 2010 16:45 > > > To: General Mark Logic Developer Discussion > > > Subject: RE: [MarkLogic Dev General] Unique key construction > > > > > > > > > Here is a UUID version 3 implementation that we use for > > URLs that is unique to the name, host and time created: > > > > > > > > > > > > declare variable $g_defaultNamespace as xs:string := > > xdmp:host-name(xdmp:host()); > > > declare variable $g_uuidVersion as xs:unsignedLong := 3; > > > declare variable $g_uuidReserved as xs:unsignedLong := 8; > > > > > > (: > > > Calculate the UUID and set the UUID property on the file. > > > > > > The layout of a UUID is as follows. > > > 0 1 2 3 > > > 0 1 2 3 4 5 6 7 8 9 a b c d e f 0 1 2 3 4 5 6 7 8 9 a b c d e f > > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > > | time_low | > > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > > | time_mid | time_hi |version| > > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > > |clk_seq_hi |res| clk_seq_low | node (0-1) | > > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > > | node (2-5) | > > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > > 0 1 2 3 4 5 6 7 8 9 a b c d e f 0 1 2 3 4 5 6 7 8 9 a b c d e f > > > > > > This implements version 3 of the UUID specification. > > > The timestamp is a 60-bit value. > > > The clock sequence is a 14 bit value. > > > The node is a 48-bit name value. > > > :) > > > declare function meta:uuid( $uri as xs:string ) > > > as xs:string > > > { > > > meta:uuid( $uri, $g_defaultNamespace ) > > > }; > > > > > > declare function meta:uuid( $uri as xs:string, $namespace > > as xs:string ) > > > as xs:string > > > { > > > (: calculate md5 with a dateTime for our random values :) > > > let $hash := xdmp:md5( concat( $uri, > > xs:string(current-dateTime()), $namespace ) ) > > > return > > > concat ( > > > substring( $hash, 1, 15 ), > > > (: set version bits :) > > > xdmp:integer-to-hex($g_uuidVersion), > > > (: set reserved bits :) > > > substring( $hash, 17, 1 ), > > > > > xdmp:integer-to-hex((xdmp:hex-to-integer(substring($hash, 18, > > 1)) idiv 4) + $g_uuidReserved), > > > substring( $hash, 19, 14 ) > > > ) > > > }; > > > > > > > > > > > > > > > Keith L. Breinholt > > > > > > [email protected] > > > > > > > > > > > > > > > > > > -----Original Message----- > > > From: [email protected] > > [mailto:[email protected]] On Behalf Of > > Geert Josten > > > Sent: Monday, March 29, 2010 2:35 AM > > > To: General Mark Logic Developer Discussion > > > Subject: RE: [MarkLogic Dev General] Unique key construction > > > > > > > > > > > > Hi Deepak, > > > > > > > > > > > > Look in the Modules, not the Admin folder. Here is a > > typical case taken from triggers.xqy: > > > > > > > > > > > > define function > > > > > > get-unique-trigger-id() > > > > > > as xs:unsignedLong > > > > > > { > > > > > > let $col := triggers-collection(), > > > > > > $attempt := xdmp:random(), > > > > > > $doc-uri := fn:concat(triggers-uri(), xs:string($attempt)), > > > > > > $docs := for $d in fn:collection($col) return fn:base-uri($d) > > > > > > return > > > > > > if ($doc-uri = $docs) > > > > > > then get-unique-trigger-id() > > > > > > else $attempt > > > > > > } > > > > > > > > > > > > You can make it slightly more robust by counting the > > attempts and throwing an error if you need to many attempts. > > The number of unique numbers generated by xdmp:random is > > large though, so that is only necessary when you have good > > reason to believe that you could approach that limit.. > > > > > > > > > > > > Kind regards, > > > > > > Geert > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > drs. G.P.H. (Geert) Josten > > > > > > Consultant > > > > > > > > > > > > > > > > > > Daidalos BV > > > > > > Hoekeindsehof 1-4 > > > > > > 2665 JZ Bleiswijk > > > > > > > > > > > > T +31 (0)10 850 1200 > > > > > > F +31 (0)10 850 1199 > > > > > > > > > > > > mailto:[email protected] > > > > > > http://www.daidalos.nl/ > > > > > > > > > > > > KvK 27164984 > > > > > > > > > > > > P Please consider the environment before printing this mail. > > > > > > De informatie - verzonden in of met dit e-mailbericht - is > > afkomstig van Daidalos BV en is uitsluitend bestemd voor de > > geadresseerde. Indien u dit bericht onbedoeld hebt ontvangen, > > verzoeken wij u het te verwijderen. Aan dit bericht kunnen > > geen rechten worden ontleend. > > > > > > > > > > > >> From: [email protected] > > > > > >> [mailto:[email protected]] On Behalf Of > > > > > >> deepak mohan > > > > > >> Sent: zondag 28 maart 2010 17:44 > > > > > >> To: [email protected] > > > > > >> Subject: [MarkLogic Dev General] Unique key construction > > > > > >> > > > > > >> Hi All, > > > > > >> > > > > > >> Please tell me how ML constructs the unique ID while creating > > > > > >> any ML entities(App Servers, DB, forest etc.). I need to > > > > > >> generate a unique ID. I tried dig into the ML Admin modules > > > > > >> APi, I couldnot find the algo. Are they using random() and > > > > > >> check for existence? > > > > > >> > > > > > >> Thanks, > > > > > >> Deepak Mohanakrishnan. > > > > > >> > > > > > >> > > > > > >> ________________________________ > > > > > >> > > > > > >> Your Mail works best with the New Yahoo Optimized IE8. Get it > > > > > >> NOW! > > > > > >> > > <http://in.rd.yahoo.com/tagline_ie8_new/*http://downloads.yaho > > <http://in.rd.yahoo.com/tagline_ie8_new/*http:/downloads.yaho> > > > > > >> o.com/in/internetexplorer/> . > > > > > >> > > > > > > _______________________________________________ > > > > > > General mailing list > > > > > > [email protected] > > > > > > http://xqzone.com/mailman/listinfo/general > > > > > > > > > NOTICE: This email message is for the sole use of the > > intended recipient(s) and may contain confidential and > > privileged information. Any unauthorized review, use, > > disclosure or distribution is prohibited. If you are not the > > intended recipient, please contact the sender by reply email > > and destroy all copies of the original message. > > > > > > > > > _______________________________________________ > > > General mailing list > > > [email protected] > > > http://xqzone.com/mailman/listinfo/general > > > > _______________________________________________ > > General mailing list > > [email protected] > > http://xqzone.com/mailman/listinfo/general > > > > _______________________________________________ > > General mailing list > > [email protected] > > http://xqzone.com/mailman/listinfo/general > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > General mailing list > [email protected] > http://xqzone.com/mailman/listinfo/general > > > NOTICE: This email message is for the sole use of the intended recipient(s) > and may contain confidential and privileged information. Any unauthorized > review, use, disclosure or distribution is prohibited. If you are not the > intended recipient, please contact the sender by reply email and destroy all > copies of the original message. > > > _______________________________________________ > General mailing list > [email protected] > http://xqzone.com/mailman/listinfo/general
_______________________________________________ General mailing list [email protected] http://xqzone.com/mailman/listinfo/general
