Re: Why we cannot do this now?

[Erik Assum]

Using Cider and clj-refactor, you can do M-x cljr-add-project-dependency to add 
(or update) a dependency. 

Erik. 
-- 
i farta

> Den 22. jun. 2016 kl. 23.50 skrev Ritchie Cai :
> 
> Just curious, how do you add new dependencies to a running REPL? Do you just 
> restart REPL? I used to able to add new dependencies using alembic, but it 
> seems it's not really compatible with clojure 1.8, not an option anymore.
> 
> Thanks
> Ritchie
> 
>> On Monday, June 20, 2016 at 3:42:02 PM UTC-5, Sean Corfield wrote:
>> On 6/19/16, 4:11 PM, "Sungjin Chun" > chu...@castlesoft.co.kr> wrote: 
>> > yes, I found that java -jar clojure.jar thing is very fast compared to 
>> > boot repl 
>> 
>> It’s probably worth pointing out here that most developers’ typical workflow 
>> involves starting up a REPL and then just leaving it running for a long time 
>> so they don’t have to deal with the startup time very often. See things like 
>> Component et al and various people talking about the “Clojure Reloaded 
>> Workflow” in blog posts and conference talks. 
>> 
>> I use Emacs + CIDER and probably only start up a REPL two or three times a 
>> week. The rest of the time, I’m just eval’ing into an existing REPL process. 
>> 
>> Sean Corfield -- (904) 302-SEAN 
>> An Architect's View -- http://corfield.org/ 
>> 
>> "Perfection is the enemy of the good." 
>> -- Gustave Flaubert, French realist novelist (1821-1880)
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with your 
> first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] core.async 0.2.382

[Daniel Compton]

I knew I must be missing something. You can also set the core.async
threadpool size with

(System/setProperty "clojure.core.async.pool-size" "42")

as long as that runs before any core.async code tries to use the
threadpool. Some context on why the change was made to 8 would be good, but
it is easy to set this value yourself.

On Thu, Jun 23, 2016 at 9:13 AM Daniel Compton <
daniel.compton.li...@gmail.com> wrote:

> Bumping this too. What was the context for this change? I don’t see any
> JIRA tickets linked in the commits
> 
>  with
> a rationale for why 8 was picked as the thread pool size. This has the
> potential for breaking or reducing efficiency for apps that were relying on
> the 42 + 2 * cores behaviour. I’m happy that this is now configurable, but
> puzzled as to why the default wasn’t kept as 42 + 2 * cores?
>
> Also, because this needs to be passed as a java property, if we want our
> applications to be able to handle differing core counts (e.g. running on
> heterogenous servers) it seems like we will need to write a bash script to
> calculate our threadpool size before starting up.
>
> Perhaps I’m missing something really obvious here, if so, please let me
> know :)
>
> Tim Ewald, can you help with this?
>
> On Tue, Jun 14, 2016 at 9:10 AM Fluid Dynamics  wrote:
>
>> On Monday, June 13, 2016 at 4:14:25 PM UTC-4, Alex Miller wrote:
>>>
>>> core.async 0.2.382 is now available.
>>>
>>> Try it via:  [org.clojure/core.async "0.2.382"]
>>>
>>> 0.2.382 includes the following changes:
>>>
>>> - Change default dispatch thread pool max size to 8.
>>> - Add Java system property clojure.core.async.pool-size to override the
>>> dispatch thread pool max size
>>>
>>
>> Why 8, rather than, say, (.availableProcessors (Runtime/getRuntime))?
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> —
> Daniel
>
-- 
—
Daniel

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Joy of Clojure : Backward running lisp ??

[Baishampayan Ghose]

Running "backwards" here pertains to logic/relational programming in
MiniKanren/core.logic style. Roughly here programs are expressed in terms
of relations between the input and output. So given an input and an output
query you'll run it forwards and by making the input itself a variable with
a fixed output will generate a series of possible inputs, that'd be running
it backwards. Useful for generating programs :-)

Here is an example:
http://jvns.ca/blog/2013/11/20/day-31-logic-programming-pretty-music/

~BG

On Thu, Jun 23, 2016 at 7:52 AM, Ashish Negi 
wrote:

> I am reading joy of clojure. In the "forward to second edition" William E
> Byrd and Daniel P Firedman says :
>
>
>
> *As with recursion, the art of defining little languages encourages—and
> rewards—wishful thinking. You might think to yourself, “If only I had a
> language for expressing the rules for legal passwords for my login system.”
> A more involved example—a story, really—started several years ago, when we
> thought to ourselves, “If only we had the right relational language, we
> could write a Lisp interpreter that runs backward.”[2]
> 
> What does this mean? *
>
>
> *An interpreter can be thought of as a function that maps an input
> expression, such as (+ 5 1), onto a value—in this case, 6. We wanted to
> write an interpreter in the style of a relational database, in which either
> the expression being interpreted or the value of that expression, or both,
> can be treated as unknown variables. We can run the interpreter forward
> using the query (interpret ‘(+ 5 1) x), which associates the query variable
> x with the value 6. Better yet, we can run the interpreter backward with
> the query (interpret x 6), which associates x with an infinite stream of
> expressions that evaluate to 6, including (+ 5 1) and ((lambda (n) (* n 2))
> 3). (Brainteaser: determine the behavior of the query (interpret x x).)*
>
> Although the writer gave an example of `*(interpret x 6)*` i could not
> imagine the use case of `*lisp interpreter running backwards*` ?
> I am not even sure what he meant exactly.
>
> Thinking on it, i could only relate this to *theorem prover*s where you
> run backwards from the result.
> Can somebody explain this ?
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Baishampayan Ghose
b.ghose at gmail.com

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Joy of Clojure : Backward running lisp ??

[Ashish Negi]

I am reading joy of clojure. In the "forward to second edition" William E 
Byrd and Daniel P Firedman says :



*As with recursion, the art of defining little languages encourages—and 
rewards—wishful thinking. You might think to yourself, “If only I had a 
language for expressing the rules for legal passwords for my login system.” 
A more involved example—a story, really—started several years ago, when we 
thought to ourselves, “If only we had the right relational language, we 
could write a Lisp interpreter that runs backward.”[2] 
 
What does this mean? *


*An interpreter can be thought of as a function that maps an input 
expression, such as (+ 5 1), onto a value—in this case, 6. We wanted to 
write an interpreter in the style of a relational database, in which either 
the expression being interpreted or the value of that expression, or both, 
can be treated as unknown variables. We can run the interpreter forward 
using the query (interpret ‘(+ 5 1) x), which associates the query variable 
x with the value 6. Better yet, we can run the interpreter backward with 
the query (interpret x 6), which associates x with an infinite stream of 
expressions that evaluate to 6, including (+ 5 1) and ((lambda (n) (* n 2)) 
3). (Brainteaser: determine the behavior of the query (interpret x x).)*

Although the writer gave an example of `*(interpret x 6)*` i could not 
imagine the use case of `*lisp interpreter running backwards*` ?
I am not even sure what he meant exactly.

Thinking on it, i could only relate this to *theorem prover*s where you run 
backwards from the result.
Can somebody explain this ?

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Too over complicated

[Alan Thompson]

Olek - You may find some of the convience functions you are missing in the
Tupelo library.  In particular:

https://github.com/cloojure/tupelo#convenience-in-testing-seqs
https://github.com/cloojure/tupelo#keeping-it-simple-with-not-nil

Keeping It Simple with not-nil?

Clojure has the build-in function some to return the first *truthy value* from
a *sequence* argument. It also has the poorly named function some? which
returns the *value* true if a *scalar* argument satisfies (not (nil? arg)).
It is easy to confuse someand some?, not only in their return type but also
in the argument they accept (sequence or scalar). In keeping with the style
for other basic test functions, we provide the function not-nil? as the
opposite of nil?.

The unit tests show how not-nil? leads to a more natural code syntax:

(let [data [true :a 'my-symbol 1 "hello" \x false nil] ]
  (let [notties   (keep-if not-nil? data)
nillies   (drop-if not-nil? data) ]
(is (and  (= notties [true :a 'my-symbol 1 "hello" \x false] )
  (= nillies [nil] )))
(is (every?   not-nil? notties)); the 'not' can be used
(is (not-any? nil? notties)))   ;   in either first or 2nd positon

  (let [count-if (comp count keep-if) ]
(let [num-valid-1 (count-if some?data); awkward
phrasing, doesn't feel natural
  num-valid-2 (count-if not-nil? data); matches intent
much better
  num-nil (count-if nil? data) ]  ; intent is plain
  (is (and (= 7 num-valid-1 num-valid-2 )
   (= 1 num-nil))


Enjoy!
Alan



On Sat, Jun 18, 2016 at 5:56 PM, James Reeves  wrote:

> On 19 June 2016 at 00:07, Olek  wrote:
>>
>> Now lets talk about "difficulties" which I have found.
>> For example when we talk about removing element from collection.
>>
>> Why there is no one operation which could behave the same for set,
>> vector, list, string, and map? Let the community talk by themselves:
>> http://stackoverflow.com/questions/28844647/why-are-disj-and-dissoc-distinct-functions-in-clojure
>>
>
> I guess the difference is intent, but you'd have to ask Rich.
>
>
>> Why there is no one operation for checking if an element is in a
>> collection/keys of map/string?
>>
>> http://stackoverflow.com/questions/3249334/test-whether-a-list-contains-a-specific-value-in-clojure
>>
>
> You can use "some" to achieve this effect for most collections. Maps are a
> little tricky, because you could be checking for the existence of a key, a
> value, or a key-value pair.
>
> It might be nice to have an "includes?" function, though, particularly one
> that was more efficient.
>
>
>> There is no even simple method for saying that element is not a nil. This
>> requires from me the (not (nil? %)) combo.
>>
>
> There is. "clojure.core/some?" is the same as #(not (nil? #)).
>
> - James
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Why we cannot do this now?

[Sean Corfield]

On 6/22/16, 2:50 PM, "Ritchie Cai"  wrote:
> Just curious, how do you add new dependencies to a running REPL? Do you just 
> restart REPL?

Our project is pretty stable so we don’t add new dependencies very often (nor 
change existing ones), so yes, we just restart the REPL in the project context.

We also run a REPL server inside several of our long-running processes so we 
can actually update running server processes on the fly if we need to (it’s 
certainly very handy for analyzing and debugging any problems we run into in 
production!).

Sean Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood



-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: tips on writing modern idiomatic code

[Rangel Spasov]

Great book resources, IMO:

Clojure Programming (Emerick, Carper, Grand; O’Reilly) - book
Clojure Applied (Alex Miller, Ben Vandgrift) - book

If you understand the majority of what those books say and why, read Zach's 
Elements of Clojure.

General recommendations:
- understand why transducers, use them where appropriate (Rich's talks are 
good intros to 'why')
- understand why core.async, CSP in general, use it where appropriate 
(Rich's talks again)
- understand clojure.spec

In terms of code readability and documentation, I would aspire to one day 
write as much documentation as Aphyr does here:

https://github.com/aphyr/tesser/blob/master/core/src/tesser/core.clj 

I think there isn't a single idiomatic way to write Clojure. If there's a 
"way", it's probably the "Out of the tar pit 
paper" http://shaffner.us/cs/papers/tarpit.pdf 

My general code quality test is can you explain/draw your code decisions to 
a stranger on a piece of paper in 60 seconds. 

Cheers,
Rangel

On Tuesday, June 21, 2016 at 5:46:22 AM UTC-7, Sergey Didenko wrote:
>
> Hi,
>
> What would you advise for writing-rewriting your Clojure code in MODERN 
> idiomatic way?
>
> Using Kibit?
>
> Pasting your code samples on some review site?
>
> Asking help in IRC channel?
>
> Asking here?
>
> Reading some noticeable open source projects? 
>
> Reading some new Clojure book?
>
> I ask about the latest Clojure specifically. 
>
> I have not given very focused attention to Clojure since version 1.4 and 
> would like to grasp the WHOLE PICTURE of "good" modern Clojure. Currently 
> it feels like a lot of latest knowledge is located in different pieces all 
> over the internet. Or may be I just don't know where to look.
>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Why we cannot do this now?

[Ritchie Cai]

Just curious, how do you add new dependencies to a running REPL? Do you 
just restart REPL? I used to able to add new dependencies using alembic, 
but it seems it's not really compatible with clojure 1.8, not an option 
anymore.

Thanks
Ritchie

On Monday, June 20, 2016 at 3:42:02 PM UTC-5, Sean Corfield wrote:
>
> On 6/19/16, 4:11 PM, "Sungjin Chun"  
> on behalf of chu...@castlesoft.co.kr > wrote: 
> > yes, I found that java -jar clojure.jar thing is very fast compared to 
> boot repl 
>
> It’s probably worth pointing out here that most developers’ typical 
> workflow involves starting up a REPL and then just leaving it running for a 
> long time so they don’t have to deal with the startup time very often. See 
> things like Component et al and various people talking about the “Clojure 
> Reloaded Workflow” in blog posts and conference talks. 
>
> I use Emacs + CIDER and probably only start up a REPL two or three times a 
> week. The rest of the time, I’m just eval’ing into an existing REPL 
> process. 
>
> Sean Corfield -- (904) 302-SEAN 
> An Architect's View -- http://corfield.org/ 
>
> "Perfection is the enemy of the good." 
> -- Gustave Flaubert, French realist novelist (1821-1880) 
>
>
>
>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] core.async 0.2.382

[Daniel Compton]

Bumping this too. What was the context for this change? I don’t see any
JIRA tickets linked in the commits

with
a rationale for why 8 was picked as the thread pool size. This has the
potential for breaking or reducing efficiency for apps that were relying on
the 42 + 2 * cores behaviour. I’m happy that this is now configurable, but
puzzled as to why the default wasn’t kept as 42 + 2 * cores?

Also, because this needs to be passed as a java property, if we want our
applications to be able to handle differing core counts (e.g. running on
heterogenous servers) it seems like we will need to write a bash script to
calculate our threadpool size before starting up.

Perhaps I’m missing something really obvious here, if so, please let me
know :)

Tim Ewald, can you help with this?

On Tue, Jun 14, 2016 at 9:10 AM Fluid Dynamics  wrote:

> On Monday, June 13, 2016 at 4:14:25 PM UTC-4, Alex Miller wrote:
>>
>> core.async 0.2.382 is now available.
>>
>> Try it via:  [org.clojure/core.async "0.2.382"]
>>
>> 0.2.382 includes the following changes:
>>
>> - Change default dispatch thread pool max size to 8.
>> - Add Java system property clojure.core.async.pool-size to override the
>> dispatch thread pool max size
>>
>
> Why 8, rather than, say, (.availableProcessors (Runtime/getRuntime))?
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
-- 
—
Daniel

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Flake 0.4.0: Decentralized, k-ordered unique ID generator

[Max Countryman]

> On Jun 22, 2016, at 06:27, Bruno Bonacci  wrote:
> 
> @Brian, The clock drift problem was in answer to your question and generally 
> it was referring to the pure System/currentTimeMillis implementation. The 
> System/nanoTime, as it is now, is completely broken as it offsets against the 
> wall clock and not the previous System/nanotime, but that it is easy to fix.

Note that the current implementation mirrors Skuld’s.

When calculating a timestamp we provide an epoch, derived at startup. The epoch 
is constructed by taking a sampling of the difference between 
System/currentTimeMillis and System/nanoTime and averaging the results: this is 
meant to detect jitter. We don’t use the wall clock at any point after that for 
constructing flakes.

Because timestamps are persisted to disk, the next time we run our generator, 
we ensure that the epoch used is always larger than the last timestamp written. 
(We could also use the epoch to derive a timestamp and ensure the derived 
timestamp is larger, which might be better thinking about it now.) This 
protects against clock skew between runs.

To address your question about clock skew during runs, it’s true that 
System/nanoTime is not necessarily monotonic. However the process of generating 
a new flake always rejects timestamps that appear in the past relative to the 
last flake. So a monotonic property is built into our program. As you note, 
this could result in thrashing if the delta is extreme or System/nanoTime is 
somehow always returning smaller values—but I don’t know of a better way of 
addressing that issue currently.

I think the salient point here though is that we do enforce a monotonic 
property at the application level which should protect us against duplicate IDs.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Flake 0.4.0: Decentralized, k-ordered unique ID generator

[Bruno Bonacci]

On Wed, Jun 22, 2016 at 2:17 PM, Brian Platz 
wrote:

>
>
> On Wednesday, June 22, 2016 at 7:25:50 AM UTC-4, Bruno Bonacci wrote:
>>
>>
>> To answer Brian on the "potential" problem of the clock drift I would
>> recommend to have a look to
>> https://aphyr.com/posts/299-the-trouble-with-timestamps. Beside the
>> hardware problems you have to account for things like ntpd daemon which is
>> meant to synchronize clocks.
>> To keep them in sync it accelerates or decelerates the clock speed on
>> your machine, however if it falls too much behind it will do a hard-reset,
>> so what you might see is that your System/currentTimeMillis calls jump
>> back and forward (non-monotonic).
>> With VMs (such as cloud environment) this problem is way worse and more
>> frequent.
>>
>
> In Max's library he is only calling out to System/currentTimeMillis once
> at startup, he then determines the nanoTime offset and only uses nanoTime
> from there. nanoTime is supposed to be immune from wall clock time, and
> thus ntpd changes.
>
> Because it will ignore ntpd changes, there could be a delta from wall
> clock as ntpd changes the time. So that is a risk to be aware of, and if
> you were super concerned about it a method to re-sync could probably be
> devised (similar to what is used for leap-seconds). I've noticed this
> particular issue with a sleeping laptop. Assuming the process isn't
> extremely long-lived I think you'd be sufficiently close to not worry about
> it.
>


@Brian, The clock drift problem was in answer to your question and
generally it was referring to the pure System/currentTimeMillis
implementation. The System/nanoTime, as it is now, is completely broken as
it offsets against the wall clock and not the previous System/nanotime, but
that it is easy to fix.



>
> The process ID in many platform is just 16bits, however some platform have
>> 32bits (
>> http://unix.stackexchange.com/questions/16883/what-is-the-maximum-value-of-the-pid-of-a-process),
>> and the ThreadID in java is a long (
>> https://docs.oracle.com/javase/7/docs/api/java/lang/Thread.html#getId()
>> )
>> which is unfortunate. It would be nice if there was a way to read which CPU
>> core physical thread is executing a particular thread (when the thread is
>> running), i such way you could replace the processId and the threadId which
>> just a CPU core ID. The number of physical threads (core + hyperthreads)
>> is typically much smaller and it fits in a 16 bits. However to my
>> knowledge there is no way in java to retrieve such value.
>>
>
> If you were willing to take on another 64 bits, an idea could be to take
> the last 16 (or maybe 32) bits from the ThreadID combined with a random
> bits to round out the 64.
>
> The random bits could be done once per thread combined with keeping a
> ThreadLocal counter where a CAS is still done to avoid issuing the same ID
> at the same time increment in the same thread -- or the counter could be
> ignored entirely and the random bits generated with every ID. I'm not sure
> which would perform better, but I like the randomness per ID which makes
> IDs harder to guess.
>

I thought about adding randomness too, but not too sure that it will
actually solve any problem at all rather than creating new ones (true
random generator vs hybrid ones). Need more thinking on this.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

ANN: ClojureScript 1.9.89 - cljs.spec & :preloads

[David Nolen]

ClojureScript, the Clojure compiler that emits JavaScript source code.

README and source code: https://github.com/clojure/clojurescript

Leiningen dependency information:

[org.clojure/clojurescript "1.9.89"]

This release fixes several cljs.spec issues. It also includes a new
compiler option - :preloads. This feature is designed to simplify (but
not limited to) the loading of development time side effects such as
enabling printing, REPL connections, and browser tool
integration. With :preloads you may now specify a sequence of
namespaces as symbols to load in order immediately after cljs.core. This
also
means there's a path for providing typical boilerplate as documented
namespaces that users can specify in their dev builds.

As always feedback welcome!

## 1.9.89

### Enhancements
* CLJS-1688: :preloads compiler option for loading other entry points prior
to :main
* cljs.spec - support gen overrides by name in addition to path
* cljs.spec - every and every-kv

### Changes
* added bounded-count

### Fixes
* missing cljs.spec/fn-specs -> cljs.spec/get-spec in cljs.spec.test ns
* CLJS-1687: Self-host: cljs.spec: inst-in-range? and int-in-range? need
qualification
* CLJS-1668: cljs.spec: c alias needs expansion in int-in

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Flake 0.4.0: Decentralized, k-ordered unique ID generator

[Brian Platz]

On Wednesday, June 22, 2016 at 7:25:50 AM UTC-4, Bruno Bonacci wrote:
>
>
> To answer Brian on the "potential" problem of the clock drift I would 
> recommend to have a look to 
> https://aphyr.com/posts/299-the-trouble-with-timestamps. Beside the 
> hardware problems you have to account for things like ntpd daemon which is 
> meant to synchronize clocks.
> To keep them in sync it accelerates or decelerates the clock speed on your 
> machine, however if it falls too much behind it will do a hard-reset,
> so what you might see is that your System/currentTimeMillis calls jump 
> back and forward (non-monotonic).
> With VMs (such as cloud environment) this problem is way worse and more 
> frequent.
>

In Max's library he is only calling out to System/currentTimeMillis once at 
startup, he then determines the nanoTime offset and only uses nanoTime from 
there. nanoTime is supposed to be immune from wall clock time, and thus 
ntpd changes. 

Because it will ignore ntpd changes, there could be a delta from wall clock 
as ntpd changes the time. So that is a risk to be aware of, and if you were 
super concerned about it a method to re-sync could probably be devised 
(similar to what is used for leap-seconds). I've noticed this particular 
issue with a sleeping laptop. Assuming the process isn't extremely 
long-lived I think you'd be sufficiently close to not worry about it.

The process ID in many platform is just 16bits, however some platform have 
> 32bits (
> http://unix.stackexchange.com/questions/16883/what-is-the-maximum-value-of-the-pid-of-a-process),
>  
> and the ThreadID in java is a long (
> https://docs.oracle.com/javase/7/docs/api/java/lang/Thread.html#getId() 
> )
>  
> which is unfortunate. It would be nice if there was a way to read which CPU 
> core physical thread is executing a particular thread (when the thread is 
> running), i such way you could replace the processId and the threadId which 
> just a CPU core ID. The number of physical threads (core + hyperthreads) 
> is typically much smaller and it fits in a 16 bits. However to my 
> knowledge there is no way in java to retrieve such value.
>

If you were willing to take on another 64 bits, an idea could be to take 
the last 16 (or maybe 32) bits from the ThreadID combined with a random 
bits to round out the 64.

The random bits could be done once per thread combined with keeping a 
ThreadLocal counter where a CAS is still done to avoid issuing the same ID 
at the same time increment in the same thread -- or the counter could be 
ignored entirely and the random bits generated with every ID. I'm not sure 
which would perform better, but I like the randomness per ID which makes 
IDs harder to guess.

-Brian

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Flake 0.4.0: Decentralized, k-ordered unique ID generator

[Bruno Bonacci]

Hi,

Yes adding more bits to the id is certainly undesirable, however it isn't
uncommon. For example Linked just published a paper on Ambry a distributed
datastore which uses a total of *40 bytes* to identify a blob (8 bytes for
the partition + 32 bytes for UUID, see:
http://dprg.cs.uiuc.edu/docs/SIGMOD2016-a/ambry.pdf)

To answer Brian on the "potential" problem of the clock drift I would
recommend to have a look to
https://aphyr.com/posts/299-the-trouble-with-timestamps. Beside the
hardware problems you have to account for things like ntpd daemon which is
meant to synchronize clocks.
To keep them in sync it accelerates or decelerates the clock speed on your
machine, however if it falls too much behind it will do a hard-reset,
so what you might see is that your System/currentTimeMillis calls jump back
and forward (non-monotonic).
With VMs (such as cloud environment) this problem is way worse and more
frequent.

The process ID in many platform is just 16bits, however some platform have
32bits (
http://unix.stackexchange.com/questions/16883/what-is-the-maximum-value-of-the-pid-of-a-process),
and the ThreadID in java is a long (
https://docs.oracle.com/javase/7/docs/api/java/lang/Thread.html#getId())
which is unfortunate. It would be nice if there was a way to read which CPU
core physical thread is executing a particular thread (when the thread is
running), i such way you could replace the processId and the threadId which
just a CPU core ID. The number of physical threads (core + hyperthreads)
is typically much smaller and it fits in a 16 bits. However to my knowledge
there is no way in java to retrieve such value.

The other consideration to make is that even if running your benchmarks you
can get over 1 million IDs per second on a single thread,
the required coordination of the CAS and the retry formula sent earlier
will greatly reduce the performances as you scale the number of threads.
This falls under the Amdahl's law (
https://en.wikipedia.org/wiki/Amdahl%27s_law) and more precisely under the
USL Universal Scalability Law (
http://www.perfdynamics.com/Manifesto/USLscalability.html and
https://www.infoq.com/articles/top-10-performance-mistakes) for which as
you increase the number of processor the performance hits a plateau
(Amdahl) or even degrades (USL). If you run these tests I'm pretty sure
you'll see similar effects.

As such is impossible to generate TOTALLY unique IDs without coordination,
the real question is that what is the price of the coordination for a given
risk of collisions.

I guess each project should decide whether it is more important the
monotonicity or the performance.

@Max
In your latest version there is a subtle issue here:
https://github.com/maxcountryman/flake/blob/master/src/flake/utils.clj#L43
and
https://github.com/maxcountryman/flake/blob/master/src/flake/utils.clj#L56.
System/nanoTime has NO RELATION with the wall clock and i must not be used
in such way.
System/nanoTime can have also negative values (this is platform dependent)
it is guaranteed to be monotonic but not in respect of
System/currentTimeMillis. The change i was suggesting is like following:

  * at init! you initialize your base timestamp with "start-timestamp =
(System/currentTimeMillis) and in addition you create a new field called
"timer = (system/nanoTime)"
  * when you create a new flake instead of reading the new value of
System/currentTimeMillis you use a new call (system/nanoTime) and you check
the time elapsed since "timer" so new-timestap = (start-timestamp +
((system/nanoTime) - timer) / 100)

There is another potential issue with the generate! in case of clock hard
reset. If a reset happens exception will be thrown in tight loop
potentially causing a DOS.

Bruno

On Wed, Jun 22, 2016 at 12:29 AM, Max Countryman  wrote:

> Brian,
>
> I think you make good points here, especially with regard to the size of
> IDs.
>
> I’d also like to point out that while adding the process and thread IDs
> helps, it doesn’t eliminate the possibility of duplicate IDs: this is why
> it’s necessary to write out the last used timestamp in a separate thread.
>
> Just a clarification with regard to disk persistence: we aren’t writing
> out the epoch, we’re writing out the last used timestamp periodically, in
> its own thread. Yes, the `init!` API is cumbersome, but it’s an important
> safety valve which helps protect against duplicate IDs.
>
> My understanding from reading the documentation and various StackOverflow
> answers is that System/nanoTime is monotonic, but I don’t know what
> guarantees it makes across threads.
>
>
> Max
>
>
> On Jun 21, 2016, at 10:00, Brian Platz  wrote:
>
> Bruno,
>
> I think the more you can reduce the chance of collision the better and the
> thread-local capability is a good idea, but in the process you've almost
> doubled the bits.
>
> For me anyhow, an ID need to be produceable at a reasonable rate (1
> million a second per machine is good for me), have near-zero probability of
> co