Re: Understanding GraalVM and Clojure

[Jason Kapp]

Thank you for this detailed explanation.

On Thursday, April 19, 2018 at 11:55:25 AM UTC-6, tbc++ wrote:
>
> GraalVM does a lot of things, and I think it's important to separate these 
> terms. 
>
> GraalVM - most often this is used to refer to a project that was 
> originally designed to be a implementation of the JVM in Java. So when 
> people ask "does X run on GrallVM" the question is really "does X run in 
> the JVM". Clojure runs on the JVM therefore it runs on GraalVM. I've done 
> this, and it's not that hard to setup. 
>
> Truffle - is an AST interpreter framework that allows for highly dynamic 
> languages to run efficiently on GraalVM. Truffle is valid Java code, so you 
> can run Truffle on a stock JVM, but it's much faster on GraalVM (or on JVM 
> of version >= 9). Notice my use of "highly dynamic" earlier. Surprisingly, 
> Clojure is mostly static, so there's no clear win here to translating 
> Clojure to Truffle. The exception to this is primitive math, and situations 
> that use lots of HOF where Truffle could perhaps offer more localized 
> optimizations that fit into the Clojure programming model. However, you pay 
> for all this with some rather massive startup time penalties. Most examples 
> I've seen show Truffle adding seconds on to the startup time of a language.
>
> SubstrateVM - (aka native-image), SubstrateVM attempts to improve the 
> performance of a Truffle based language by doing image freezing. This 
> method has been used in other languages, and has existed in the PyPy 
> toolchain (RPython) for well over a decade. The idea is that you write an 
> interpreter, then hand SVM a pointer to the start of your interpreter (the 
> main() function). The framework then analyzes the data needed by that 
> function and all the functions it calls. All data required by those 
> functions are also recorded. Then the call graph required to run all these 
> functions is written out (normally to C or C++) and compiled. The 
> side-effect is that all the startup time is removed since the interpreter 
> is "frozen" in a started state. In the terms of Clojure this means that 
> metadata would be serialized as-is, instead of running the code to create 
> that metadata on every startup. 
>
> Polyglot VM - so in truffle you can have multiple type systems, and 
> multiple languages all running on Truffle. The framework then allows cheap 
> interop between these languages. So when GraalVM docs talk about "zero 
> overhead interop" what they mean is that it's possible to inline a Truffle 
> based function inside any other truffle based function. Since Truffle 
> implementations exist for Ruby, Python, LLVM bytecode (think C/C++), 
> JavaScript, etc. It's easy to see how its possible to have all of these 
> languages efficiently calling each other on the same VM. 
>
> It's not all awesome though. By using SubstrateVM you give up Java 
> reflection/interop, or have to find a way to embed a full JVM as a Truffle 
> language (this is being worked on), but every language you add into your 
> SVM image comes at a cost of startup times, memory usage etc. So most of 
> the time SVM images stick to a few languages. TruffleRuby for example only 
> uses Ruby and LLVM (for c interop).
>
> To finish, I think Clojure on Truffle is possible, but it would take a TON 
> of work. Basically most of clojure.lang.RT would need to be rewritten from 
> scratch, and I'm not sure how much more in clojure.lang.* would also need 
> to be rewritten. So the tradeoff is:
>
> A. Current state
> - Good enough performance
> - Fast enough startup (Clojure starts quickly, it's the deps/tooling that 
> are slow)
> - Requires type hinting to avoid reflection
>
> B. Clojure on Truffle
> - More-or-less the same performance 
> - Faster un-type-hinted math
> - (Possibly) Faster transducers/HOF over primitive collections
> - No need for type hints
> - Massive rewrite
> - Huge startup time penalty
> - Requires a modern/uncommon JVM (JVM9+ or GraalVM)
>
> Comparing these side-by-side, it looks to me to be a wash. And because of 
> that I doubt we'll see a TruffleClojure any time soon.
>
> Timothy
>
> On Thu, Apr 19, 2018 at 4:00 AM, Khalid Jebbari  > wrote:
>
>> Hello,
>>
>> Oracle has just announced GraalVM 1.0 release candidate: 
>> https://blogs.oracle.com/developers/announcing-graalvm
>>
>> It mentions a few JVM-based language but not Clojure (maybe just because 
>> of popularity).
>> - Does it mean Clojure is not "compatible" with GraalVM ? 
>> - Does it mean Clojure needs to be reimplemented in terms of GraalVM's 
>> Truffle framework ?
>> - Does it mean Clojure can be run as a native binary with fast startup 
>> time and minimized memory footprint ?
>>
>> If someone with some knowledge could explain to me the relationships 
>> between Clojure and GraalVM ? Bonus points if Alex Miller answers and share 
>> the plans, if any, about their integration/interaction.
>>
>> Thanks a lot in advance. really curious to 

Re: [ANN] Schism, a set of CRDTs for Clojure and ClojureScript

[Alex Redington]

Another good question, and the answer is "it depends". As a guiding 
principle for the expectations of this abstraction, convergence should 
yield a collection as if it had all of the operations applied to it in the 
order that they were applied, locally. So let's work with M, and nodes A 
and B.

A creates M as a simple map {:foo true} and replicates it to B. A has M 
({:foo true}), and B has M' ({:foo true}).
A performs (assoc M :bar false}. A has M ({:foo true, :bar false}) and B 
has M' ({:foo true}).
B performs (dissoc M' :foo). A has M ({:foo true, :bar false}) and B has M' 
({}).
B sends M' to A, and A synchronizes it. A has M ({:bar false}). B has M' 
({}).
A sends M to B, and B synchronizes it. A has M ({:bar false}). B has M' 
({:bar false}).

That's a simple case because A and B are modifying different keys and 
converge to the same value without contention over those keys. But what if 
A had done (assoc M :foo false) instead? There's still a synchronization 
conflict between the two, but the key :foo has been put in contention 
between A and B. In schism, this is resolved by the last writer winning, 
because that is what would happen on an isolated node working with a data 
structure locally. 

On Thursday, April 19, 2018 at 1:46:41 PM UTC-4, John Newman wrote:
>
> Alex, yeah that explains it for me. I'll probably want to use a fully 
> connected address space, via a mesh or a hub and spoke topology, that I 
> fully manage - so this is the perfect level of abstraction for me. I'd like 
> to see that ZQ implementation though.
>
> So, to be clear, if node A sends schism-map M to node B, both A and B 
> update M, B sends M back to A, A converges M1 and M2: If M1 and M2 have 
> destructive conflicts, does the most "recent" change win? Or does the local 
> copy win?
>
> Thanks,
>
> V/r
>
> John
>
> John
>
> On Thu, Apr 19, 2018 at 8:55 AM, Alex Redington  > wrote:
>
>> I'll try to answer this question and John's at the same time.
>>
>> Schism does not try to manage state for you over time; there are a lot of 
>> good tools for doing that already (atoms, refs, agents, channels, etc). 
>> Schism is a set of collections that augment basic clojure collections with 
>> enough additional information that you can take two persistent collections 
>> with common ancestry and merge them together. So, you could take a schism 
>> map held in an atom, dereference it and send it from a pedestal process to 
>> an om application as edn, and read-string it in the om app to have a 
>> replicated copy of the map. The om app could make some changes (dissoc one 
>> key, update another) without communicating to the pedestal app for each 
>> operation. Then send it back to the pedestal app in a POST body, and the 
>> pedestal app would swap! the atom using schism.core/converge and the value 
>> it just received from om. The changes, both additive and destructive, would 
>> be replicated in the atom's value.
>>
>> So, yes, you could build a serverless chatroom where each client held an 
>> atom with a schism collection, and they sent their copies around using 
>> WebRTC to stay in sync. You could push a schism edn serialization over a 
>> message queue to be synchronized on the other end. I anticipate pushing 
>> each discrete state over the mq with an add-watch hook might yield bad 
>> performance for little semantic gain. Schism collections are necessarily 
>> more expensive than Clojure collections in serialization size, and 
>> synchronization is where all of the signficant computational expense of 
>> these kinds of data structures resides, so doing it less frequently than 
>> every discrete update will probably be best. (A debounced send & sync that 
>> guaranteed transmission after n milliseconds of being inert would make 
>> sense to me) 
>>
>> I'm not presently confronting a problem that these solve in my day job; 
>> my intent was to build and have the tool ready if and when we wanted to 
>> have better answers for "offline sync" in a single page ClojureScript 
>> application.
>>
>> Thank you for the great questions!
>>
>> -Alex
>>
>> On Wednesday, April 18, 2018 at 9:55:36 PM UTC-4, Luke Burton wrote:
>>>
>>>
>>> This is cool! 
>>>
>>> There seems to be some room for a recommendation on how to apply these 
>>> data structures effectively in practice, especially around the distribution 
>>> mechanism. For instance, I can imagine storing these in an atom and using 
>>> watchers to push changes out over zero-mq. Or is that the Worst Idea Ever™? 
>>> I feel like the transit layer probably has *some* impact on whether CRDTs 
>>> are the right solution to the problem, and I'm interested to know what 
>>> those boundaries are.
>>>
>>> Have you used them in production over the network, and if so, what 
>>> solution did you use?
>>>
>>> Thanks again!
>>>
>>>
>>>
>>> On Apr 18, 2018, at 5:57 PM, Alex Redington  wrote:
>>>
>>> Good evening!
>>>
>>> I submit for your evaluation and 

Re: Understanding GraalVM and Clojure

[Nathan Fisher]

I was thinking it would be interesting to “remove” the use of Java interop
in core instead replacing it with a reserved namespace that maps to
whatever the underlying runtime/reader wants to. I suppose you can do the
same with the exisiting RT stuff but naively feels like it would be
cleaner/provide easier porting if it were one namespace which could have
Clojure docs generated for it.

On Thu, Apr 19, 2018 at 2:55 PM, Timothy Baldridge 
wrote:

> GraalVM does a lot of things, and I think it's important to separate these
> terms.
>
> GraalVM - most often this is used to refer to a project that was
> originally designed to be a implementation of the JVM in Java. So when
> people ask "does X run on GrallVM" the question is really "does X run in
> the JVM". Clojure runs on the JVM therefore it runs on GraalVM. I've done
> this, and it's not that hard to setup.
>
> Truffle - is an AST interpreter framework that allows for highly dynamic
> languages to run efficiently on GraalVM. Truffle is valid Java code, so you
> can run Truffle on a stock JVM, but it's much faster on GraalVM (or on JVM
> of version >= 9). Notice my use of "highly dynamic" earlier. Surprisingly,
> Clojure is mostly static, so there's no clear win here to translating
> Clojure to Truffle. The exception to this is primitive math, and situations
> that use lots of HOF where Truffle could perhaps offer more localized
> optimizations that fit into the Clojure programming model. However, you pay
> for all this with some rather massive startup time penalties. Most examples
> I've seen show Truffle adding seconds on to the startup time of a language.
>
> SubstrateVM - (aka native-image), SubstrateVM attempts to improve the
> performance of a Truffle based language by doing image freezing. This
> method has been used in other languages, and has existed in the PyPy
> toolchain (RPython) for well over a decade. The idea is that you write an
> interpreter, then hand SVM a pointer to the start of your interpreter (the
> main() function). The framework then analyzes the data needed by that
> function and all the functions it calls. All data required by those
> functions are also recorded. Then the call graph required to run all these
> functions is written out (normally to C or C++) and compiled. The
> side-effect is that all the startup time is removed since the interpreter
> is "frozen" in a started state. In the terms of Clojure this means that
> metadata would be serialized as-is, instead of running the code to create
> that metadata on every startup.
>
> Polyglot VM - so in truffle you can have multiple type systems, and
> multiple languages all running on Truffle. The framework then allows cheap
> interop between these languages. So when GraalVM docs talk about "zero
> overhead interop" what they mean is that it's possible to inline a Truffle
> based function inside any other truffle based function. Since Truffle
> implementations exist for Ruby, Python, LLVM bytecode (think C/C++),
> JavaScript, etc. It's easy to see how its possible to have all of these
> languages efficiently calling each other on the same VM.
>
> It's not all awesome though. By using SubstrateVM you give up Java
> reflection/interop, or have to find a way to embed a full JVM as a Truffle
> language (this is being worked on), but every language you add into your
> SVM image comes at a cost of startup times, memory usage etc. So most of
> the time SVM images stick to a few languages. TruffleRuby for example only
> uses Ruby and LLVM (for c interop).
>
> To finish, I think Clojure on Truffle is possible, but it would take a TON
> of work. Basically most of clojure.lang.RT would need to be rewritten from
> scratch, and I'm not sure how much more in clojure.lang.* would also need
> to be rewritten. So the tradeoff is:
>
> A. Current state
> - Good enough performance
> - Fast enough startup (Clojure starts quickly, it's the deps/tooling that
> are slow)
> - Requires type hinting to avoid reflection
>
> B. Clojure on Truffle
> - More-or-less the same performance
> - Faster un-type-hinted math
> - (Possibly) Faster transducers/HOF over primitive collections
> - No need for type hints
> - Massive rewrite
> - Huge startup time penalty
> - Requires a modern/uncommon JVM (JVM9+ or GraalVM)
>
> Comparing these side-by-side, it looks to me to be a wash. And because of
> that I doubt we'll see a TruffleClojure any time soon.
>
> Timothy
>
> On Thu, Apr 19, 2018 at 4:00 AM, Khalid Jebbari 
> wrote:
>
>> Hello,
>>
>> Oracle has just announced GraalVM 1.0 release candidate:
>> https://blogs.oracle.com/developers/announcing-graalvm
>>
>> It mentions a few JVM-based language but not Clojure (maybe just because
>> of popularity).
>> - Does it mean Clojure is not "compatible" with GraalVM ?
>> - Does it mean Clojure needs to be reimplemented in terms of GraalVM's
>> Truffle framework ?
>> - Does it mean Clojure can be run as a native binary with fast 

Re: Clojurists Together Q2 2018 Call for Proposals

[Daniel Compton]

Hi folks

Clojurists Together Q2 funding applications close on 22 April, so if you
were thinking of applying ,
now is the time. We haven't had too many applications yet, it would be
great to get some more from library and tool maintainers.

Thanks, Daniel.

On Wed, Apr 11, 2018 at 9:16 AM Daniel Compton <
daniel.compton.li...@gmail.com> wrote:

> Hi folks
>
> Clojurists Together is an organisation dedicated to funding critical open
> source Clojure projects. We fund projects on a 3-month cycle. Funding
> varies based on member support, but in our current cycle we were able to
> fund clj-http and Figwheel $1800USD/mo each for three months. Both projects
> have made amazing progress, you can read more about them in our Feb
>  and
> March 
> updates.
>
> We're gearing up for our next cycle and are soliciting proposals from the
> Clojure/ClojureScript community. We recently surveyed
>  our members
> to see what they wanted us to fund. The main areas they were focused on
> were documentation, build tooling, and libraries. *If you maintain a
> Clojure project that is important to some or all of the Clojure community
> please apply  for our second
> funding round.*
>
> *If you run or manage a company that relies on the Clojure open source
> ecosystem, we'd encourage you to consider joining as a company member
> . *We've had a great response from
> individual developers, but only a handful of companies have joined as
> sponsors. A very special thanks to all of our members
> , we couldn't do any of this
> without your support.
>
> Let us know if you have any questions/concerns, either here or at
> h...@clojuriststogether.org
>
> Thanks, Daniel.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] clj-new -- creating new Clojure projects using the clj CLI

[John Newman]

These are nice ideas... Sort of like nmp. But you could implement this
yourself as a lib and stick it behind a `clj -A:install ...` tool.

@dominicm (on clojurians.slack) has build an "injector" tool into pack,
which you could reimplement into something that you're describing:
https://github.com/juxt/pack.alpha/blob/master/src/mach/pack/alpha/inject.clj

Who knows, maybe if some of these aliases get super popular and
standardized, we can convince Alex to fold some of them into the tools over
time :)

V/r

John

John

On Thu, Apr 19, 2018 at 3:18 AM, Didier  wrote:

> Ah, awesome.
>
> I guess I'm starting to think, it would be great if clj supported
> something like:
>
> clj --install alias-name alias-url
>
> So maybe in a repo, you could put a file of a given convention in the root
> with a common name. Clj could pull it, it would contain a clj alias in it,
> and clj would auto add it to your deps.edn file as an alias. You could add
> a -g option to add it to your global deps.edn, if you wanted to "install"
> it globally.
>
> Nothing else would happen.
>
> But then, you could run your newly "installed" clojure program by doing:
>
> clj -A:alias-name args ...
>
> If you omit the alias name, it could default to something specified in the
> alias file.
>
> What that alias url resolves too, I'm not sure. I'm thinking a git url to
> an alias-install.edn file which contains something like:
>
> {:default-alias-name alias-map-to-add-as-an-alias}
>
>
> And similarly, you could add:
>
> clj --uninstall alias-name
>
> And it would remove the alias from your deps.edn, or if given -g, from
> your global deps.edn config.
>
> Ideally, it would remember the alias-url in the deps.edn file, so you
> could run:
>
> clj --update alias-name
>
> Maybe there could even be a kind of global url repo, with a list of
> registered alias names, so one could do:
>
> clj --install alias-name
>
> And it would know where to pull the alias-install.edn file from based on
> some global repo of distributed Clojure programs.
>
> I'd love this. Would seem like a pretty great way to deliver Clojure
> programs, which does not require any bash script, adding anything to your
> PATH, or having to edit a file.
>
> So for clj-new, ideally one could do:
>
> clj --install -g clj-new
>
> And clj would take care of adding the alias for it in my global deps.edn.
>
> But at the very least, without the alias registry, once could simply do:
>
> clj --install -g clj-new https://github.com/seancorfield/clj-new.git
>
> Which would auto-install the alias from a top level alias-install.edn file
> inside the repo.
>
> Something of the sort.
>
>
>
> On Tuesday, 17 April 2018 20:06:16 UTC-7, Sean Corfield wrote:
>>
>> clj-new -- https://github.com/seancorfield/clj-new
>>
>>
>>
>> This will generate new projects for you, either based on `clj`, or from
>> any existing Leiningen or Boot template (or, I hope in the future,
>> `clj-template` projects!).
>>
>>
>>
>> You'll probably want to add clj-new as an alias in your
>> ~/.clojure/deps.edn like this:
>>
>>
>>
>> {:aliases
>>
>> {:new {:extra-deps {seancorfield/clj-new
>>
>>  {:git/url "https://github.com/seancorfield/clj-new;
>>
>>   :sha "492bb2e7ad7373a8b5958124a86cddc4c7a123d5"}}
>>
>> :main-opts ["-m" "clj-new.create"]}}
>>
>> ...}
>>
>>
>>
>> Create a basic application:
>>
>>
>>
>> clj -A:new app myname/myapp
>>
>> cd myapp
>>
>> clj -m myname.myapp
>>
>>
>>
>> Run the tests:
>>
>>
>>
>> clj -A:test:runner
>>
>>
>>
>> Yes, the `deps.edn` generated from the `app` (and `lib`) built-in
>> template includes aliases to include your `test` folder, add `test.check`
>> as a dependency, and bring in and run Cognitect's `test-runner`
>>
>>
>>
>> The project name should either be a qualified Clojure symbol or a
>> multi-segment name -- single segment project names are not allowed!
>>
>>
>>
>> For a qualified Clojure symbol, the first part is typically your GitHub
>> account name or your organization's domain reversed, e.g., `com.acme`, and
>> the second part is the "local" name for your project (and is used as the
>> name of the folder in which the project is created).
>>
>>
>>
>> For a multi-segment project name, such as `foo.bar`, the folder that will
>> be created would be called `foo.bar` and will contain `src/foo/bar.clj`.
>>
>>
>>
>> Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
>> An Architect's View -- http://corfield.org/
>>
>> "If you're not annoying somebody, you're not really alive."
>> -- Margaret Atwood
>>
>>
>>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> 

Re: Understanding GraalVM and Clojure

[Timothy Baldridge]

GraalVM does a lot of things, and I think it's important to separate these
terms.

GraalVM - most often this is used to refer to a project that was originally
designed to be a implementation of the JVM in Java. So when people ask
"does X run on GrallVM" the question is really "does X run in the JVM".
Clojure runs on the JVM therefore it runs on GraalVM. I've done this, and
it's not that hard to setup.

Truffle - is an AST interpreter framework that allows for highly dynamic
languages to run efficiently on GraalVM. Truffle is valid Java code, so you
can run Truffle on a stock JVM, but it's much faster on GraalVM (or on JVM
of version >= 9). Notice my use of "highly dynamic" earlier. Surprisingly,
Clojure is mostly static, so there's no clear win here to translating
Clojure to Truffle. The exception to this is primitive math, and situations
that use lots of HOF where Truffle could perhaps offer more localized
optimizations that fit into the Clojure programming model. However, you pay
for all this with some rather massive startup time penalties. Most examples
I've seen show Truffle adding seconds on to the startup time of a language.

SubstrateVM - (aka native-image), SubstrateVM attempts to improve the
performance of a Truffle based language by doing image freezing. This
method has been used in other languages, and has existed in the PyPy
toolchain (RPython) for well over a decade. The idea is that you write an
interpreter, then hand SVM a pointer to the start of your interpreter (the
main() function). The framework then analyzes the data needed by that
function and all the functions it calls. All data required by those
functions are also recorded. Then the call graph required to run all these
functions is written out (normally to C or C++) and compiled. The
side-effect is that all the startup time is removed since the interpreter
is "frozen" in a started state. In the terms of Clojure this means that
metadata would be serialized as-is, instead of running the code to create
that metadata on every startup.

Polyglot VM - so in truffle you can have multiple type systems, and
multiple languages all running on Truffle. The framework then allows cheap
interop between these languages. So when GraalVM docs talk about "zero
overhead interop" what they mean is that it's possible to inline a Truffle
based function inside any other truffle based function. Since Truffle
implementations exist for Ruby, Python, LLVM bytecode (think C/C++),
JavaScript, etc. It's easy to see how its possible to have all of these
languages efficiently calling each other on the same VM.

It's not all awesome though. By using SubstrateVM you give up Java
reflection/interop, or have to find a way to embed a full JVM as a Truffle
language (this is being worked on), but every language you add into your
SVM image comes at a cost of startup times, memory usage etc. So most of
the time SVM images stick to a few languages. TruffleRuby for example only
uses Ruby and LLVM (for c interop).

To finish, I think Clojure on Truffle is possible, but it would take a TON
of work. Basically most of clojure.lang.RT would need to be rewritten from
scratch, and I'm not sure how much more in clojure.lang.* would also need
to be rewritten. So the tradeoff is:

A. Current state
- Good enough performance
- Fast enough startup (Clojure starts quickly, it's the deps/tooling that
are slow)
- Requires type hinting to avoid reflection

B. Clojure on Truffle
- More-or-less the same performance
- Faster un-type-hinted math
- (Possibly) Faster transducers/HOF over primitive collections
- No need for type hints
- Massive rewrite
- Huge startup time penalty
- Requires a modern/uncommon JVM (JVM9+ or GraalVM)

Comparing these side-by-side, it looks to me to be a wash. And because of
that I doubt we'll see a TruffleClojure any time soon.

Timothy

On Thu, Apr 19, 2018 at 4:00 AM, Khalid Jebbari 
wrote:

> Hello,
>
> Oracle has just announced GraalVM 1.0 release candidate:
> https://blogs.oracle.com/developers/announcing-graalvm
>
> It mentions a few JVM-based language but not Clojure (maybe just because
> of popularity).
> - Does it mean Clojure is not "compatible" with GraalVM ?
> - Does it mean Clojure needs to be reimplemented in terms of GraalVM's
> Truffle framework ?
> - Does it mean Clojure can be run as a native binary with fast startup
> time and minimized memory footprint ?
>
> If someone with some knowledge could explain to me the relationships
> between Clojure and GraalVM ? Bonus points if Alex Miller answers and share
> the plans, if any, about their integration/interaction.
>
> Thanks a lot in advance. really curious to understand more.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to

Re: [ANN] Schism, a set of CRDTs for Clojure and ClojureScript

[John Newman]

Alex, yeah that explains it for me. I'll probably want to use a fully
connected address space, via a mesh or a hub and spoke topology, that I
fully manage - so this is the perfect level of abstraction for me. I'd like
to see that ZQ implementation though.

So, to be clear, if node A sends schism-map M to node B, both A and B
update M, B sends M back to A, A converges M1 and M2: If M1 and M2 have
destructive conflicts, does the most "recent" change win? Or does the local
copy win?

Thanks,

V/r

John

John

On Thu, Apr 19, 2018 at 8:55 AM, Alex Redington 
wrote:

> I'll try to answer this question and John's at the same time.
>
> Schism does not try to manage state for you over time; there are a lot of
> good tools for doing that already (atoms, refs, agents, channels, etc).
> Schism is a set of collections that augment basic clojure collections with
> enough additional information that you can take two persistent collections
> with common ancestry and merge them together. So, you could take a schism
> map held in an atom, dereference it and send it from a pedestal process to
> an om application as edn, and read-string it in the om app to have a
> replicated copy of the map. The om app could make some changes (dissoc one
> key, update another) without communicating to the pedestal app for each
> operation. Then send it back to the pedestal app in a POST body, and the
> pedestal app would swap! the atom using schism.core/converge and the value
> it just received from om. The changes, both additive and destructive, would
> be replicated in the atom's value.
>
> So, yes, you could build a serverless chatroom where each client held an
> atom with a schism collection, and they sent their copies around using
> WebRTC to stay in sync. You could push a schism edn serialization over a
> message queue to be synchronized on the other end. I anticipate pushing
> each discrete state over the mq with an add-watch hook might yield bad
> performance for little semantic gain. Schism collections are necessarily
> more expensive than Clojure collections in serialization size, and
> synchronization is where all of the signficant computational expense of
> these kinds of data structures resides, so doing it less frequently than
> every discrete update will probably be best. (A debounced send & sync that
> guaranteed transmission after n milliseconds of being inert would make
> sense to me)
>
> I'm not presently confronting a problem that these solve in my day job; my
> intent was to build and have the tool ready if and when we wanted to have
> better answers for "offline sync" in a single page ClojureScript
> application.
>
> Thank you for the great questions!
>
> -Alex
>
> On Wednesday, April 18, 2018 at 9:55:36 PM UTC-4, Luke Burton wrote:
>>
>>
>> This is cool!
>>
>> There seems to be some room for a recommendation on how to apply these
>> data structures effectively in practice, especially around the distribution
>> mechanism. For instance, I can imagine storing these in an atom and using
>> watchers to push changes out over zero-mq. Or is that the Worst Idea Ever™?
>> I feel like the transit layer probably has *some* impact on whether CRDTs
>> are the right solution to the problem, and I'm interested to know what
>> those boundaries are.
>>
>> Have you used them in production over the network, and if so, what
>> solution did you use?
>>
>> Thanks again!
>>
>>
>>
>> On Apr 18, 2018, at 5:57 PM, Alex Redington  wrote:
>>
>> Good evening!
>>
>> I submit for your evaluation and reasoned feedback a library I've been
>> working on to provide a set of convergent replicated data types to Clojure
>> and ClojureScript:
>>
>> https://github.com/aredington/schism
>> [com.holychao/schism "0.1.0"]
>>
>> Schism provides convergent collections for sets, vectors, lists, and
>> maps, along with edn readers and writers out of the gate. My intent is to
>> provide you with all the tools necessary to replicate a collection across
>> two or more computing nodes.
>>
>> Replication is a hard problem, so there are a few caveats to these tools
>> as I provide them:
>>
>> - You must identify nodes within your cluster. You may choose to identify
>> nodes with a random UUID, in which case schism will help you to do so. Node
>> identifiers must be clojure serializable data.
>> - Schism purposefully avoids carrying around a monotonically increasing
>> historical collection of data about deleted entries. Consequently there are
>> some ambiguities during convergence that may not exactly mirror local
>> modification.
>> - Schism is only solving the problem of synchronizing two in memory sets.
>> Maintaining identity of those two sets, tracking state changes, and long
>> term durability are responsibilities left to the schism user.
>>
>> Schism collections are persistent collections, so you should feel free to
>> work with them as a drop in replacement for a function which would work
>> against a Clojure collection. 

RE: [ANN] clj-new -- creating new Clojure projects using the clj CLI

[Sean Corfield]

I had the one-liner in an earlier version of the README but decided it was 
unreadable (a single long line is hard to read when it scrolls so much).

Maybe I’ll put it back somewhere in there…

Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood


From: clojure@googlegroups.com  on behalf of Arnout 
Roemers 
Sent: Thursday, April 19, 2018 1:50:16 AM
To: Clojure
Subject: Re: [ANN] clj-new -- creating new Clojure projects using the clj CLI

Nice work! Another addition to the CLI tools ecosystem.

Maybe worth mentioning that a oneliner may suffice, for those just wanting to 
try it out without altering the global deps.edn:

clj -Sdeps '{:deps {seancorfield/clj-new {:git/url 
"https://github.com/seancorfield/clj-new; :sha 
"492bb2e7ad7373a8b5958124a86cddc4c7a123d5"}}}' -m clj-new.create ...



--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 
clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Schism, a set of CRDTs for Clojure and ClojureScript

[Alex Redington]

I'll try to answer this question and John's at the same time.

Schism does not try to manage state for you over time; there are a lot of 
good tools for doing that already (atoms, refs, agents, channels, etc). 
Schism is a set of collections that augment basic clojure collections with 
enough additional information that you can take two persistent collections 
with common ancestry and merge them together. So, you could take a schism 
map held in an atom, dereference it and send it from a pedestal process to 
an om application as edn, and read-string it in the om app to have a 
replicated copy of the map. The om app could make some changes (dissoc one 
key, update another) without communicating to the pedestal app for each 
operation. Then send it back to the pedestal app in a POST body, and the 
pedestal app would swap! the atom using schism.core/converge and the value 
it just received from om. The changes, both additive and destructive, would 
be replicated in the atom's value.

So, yes, you could build a serverless chatroom where each client held an 
atom with a schism collection, and they sent their copies around using 
WebRTC to stay in sync. You could push a schism edn serialization over a 
message queue to be synchronized on the other end. I anticipate pushing 
each discrete state over the mq with an add-watch hook might yield bad 
performance for little semantic gain. Schism collections are necessarily 
more expensive than Clojure collections in serialization size, and 
synchronization is where all of the signficant computational expense of 
these kinds of data structures resides, so doing it less frequently than 
every discrete update will probably be best. (A debounced send & sync that 
guaranteed transmission after n milliseconds of being inert would make 
sense to me) 

I'm not presently confronting a problem that these solve in my day job; my 
intent was to build and have the tool ready if and when we wanted to have 
better answers for "offline sync" in a single page ClojureScript 
application.

Thank you for the great questions!

-Alex

On Wednesday, April 18, 2018 at 9:55:36 PM UTC-4, Luke Burton wrote:
>
>
> This is cool! 
>
> There seems to be some room for a recommendation on how to apply these 
> data structures effectively in practice, especially around the distribution 
> mechanism. For instance, I can imagine storing these in an atom and using 
> watchers to push changes out over zero-mq. Or is that the Worst Idea Ever™? 
> I feel like the transit layer probably has *some* impact on whether CRDTs 
> are the right solution to the problem, and I'm interested to know what 
> those boundaries are.
>
> Have you used them in production over the network, and if so, what 
> solution did you use?
>
> Thanks again!
>
>
>
> On Apr 18, 2018, at 5:57 PM, Alex Redington  > wrote:
>
> Good evening!
>
> I submit for your evaluation and reasoned feedback a library I've been 
> working on to provide a set of convergent replicated data types to Clojure 
> and ClojureScript:
>
> https://github.com/aredington/schism
> [com.holychao/schism "0.1.0"]
>
> Schism provides convergent collections for sets, vectors, lists, and maps, 
> along with edn readers and writers out of the gate. My intent is to provide 
> you with all the tools necessary to replicate a collection across two or 
> more computing nodes.
>
> Replication is a hard problem, so there are a few caveats to these tools 
> as I provide them:
>
> - You must identify nodes within your cluster. You may choose to identify 
> nodes with a random UUID, in which case schism will help you to do so. Node 
> identifiers must be clojure serializable data.
> - Schism purposefully avoids carrying around a monotonically increasing 
> historical collection of data about deleted entries. Consequently there are 
> some ambiguities during convergence that may not exactly mirror local 
> modification.
> - Schism is only solving the problem of synchronizing two in memory sets. 
> Maintaining identity of those two sets, tracking state changes, and long 
> term durability are responsibilities left to the schism user.
>
> Schism collections are persistent collections, so you should feel free to 
> work with them as a drop in replacement for a function which would work 
> against a Clojure collection. The usual utilities such as conj, assoc, 
> dissoc, disj, and rest are pure functions which will return you derived 
> copies, implicitly soing the convergence bookkeeping necessary in the 
> background. As you work with it, schism will maintain node and timestamp 
> information, with the goal of convergence providing the same result as if 
> all previous invocations had occurred on one local collection in memory. 
> Schism's requirements are your expectations of a Clojure collection, so 
> hash, =, and support for meta are all included, as well as many other 
> functions defined against Clojure's own collections. Particularly with hash 
> and =, you 

Understanding GraalVM and Clojure

[Khalid Jebbari]

Hello,

Oracle has just announced GraalVM 1.0 release candidate: 
https://blogs.oracle.com/developers/announcing-graalvm

It mentions a few JVM-based language but not Clojure (maybe just because of 
popularity).
- Does it mean Clojure is not "compatible" with GraalVM ? 
- Does it mean Clojure needs to be reimplemented in terms of GraalVM's 
Truffle framework ?
- Does it mean Clojure can be run as a native binary with fast startup time 
and minimized memory footprint ?

If someone with some knowledge could explain to me the relationships 
between Clojure and GraalVM ? Bonus points if Alex Miller answers and share 
the plans, if any, about their integration/interaction.

Thanks a lot in advance. really curious to understand more.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] clj-new -- creating new Clojure projects using the clj CLI

[Arnout Roemers]

Nice work! Another addition to the CLI tools ecosystem.

Maybe worth mentioning that a oneliner may suffice, for those just wanting 
to try it out without altering the global deps.edn:

clj -Sdeps '{:deps {seancorfield/clj-new {:git/url 
"https://github.com/seancorfield/clj-new; :sha 
"492bb2e7ad7373a8b5958124a86cddc4c7a123d5"}}}' -m clj-new.create ...


-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] clj-new -- creating new Clojure projects using the clj CLI

[Alan Moore]

Nice... this looks super helpful, thanks!

Alan

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] clj-new -- creating new Clojure projects using the clj CLI

[Didier]

Ah, awesome.

I guess I'm starting to think, it would be great if clj supported something 
like:

clj --install alias-name alias-url

So maybe in a repo, you could put a file of a given convention in the root 
with a common name. Clj could pull it, it would contain a clj alias in it, 
and clj would auto add it to your deps.edn file as an alias. You could add 
a -g option to add it to your global deps.edn, if you wanted to "install" 
it globally.

Nothing else would happen.

But then, you could run your newly "installed" clojure program by doing:

clj -A:alias-name args ...

If you omit the alias name, it could default to something specified in the 
alias file.

What that alias url resolves too, I'm not sure. I'm thinking a git url to 
an alias-install.edn file which contains something like:

{:default-alias-name alias-map-to-add-as-an-alias}


And similarly, you could add:

clj --uninstall alias-name

And it would remove the alias from your deps.edn, or if given -g, from your 
global deps.edn config.

Ideally, it would remember the alias-url in the deps.edn file, so you could 
run:

clj --update alias-name

Maybe there could even be a kind of global url repo, with a list of 
registered alias names, so one could do:

clj --install alias-name

And it would know where to pull the alias-install.edn file from based on 
some global repo of distributed Clojure programs.

I'd love this. Would seem like a pretty great way to deliver Clojure 
programs, which does not require any bash script, adding anything to your 
PATH, or having to edit a file.

So for clj-new, ideally one could do:

clj --install -g clj-new

And clj would take care of adding the alias for it in my global deps.edn.

But at the very least, without the alias registry, once could simply do:

clj --install -g clj-new https://github.com/seancorfield/clj-new.git

Which would auto-install the alias from a top level alias-install.edn file 
inside the repo.

Something of the sort.


On Tuesday, 17 April 2018 20:06:16 UTC-7, Sean Corfield wrote:
>
> clj-new -- https://github.com/seancorfield/clj-new 
>
>  
>
> This will generate new projects for you, either based on `clj`, or from 
> any existing Leiningen or Boot template (or, I hope in the future, 
> `clj-template` projects!).
>
>  
>
> You'll probably want to add clj-new as an alias in your 
> ~/.clojure/deps.edn like this:
>
>  
>
> {:aliases
>
> {:new {:extra-deps {seancorfield/clj-new
>
>  {:git/url "https://github.com/seancorfield/clj-new;
>
>   :sha "492bb2e7ad7373a8b5958124a86cddc4c7a123d5"}}
>
> :main-opts ["-m" "clj-new.create"]}}
>
> ...}
>
>  
>
> Create a basic application:
>
>  
>
> clj -A:new app myname/myapp
>
> cd myapp
>
> clj -m myname.myapp
>
>  
>
> Run the tests:
>
>  
>
> clj -A:test:runner
>
>  
>
> Yes, the `deps.edn` generated from the `app` (and `lib`) built-in template 
> includes aliases to include your `test` folder, add `test.check` as a 
> dependency, and bring in and run Cognitect's `test-runner`
>
>  
>
> The project name should either be a qualified Clojure symbol or a 
> multi-segment name -- single segment project names are not allowed!
>
>  
>
> For a qualified Clojure symbol, the first part is typically your GitHub 
> account name or your organization's domain reversed, e.g., `com.acme`, and 
> the second part is the "local" name for your project (and is used as the 
> name of the folder in which the project is created).
>
>  
>
> For a multi-segment project name, such as `foo.bar`, the folder that will 
> be created would be called `foo.bar` and will contain `src/foo/bar.clj`.
>
>  
>
> Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
> An Architect's View -- http://corfield.org/
>
> "If you're not annoying somebody, you're not really alive."
> -- Margaret Atwood
>
>  
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.