Re: how do I debug a cryptic XML error?

[Laws]

Okay, this seemed to fix the problem:


cpe-dictionary (-> "official-cpe-dictionary_v2.3.xml"
   (java.io.FileInputStream.)
   (xml/parse))


xmlzipper (clojure.zip/xml-zip cpe-dictionary)

xmlnode (-> xmlzipper
zip/down
zip/right
zip/node)




On Tuesday, February 15, 2022 at 10:44:13 PM UTC-5 Laws wrote:

>
> I changed the code a bit:
>
> cpe-dictionary (-> "official-cpe-dictionary_v2.3.xml"
>(java.io.StringReader.)
>(xml/parse))
>
> xmlzipper (clojure.zip/xml-zip cpe-dictionary)
>
> Now I get this:
>
>   clojure.data.xml/*parse*   
> xml.clj:  84
>
>   clojure.data.xml/*parse*   
> xml.clj: 109
>
> clojure.data.xml.tree/*event-tree*
>   tree.clj:  70
>
>  clojure.core/*ffirst*
>   core.clj: 105
>
>   clojure.core/*first*
>   core.clj:  55
>
>  ...  
>  
>
>clojure.data.xml.tree/seq-tree/*fn*
>   tree.clj:  39
>
> clojure.core/*seq*
>   core.clj: 139
>
>  ...  
>  
>
>   clojure.data.xml.jvm.parse/pull-seq/*fn*   
>   parse.clj:  78
>
> com.sun.org.apache.xerces.internal.impl.XMLStreamReaderImpl.next  
> XMLStreamReaderImpl.java: 652
>
> *javax.xml.stream.XMLStreamException*: *ParseError at [row,col]:[1,1]*
>
> * Message: Content is not allowed in 
> prolog.*
>
> *location*: 
> #object[com.sun.org.apache.xerces.internal.impl.XMLStreamReaderImpl$1 
> 0x296bfddb "Line number = 1\nColumn number = 1\nSystem Id = null\nPublic Id 
> = null\nLocation Uri= null\nCharacterOffset = 0\n"]
>
>
>
>
> On Tuesday, February 15, 2022 at 7:50:12 PM UTC-5 Laws wrote:
>
>> So, I went to the government NVD website:
>>
>> https://nvd.nist.gov/products/cpe
>>
>> I downloaded the CPE Dictionary and unpacked it. It looks like standard 
>> XML. 
>>
>> I copy and paste the standard XML example given on the Clojure XML 
>> documentation page:
>>
>>  cpe-dictionary (-> "official-cpe-dictionary_v2.3.xml" io/resource 
>> io/file clj-xml/parse zip/xml-zip)
>>
>> I get:
>>
>> clojure.xml/*startparse-sax*xml.clj:  76
>> ...   
>>
>>jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke  
>> DelegatingMethodAccessorImpl.java:  43
>>
>>jdk.internal.reflect.NativeMethodAccessorImpl.invoke  
>> NativeMethodAccessorImpl.java:  77
>>
>>   jdk.internal.reflect.NativeMethodAccessorImpl.invoke0   
>> NativeMethodAccessorImpl.java
>>
>> com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse   
>>   SAXParserImpl.java: 317
>>
>> *java.lang.IllegalArgumentException*: 
>>
>>
>> I'm wondering, how do I figure out what is wrong here? I'm going to 
>> assume the government is offering reasonably standard XML, so where would 
>> the problem arise? How do I figure out a way around this? 
>>
>> The CPE dictionary is 386 megabytes so I can't share the hold file here, 
>> but when I run "head" on it, the beginning looks like this:
>>
>>
>> 
>>
>> http://scap.nist.gov/schema/configuration/0.1; 
>> xmlns="http://cpe.mitre.org/dictionary/2.0; xmlns:xsi="
>> http://www.w3.org/2001/XMLSchema-instance; xmlns:scap-core="
>> http://scap.nist.gov/schema/scap-core/0.3; xmlns:cpe-23="
>> http://scap.nist.gov/schema/cpe-extension/2.3; xmlns:ns6="
>> http://scap.nist.gov/schema/scap-core/0.1; xmlns:meta="
>> http://scap.nist.gov/schema/cpe-dictionary-metadata/0.2; 
>> xsi:schemaLocation="http://scap.nist.gov/schema/cpe-extension/2.3 
>> https://scap.nist.gov/schema/cpe/2.3/cpe-dictionary-extension_2.3.xsd 
>> http://cpe.mitre.org/dictionary/2.0 
>> https://scap.nist.gov/schema/cpe/2.3/cpe-dictionary_2.3.

Re: how do I debug a cryptic XML error?

[Laws]

I changed the code a bit:

cpe-dictionary (-> "official-cpe-dictionary_v2.3.xml"
   (java.io.StringReader.)
   (xml/parse))

xmlzipper (clojure.zip/xml-zip cpe-dictionary)

Now I get this:

  clojure.data.xml/*parse* 
  xml.clj:  84

  clojure.data.xml/*parse* 
  xml.clj: 109

clojure.data.xml.tree/*event-tree*  
tree.clj:  70

 clojure.core/*ffirst*  
core.clj: 105

  clojure.core/*first*  
core.clj:  55

 ...
   

   clojure.data.xml.tree/seq-tree/*fn*  
tree.clj:  39

clojure.core/*seq*  
core.clj: 139

 ...
   

  clojure.data.xml.jvm.parse/pull-seq/*fn* 
parse.clj:  78

com.sun.org.apache.xerces.internal.impl.XMLStreamReaderImpl.next  
XMLStreamReaderImpl.java: 652

*javax.xml.stream.XMLStreamException*: *ParseError at [row,col]:[1,1]*

* Message: Content is not allowed in 
prolog.*

*location*: 
#object[com.sun.org.apache.xerces.internal.impl.XMLStreamReaderImpl$1 
0x296bfddb "Line number = 1\nColumn number = 1\nSystem Id = null\nPublic Id 
= null\nLocation Uri= null\nCharacterOffset = 0\n"]




On Tuesday, February 15, 2022 at 7:50:12 PM UTC-5 Laws wrote:

> So, I went to the government NVD website:
>
> https://nvd.nist.gov/products/cpe
>
> I downloaded the CPE Dictionary and unpacked it. It looks like standard 
> XML. 
>
> I copy and paste the standard XML example given on the Clojure XML 
> documentation page:
>
>  cpe-dictionary (-> "official-cpe-dictionary_v2.3.xml" io/resource io/file 
> clj-xml/parse zip/xml-zip)
>
> I get:
>
> clojure.xml/*startparse-sax*xml.clj:  76  
>   ...   
>
>jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke  
> DelegatingMethodAccessorImpl.java:  43
>
>jdk.internal.reflect.NativeMethodAccessorImpl.invoke  
> NativeMethodAccessorImpl.java:  77
>
>   jdk.internal.reflect.NativeMethodAccessorImpl.invoke0   
> NativeMethodAccessorImpl.java
>
> com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse   
>   SAXParserImpl.java: 317
>
> *java.lang.IllegalArgumentException*: 
>
>
> I'm wondering, how do I figure out what is wrong here? I'm going to assume 
> the government is offering reasonably standard XML, so where would the 
> problem arise? How do I figure out a way around this? 
>
> The CPE dictionary is 386 megabytes so I can't share the hold file here, 
> but when I run "head" on it, the beginning looks like this:
>
>
> 
>
> http://scap.nist.gov/schema/configuration/0.1; 
> xmlns="http://cpe.mitre.org/dictionary/2.0; xmlns:xsi="
> http://www.w3.org/2001/XMLSchema-instance; xmlns:scap-core="
> http://scap.nist.gov/schema/scap-core/0.3; xmlns:cpe-23="
> http://scap.nist.gov/schema/cpe-extension/2.3; xmlns:ns6="
> http://scap.nist.gov/schema/scap-core/0.1; xmlns:meta="
> http://scap.nist.gov/schema/cpe-dictionary-metadata/0.2; 
> xsi:schemaLocation="http://scap.nist.gov/schema/cpe-extension/2.3 
> https://scap.nist.gov/schema/cpe/2.3/cpe-dictionary-extension_2.3.xsd 
> http://cpe.mitre.org/dictionary/2.0 
> https://scap.nist.gov/schema/cpe/2.3/cpe-dictionary_2.3.xsd 
> http://scap.nist.gov/schema/cpe-dictionary-metadata/0.2 
> https://scap.nist.gov/schema/cpe/2.1/cpe-dictionary-metadata_0.2.xsd 
> http://scap.nist.gov/schema/scap-core/0.3 
> https://scap.nist.gov/schema/nvd/scap-core_0.3.xsd 
> http://scap.nist.gov/schema/configuration/0.1 
> https://scap.nist.gov/schema/nvd/configuration_0.1.xsd 
> http://scap.nist.gov/schema/scap-core/0.1 
> https://scap.nist.gov/schema/nvd/scap-core_0.1.xsd;>
>
>   
>
> National Vulnerability Database (NVD)
>
> 4.9
>
> 2.3
>
> 2022-01-25T04:50:56.780Z
>
>   
>
>name="cpe:/a:%240.99_kindle_books_project:%240.99_kindle_books:6::~~~android~~">
>
> $0.99 Kindle Books project $0.99 Kindle Books 
> (aka com.kindle.books.for99) for android 6.0
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure&q

how do I debug a cryptic XML error?

[Laws]

So, I went to the government NVD website:

https://nvd.nist.gov/products/cpe

I downloaded the CPE Dictionary and unpacked it. It looks like standard 
XML. 

I copy and paste the standard XML example given on the Clojure XML 
documentation page:

 cpe-dictionary (-> "official-cpe-dictionary_v2.3.xml" io/resource io/file 
clj-xml/parse zip/xml-zip)

I get:

clojure.xml/*startparse-sax*xml.clj:  76
...   

   jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke  
DelegatingMethodAccessorImpl.java:  43

   jdk.internal.reflect.NativeMethodAccessorImpl.invoke  
NativeMethodAccessorImpl.java:  77

  jdk.internal.reflect.NativeMethodAccessorImpl.invoke0   
NativeMethodAccessorImpl.java

com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse 
SAXParserImpl.java: 317

*java.lang.IllegalArgumentException*: 


I'm wondering, how do I figure out what is wrong here? I'm going to assume 
the government is offering reasonably standard XML, so where would the 
problem arise? How do I figure out a way around this? 

The CPE dictionary is 386 megabytes so I can't share the hold file here, 
but when I run "head" on it, the beginning looks like this:




http://scap.nist.gov/schema/configuration/0.1; 
xmlns="http://cpe.mitre.org/dictionary/2.0; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; 
xmlns:scap-core="http://scap.nist.gov/schema/scap-core/0.3; 
xmlns:cpe-23="http://scap.nist.gov/schema/cpe-extension/2.3; 
xmlns:ns6="http://scap.nist.gov/schema/scap-core/0.1; 
xmlns:meta="http://scap.nist.gov/schema/cpe-dictionary-metadata/0.2; 
xsi:schemaLocation="http://scap.nist.gov/schema/cpe-extension/2.3 
https://scap.nist.gov/schema/cpe/2.3/cpe-dictionary-extension_2.3.xsd 
http://cpe.mitre.org/dictionary/2.0 
https://scap.nist.gov/schema/cpe/2.3/cpe-dictionary_2.3.xsd 
http://scap.nist.gov/schema/cpe-dictionary-metadata/0.2 
https://scap.nist.gov/schema/cpe/2.1/cpe-dictionary-metadata_0.2.xsd 
http://scap.nist.gov/schema/scap-core/0.3 
https://scap.nist.gov/schema/nvd/scap-core_0.3.xsd 
http://scap.nist.gov/schema/configuration/0.1 
https://scap.nist.gov/schema/nvd/configuration_0.1.xsd 
http://scap.nist.gov/schema/scap-core/0.1 
https://scap.nist.gov/schema/nvd/scap-core_0.1.xsd;>

  

National Vulnerability Database (NVD)

4.9

2.3

2022-01-25T04:50:56.780Z

  

  

$0.99 Kindle Books project $0.99 Kindle Books 
(aka com.kindle.books.for99) for android 6.0

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/526fa490-4c26-46dc-acd3-34f377d1eb91n%40googlegroups.com.

Re: What Emacs framework do you prefer?

[Laws]

Thank you, everyone, I'll investigate these suggestions. 

On Monday, August 9, 2021 at 10:50:04 PM UTC-4 Laws wrote:

> I've been away from Clojure for 3 years but I've decided my next project 
> will be pure Clojure. I'm setting everything up on a new MacBook Pro that I 
> just bought. I just installed Emacs and I'm wondering what kind of 
> starter-kit or framework do people prefer nowadays? Does anyone want to 
> either recommend a framework or share an init file that has some 
> customizations for Clojure? 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/3b75dd32-2093-40df-afb9-3a3ca75156b1n%40googlegroups.com.

Re: What Emacs framework do you prefer?

[Zack Teo]

If you are looking for a framework for Emacs as a whole - I personally use Doom 
Emacs <https://github.com/hlissner/doom-emacs> - which for the most part, I 
just enable clojure module on. 

Many others seem to use Spacemacs - of which practicalli seems to have some 
guides on here <https://practical.li/spacemacs/>. 

Lastly, there is Prelude 
<https://github.com/bbatsov/prelude/blob/f9fb902185e1f7afabe281a25f5787e69ea7b6c9/doc/modules/clojure.md>
 
which the author of CIDER manages. 

Of course, there's other community managed configurations of Emacs but 
these are the few I frequently encounter - And am aware have specific 
configurations for Clojure.





On Tuesday, August 10, 2021 at 5:18:14 PM UTC+8 Jiacai Liu wrote:

> https://github.com/flyingmachine/emacs-for-clojure
>
> I recommend flyingmachine's config, tailored for clojure 
> development.
> Also his awesome book https://www.braveclojure.com/basic-emacs/
>
> On Tue, Aug 10, 2021 at 10:50:04 AM +0800, Laws wrote:
>
> > I've been away from Clojure for 3 years but I've decided my next 
> > project
> > will be pure Clojure. I'm setting everything up on a new MacBook 
> > Pro that I
> > just bought. I just installed Emacs and I'm wondering what kind 
> > of
> > starter-kit or framework do people prefer nowadays? Does anyone 
> > want to
> > either recommend a framework or share an init file that has some
> > customizations for Clojure?
>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/0d26e1e6-96a7-4a55-9571-0382c540349dn%40googlegroups.com.

Re: What Emacs framework do you prefer?

[Jiacai Liu]

https://github.com/flyingmachine/emacs-for-clojure

I recommend flyingmachine's config, tailored for clojure 
development.

Also his awesome book https://www.braveclojure.com/basic-emacs/

On Tue, Aug 10, 2021 at 10:50:04 AM +0800, Laws wrote:

I've been away from Clojure for 3 years but I've decided my next 
project
will be pure Clojure. I'm setting everything up on a new MacBook 
Pro that I
just bought. I just installed Emacs and I'm wondering what kind 
of
starter-kit or framework do people prefer nowadays? Does anyone 
want to

either recommend a framework or share an init file that has some
customizations for Clojure?



--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/6112444b.1c69fb81.43c55.6a4cSMTPIN_ADDED_BROKEN%40gmr-mx.google.com.

Re: What Emacs framework do you prefer?

[Oleksandr Shulgin]

On Tue, Aug 10, 2021 at 4:50 AM Laws  wrote:

> Does anyone want to either recommend a framework or share an init file
> that has some customizations for Clojure?


https://cider.mx/


Cheers,
--
Alex

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/CACACo5RQ1zbeXnHWS03KBwqG6XcQ2Z0D-6ivELg32RVD-0VayQ%40mail.gmail.com.

What Emacs framework do you prefer?

[Laws]

I've been away from Clojure for 3 years but I've decided my next project 
will be pure Clojure. I'm setting everything up on a new MacBook Pro that I 
just bought. I just installed Emacs and I'm wondering what kind of 
starter-kit or framework do people prefer nowadays? Does anyone want to 
either recommend a framework or share an init file that has some 
customizations for Clojure? 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/81473f1a-f867-41ef-8dd3-2afd0946c373n%40googlegroups.com.

RE: How can I do something like (= (class ds) next.jdbc.connection)??

[Sean Corfield]

With the clue you gave me, I can look at that page and I see that 
javax.sql.DataSource is used in two protocols, so I guess I could then look to 
see where those protocols are used. But I'm thinking there might be an easier 
way to reverse-engineer an interface or class to get something that I can use 
in an if statement, when I'm adding in a runtime check?

Maybe take a step back and explain the problem you are trying to solve here?

It sounds like you’re trying to jump into some really low-level stuff that you 
almost certainly do not need to do. You asked “What is the correct way to do 
this?” but it’s not at all clear what you’re trying to do so I don’t know how 
to answer it, other than suggesting you read the docs and join the Slack 
community so you can have real time conversations where folks can better 
understand how to help you by asking you questions and looking at your code. 
The mailing list is just too slow a medium for the sort of conversations I 
think you need to have at this stage in your Clojure journey.

Sean Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood

From: Laws
Sent: Tuesday, October 22, 2019 3:17 PM
To: Clojure
Subject: Re: How can I do something like (= (class ds) next.jdbc.connection)??

> The ds binding that you have will satisfy this check (instance? 
> javax.sql.DataSource ds) -- 
> because it is an instance of that Java interface.

My experience so far has been a lot of this stuff seems like commonsense to 
those programmers who have a background with Java, but those of us who learn 
Clojure without knowing Java have a harder time picking this up. Part of my 
question was "How do I know this?" Or in your case, how did you know this? The 
reason I included the link to the Github page is I was hoping to find out what 
on that page I was supposed to look at to figure out the answer. 

With the clue you gave me, I can look at that page and I see that 
javax.sql.DataSource is used in two protocols, so I guess I could then look to 
see where those protocols are used. But I'm thinking there might be an easier 
way to reverse-engineer an interface or class to get something that I can use 
in an if statement, when I'm adding in a runtime check? I'm a little surprised 
that (class) and (type) and (ancestors) did not give me something that I could 
use. 


On Friday, October 18, 2019 at 12:23:02 AM UTC-4, Sean Corfield wrote:
You should :require the namespaces, not try to :import things.

(ns your.namespace
  (:require [next.jdbc :as jdbc]))

I suggest you start off by working through 
https://cljdoc.org/d/seancorfield/next.jdbc/1.0.9/doc/getting-started 

The ds binding that you have will satisfy this check (instance? 
javax.sql.DataSource ds) -- because it is an instance of that Java interface.

It sounds like you have quite a few misconceptions about using the library (and 
perhaps using Clojure in general?) so I highly recommend joining the Clojurians 
Slack http://clojurians.net and https://clojurians.slack.com where you can ask 
real time questions in the #beginners channel and next.jdbc-specific questions 
in the #sql channel.



On Thu, Oct 17, 2019 at 6:03 PM Laws  wrote:
If I do this:

(class ds)

I see: 

next.jdbc.connection$url_PLUS_etc$reify__555

Is there anyway I can match against this? I'd like a runtime check to know that 
the code really does have a database connection. Imagine code like this:

(if (= next.jdbc.connection (class ds))
      (println "its next.jdbc.connection")
      (println "fail, it is not next.jdbc.connection"))

I get errors such as:

java.lang.ClassNotFoundException: seancorfield/next.jdbc.Connection, 
compiling:(core.clj:1:1)

The namespace is here:

https://github.com/seancorfield/next-jdbc/blob/master/src/next/jdbc/connection.clj


   (:import
   [seancorfield/next.jdbc Connection]
   )

   (:import
   [next.jdbc Connection]
   )

   (:import
   [next.jdbc.connection]
   )


What is the correct way to do this? 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clo...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clo...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/8a486bd5-bb5f-41ef-bcb6-74e49ed58db4%40googlegroups.com.



-- 
Sean A Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
World Singles Networks, LLC. -- ht

Re: How can I do something like (= (class ds) next.jdbc.connection) ??

[Laws]

> The ds binding that you have will satisfy this check (instance? 
javax.sql.DataSource ds) -- 
> because it is an instance of that Java interface.

My experience so far has been a lot of this stuff seems like commonsense to 
those programmers who have a background with Java, but those of us who 
learn Clojure without knowing Java have a harder time picking this up. Part 
of my question was "How do I know this?" Or in your case, how did you know 
this? The reason I included the link to the Github page is I was hoping to 
find out what on that page I was supposed to look at to figure out the 
answer. 

With the clue you gave me, I can look at that page and I see that 
javax.sql.DataSource is used in two protocols, so I guess I could then look 
to see where those protocols are used. But I'm thinking there might be an 
easier way to reverse-engineer an interface or class to get something that 
I can use in an if statement, when I'm adding in a runtime check? I'm a 
little surprised that (class) and (type) and (ancestors) did not give me 
something that I could use. 


On Friday, October 18, 2019 at 12:23:02 AM UTC-4, Sean Corfield wrote:
>
> You should :require the namespaces, not try to :import things.
>
> (ns your.namespace
>   (:require [next.jdbc :as jdbc]))
>
> I suggest you start off by working through 
> https://cljdoc.org/d/seancorfield/next.jdbc/1.0.9/doc/getting-started 
>
> The ds binding that you have will satisfy this check (instance? 
> javax.sql.DataSource ds) -- because it is an instance of that Java 
> interface.
>
> It sounds like you have quite a few misconceptions about using the library 
> (and perhaps using Clojure in general?) so I highly recommend joining the 
> Clojurians Slack http://clojurians.net and https://clojurians.slack.com 
> where you can ask real time questions in the #beginners channel and 
> next.jdbc-specific questions in the #sql channel.
>
>
>
> On Thu, Oct 17, 2019 at 6:03 PM Laws > 
> wrote:
>
>> If I do this:
>>
>> (class ds)
>>
>> I see: 
>>
>> next.jdbc.connection$url_PLUS_etc$reify__555
>>
>> Is there anyway I can match against this? I'd like a runtime check to 
>> know that the code really does have a database connection. Imagine code 
>> like this:
>>
>> (if (= next.jdbc.connection (class ds))
>>   (println "its next.jdbc.connection")
>>   (println "fail, it is not next.jdbc.connection"))
>>
>> I get errors such as:
>>
>> java.lang.ClassNotFoundException: seancorfield/next.jdbc.Connection, 
>> compiling:(core.clj:1:1)
>>
>> The namespace is here:
>>
>>
>> https://github.com/seancorfield/next-jdbc/blob/master/src/next/jdbc/connection.clj
>>
>>
>>(:import
>>[seancorfield/next.jdbc Connection]
>>)
>>
>>(:import
>>[next.jdbc Connection]
>>)
>>
>>(:import
>>[next.jdbc.connection]
>>)
>>
>>
>> What is the correct way to do this? 
>>
>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com 
>> 
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clo...@googlegroups.com 
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to clo...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/clojure/8a486bd5-bb5f-41ef-bcb6-74e49ed58db4%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/clojure/8a486bd5-bb5f-41ef-bcb6-74e49ed58db4%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>
>
> -- 
> Sean A Corfield -- (904) 302-SEAN
> An Architect's View -- http://corfield.org/
> World Singles Networks, LLC. -- https://worldsinglesnetworks.com/
>
> "Perfection is the enemy of the good."
> -- Gustave Flaubert, French realist novelist (1821-1880)
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/bb43da65-d4be-4afa-bd86-5707817a5764%40googlegroups.com.

Re: How can I do something like (= (class ds) next.jdbc.connection) ??

[Sean Corfield]

You should :require the namespaces, not try to :import things.

(ns your.namespace
  (:require [next.jdbc :as jdbc]))

I suggest you start off by working through
https://cljdoc.org/d/seancorfield/next.jdbc/1.0.9/doc/getting-started

The ds binding that you have will satisfy this check (instance?
javax.sql.DataSource ds) -- because it is an instance of that Java
interface.

It sounds like you have quite a few misconceptions about using the library
(and perhaps using Clojure in general?) so I highly recommend joining the
Clojurians Slack http://clojurians.net and https://clojurians.slack.com
where you can ask real time questions in the #beginners channel and
next.jdbc-specific questions in the #sql channel.



On Thu, Oct 17, 2019 at 6:03 PM Laws  wrote:

> If I do this:
>
> (class ds)
>
> I see:
>
> next.jdbc.connection$url_PLUS_etc$reify__555
>
> Is there anyway I can match against this? I'd like a runtime check to know
> that the code really does have a database connection. Imagine code like
> this:
>
> (if (= next.jdbc.connection (class ds))
>   (println "its next.jdbc.connection")
>   (println "fail, it is not next.jdbc.connection"))
>
> I get errors such as:
>
> java.lang.ClassNotFoundException: seancorfield/next.jdbc.Connection,
> compiling:(core.clj:1:1)
>
> The namespace is here:
>
>
> https://github.com/seancorfield/next-jdbc/blob/master/src/next/jdbc/connection.clj
>
>
>(:import
>[seancorfield/next.jdbc Connection]
>)
>
>(:import
>    [next.jdbc Connection]
>)
>
>(:import
>[next.jdbc.connection]
>)
>
>
> What is the correct way to do this?
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/clojure/8a486bd5-bb5f-41ef-bcb6-74e49ed58db4%40googlegroups.com
> <https://groups.google.com/d/msgid/clojure/8a486bd5-bb5f-41ef-bcb6-74e49ed58db4%40googlegroups.com?utm_medium=email_source=footer>
> .
>


-- 
Sean A Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
World Singles Networks, LLC. -- https://worldsinglesnetworks.com/

"Perfection is the enemy of the good."
-- Gustave Flaubert, French realist novelist (1821-1880)

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/CAD4thx_%3DcAMUWv0K%3D_Z1JhEP%2B%2Bh1OR4Bv0g6sgnguz_LCvnGBw%40mail.gmail.com.

How can I do something like (= (class ds) next.jdbc.connection) ??

[Laws]

If I do this:

(class ds)

I see: 

next.jdbc.connection$url_PLUS_etc$reify__555

Is there anyway I can match against this? I'd like a runtime check to know 
that the code really does have a database connection. Imagine code like 
this:

(if (= next.jdbc.connection (class ds))
  (println "its next.jdbc.connection")
  (println "fail, it is not next.jdbc.connection"))

I get errors such as:

java.lang.ClassNotFoundException: seancorfield/next.jdbc.Connection, 
compiling:(core.clj:1:1)

The namespace is here:

https://github.com/seancorfield/next-jdbc/blob/master/src/next/jdbc/connection.clj


   (:import
   [seancorfield/next.jdbc Connection]
   )

   (:import
   [next.jdbc Connection]
   )

   (:import
   [next.jdbc.connection]
   )


What is the correct way to do this? 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/8a486bd5-bb5f-41ef-bcb6-74e49ed58db4%40googlegroups.com.

Re: Code style: how do you order map keys?

[Matching Socks]

Clarity is important.  Considering the conundrum of sorting namespace 
aliases, vs using the fully-qualified name as the implicit sort key despite 
the apparent alias; and the burden of shuffling everything after a 
refactoring; any sorted order will sooner or later be "broken", so you 
might as well aim for clarity from the start.  If other participants 
resist, you can proceed in two stages.  First, rename all variables using a 
section of Unicode where nobody knows the collation sequence.  

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/efae8723-7ece-44be-8258-37aecbb121d3%40googlegroups.com.

Code style: how do you order map keys?

[Yuri Govorushchenko]

I usually try to organize related things in different files in the same 
order to ease code reviewing, reading and writing new code. For example: 
order of protocol method declarations, their definitions in records and 
unit tests. And I was wondering what's the best way to order keys in maps 
and specs.

The simple rule could be to always sort keys lexically, so that the order 
stays the same across files and there's no additional thinking needed when 
adding new keys.
But with `s/keys` it's not that simple since `:req`/`:req-un` can break the 
consistency across the files. E.g. during spec definition the correct 
lexical order is `:req-un [:a/x :a/y ::d]`, but during "usage" it will be 
different: `{:d ... :x ... :y ...}`.
The lesser drawback of this approach is that some human-friendliness is 
lost, e.g. keys `host, port, db, user, password` must be ordered as `db, 
host, password, port, user`.

Do you try to maintain the same order of map keys (including keys in specs) 
across the files? If yes, what rules do you follow?

Thank you.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/d29d4e36-4dee-4bd6-9347-7fa397103f5e%40googlegroups.com.

Nice blog entry on not trying to do much with `:prep-tasks` in `lein`

[Alan Thompson]

Makes a good point that other tools should be used for non-clojure build
steps.

   https://grishaev.me/en/lein/

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/CAN67zA33YrnyrPSh2DoZ8UNLTv%3DtAjaVDPJvFF59Edncpd-avg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: What do Clojure developers use for recurring functions, other than at-at

[Rowan Hargreaves]

+1 for at-at. 

I've used it recently and it works well. It 
uses ScheduledThreadPoolExecutor etc under the hood, and it's only 350 
lines of readable code if you want to dig in. 

I have found its interface easy to use. As well as scheduling at specific 
times (the at function) or at regular intervals (every), it has the 
function interspaced which will schedule tasks after a specified delay 
after the previous call to that task has finished.  It also has a way of 
viewing and reseting tasks in the task pool.   

Rowan

On Monday, December 17, 2018 at 8:31:07 PM UTC, Tim Visher wrote:
>
> On Mon, Dec 17, 2018 at 2:54 PM > 
> wrote:
>
>> But at-at has not been updated in 6 years, so I assume it is abandoned. I 
>> have two questions about this:
>>
>
> A common bit of wisdom here in the Clojure community is that time since 
> last update is not always (or even often) a sign of abandonment but instead 
> of stability. If there are many and recent open issues on at-at then maybe 
> it's been abandoned. Or it could just be stable.
>
> Disclaimer, I don't know much about at-at.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: What do Clojure developers use for recurring functions, other than at-at

[Laurens Van Houtven]

Hi,

On Mon, Dec 17, 2018 at 3:46 PM  wrote:

> Laurens Van Houtven, good ideas, but then I'd also have to write some code
> to catch documents that got lost when a process died while trying to fetch
> a document from S3. If I simply check every 15 minutes, and grab everything
> that has not already been stored in the database, then I automatically
> fetch documents that were dropped due to error. It seems simple. In my
> current case, there is no penalty for fetching the same document twice, but
> it is important that we avoid missing a document.
>

I'm not sure why that makes my suggestions not work: you  create a
CloudWatch Event that just fires every 15 minutes :-) Heck, you can write
one that fires on S3 write, and then another that fires on cloudwatch
errors for when that process dies :-) But I'd probably just use a
CloudWatch event that fires every 15 minute sand run your code as an ECS
Fargate job.

lvh


>
>
> On Monday, December 17, 2018 at 2:59:22 PM UTC-5, Laurens Van Houtven
> wrote:
>>
>> Honestly I'd use CloudWatch Timed Events to kick off a Lambda or ECS
>> Fargate job (which of course you can write in Clojure) assuming you're
>> using AWS yourself anyway. If you don't care about batching maybe even just
>> attach a Lambda to the write-to-S3 bucket itself instead of checking every
>> 15m?
>>
>> If you want to do it in-process, my tool of choice is ztellman's
>> manifold:
>> https://github.com/ztellman/manifold/blob/d67a8c1b9f1268c094895d70dbbf146521f5774b/src/manifold/time.clj
>>
>> On Mon, Dec 17, 2018 at 1:54 PM  wrote:
>>
>>> I'm coming back to Clojure development after a year away. This is a fast
>>> moving community and it is hard to keep up when one is not working on it
>>> full time. I'm dusting off some code I wrote 2 years ago, and trying to
>>> bring all the dependencies up to their current versions.
>>>
>>> I have a function that fetches files from an AWS S3 bucket, every 15
>>> minutes. I had previously used the at-at library for this:
>>>
>>> https://github.com/overtone/at-at
>>>
>>> But at-at has not been updated in 6 years, so I assume it is abandoned.
>>> I have two questions about this:
>>>
>>> 1.) how else do Clojure programmers usually call recurring
>>> functionality?
>>>
>>> 2.) I am ignorant about the JVM, so I'm afraid I have to ask, at a
>>> fundamental level, how does at-at work? I know that if a function calls
>>> itself recurringly, on the JVM, one eventually gets stackoverflow. So how
>>> does at-at make its magic work?
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to clo...@googlegroups.com
>>> Note that posts from new members are moderated - please be patient with
>>> your first post.
>>> To unsubscribe from this group, send email to
>>> clojure+u...@googlegroups.com
>>> For more options, visit this group at
>>> http://groups.google.com/group/clojure?hl=en
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to clojure+u...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: What do Clojure developers use for recurring functions, other than at-at

[Adam Clements]

I think iirc that at-at uses scheduledexecutor and is a very simple and
stable library which only does one thing, but does it well and has no
feature requests. I wrote schejulure which does a very similar job but with
cron style time specifications rather than periodic and haven't touched it
for years because it just works and even though it's used in production
nobody has found any issues or needed it to do anything it doesn't already
do. I think the same is probably true of at-at.

Adam

On Mon, 17 Dec 2018, 9:46 pm  James Reeves, that does sound like the right way to go. I'll do that.
>
>
> On Monday, December 17, 2018 at 3:31:01 PM UTC-5, James Reeves wrote:
>>
>>
>> I'd use an executor:
>>
>>   (ns example.main
>> (:import [java.util.concurrent Executors TimeUnit]))
>>
>>   (def scheduler
>> (Executors/newScheduledThreadPool 32))
>>
>>   (defn fetch-files []
>> (println "Fetching files...))
>>
>>   (defn -main []
>> (.scheduleAtFixedRate scheduler ^Runnable fetch-files 15 15
>> TimeUnit/MINUTES))
>>
>> On Mon, 17 Dec 2018 at 20:14,  wrote:
>>
>>> I'm coming back to Clojure development after a year away. This is a fast
>>> moving community and it is hard to keep up when one is not working on it
>>> full time. I'm dusting off some code I wrote 2 years ago, and trying to
>>> bring all the dependencies up to their current versions.
>>>
>>> I have a function that fetches files from an AWS S3 bucket, every 15
>>> minutes. I had previously used the at-at library for this:
>>>
>>> https://github.com/overtone/at-at
>>>
>>> But at-at has not been updated in 6 years, so I assume it is abandoned.
>>> I have two questions about this:
>>>
>>> 1.) how else do Clojure programmers usually call recurring
>>> functionality?
>>>
>>> 2.) I am ignorant about the JVM, so I'm afraid I have to ask, at a
>>> fundamental level, how does at-at work? I know that if a function calls
>>> itself recurringly, on the JVM, one eventually gets stackoverflow. So how
>>> does at-at make its magic work?
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to clo...@googlegroups.com
>>> Note that posts from new members are moderated - please be patient with
>>> your first post.
>>> To unsubscribe from this group, send email to
>>> clojure+u...@googlegroups.com
>>> For more options, visit this group at
>>> http://groups.google.com/group/clojure?hl=en
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to clojure+u...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>> --
>> James Reeves
>> booleanknot.com
>>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: What do Clojure developers use for recurring functions, other than at-at

[lawrence . krubner]

Laurens Van Houtven, good ideas, but then I'd also have to write some code 
to catch documents that got lost when a process died while trying to fetch 
a document from S3. If I simply check every 15 minutes, and grab everything 
that has not already been stored in the database, then I automatically 
fetch documents that were dropped due to error. It seems simple. In my 
current case, there is no penalty for fetching the same document twice, but 
it is important that we avoid missing a document. 



On Monday, December 17, 2018 at 2:59:22 PM UTC-5, Laurens Van Houtven wrote:
>
> Honestly I'd use CloudWatch Timed Events to kick off a Lambda or ECS 
> Fargate job (which of course you can write in Clojure) assuming you're 
> using AWS yourself anyway. If you don't care about batching maybe even just 
> attach a Lambda to the write-to-S3 bucket itself instead of checking every 
> 15m?
>
> If you want to do it in-process, my tool of choice is ztellman's manifold: 
> https://github.com/ztellman/manifold/blob/d67a8c1b9f1268c094895d70dbbf146521f5774b/src/manifold/time.clj
>
> On Mon, Dec 17, 2018 at 1:54 PM > 
> wrote:
>
>> I'm coming back to Clojure development after a year away. This is a fast 
>> moving community and it is hard to keep up when one is not working on it 
>> full time. I'm dusting off some code I wrote 2 years ago, and trying to 
>> bring all the dependencies up to their current versions. 
>>
>> I have a function that fetches files from an AWS S3 bucket, every 15 
>> minutes. I had previously used the at-at library for this:
>>
>> https://github.com/overtone/at-at
>>
>> But at-at has not been updated in 6 years, so I assume it is abandoned. I 
>> have two questions about this:
>>
>> 1.) how else do Clojure programmers usually call recurring functionality? 
>>
>> 2.) I am ignorant about the JVM, so I'm afraid I have to ask, at a 
>> fundamental level, how does at-at work? I know that if a function calls 
>> itself recurringly, on the JVM, one eventually gets stackoverflow. So how 
>> does at-at make its magic work? 
>>
>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com 
>> 
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com 
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to clojure+u...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: What do Clojure developers use for recurring functions, other than at-at

[lawrence . krubner]

James Reeves, that does sound like the right way to go. I'll do that. 


On Monday, December 17, 2018 at 3:31:01 PM UTC-5, James Reeves wrote:
>
>
> I'd use an executor:
>
>   (ns example.main
> (:import [java.util.concurrent Executors TimeUnit]))
>
>   (def scheduler
> (Executors/newScheduledThreadPool 32))
>
>   (defn fetch-files []
> (println "Fetching files...))
>
>   (defn -main []
> (.scheduleAtFixedRate scheduler ^Runnable fetch-files 15 15 
> TimeUnit/MINUTES))
>
> On Mon, 17 Dec 2018 at 20:14, > wrote:
>
>> I'm coming back to Clojure development after a year away. This is a fast 
>> moving community and it is hard to keep up when one is not working on it 
>> full time. I'm dusting off some code I wrote 2 years ago, and trying to 
>> bring all the dependencies up to their current versions. 
>>
>> I have a function that fetches files from an AWS S3 bucket, every 15 
>> minutes. I had previously used the at-at library for this:
>>
>> https://github.com/overtone/at-at
>>
>> But at-at has not been updated in 6 years, so I assume it is abandoned. I 
>> have two questions about this:
>>
>> 1.) how else do Clojure programmers usually call recurring functionality? 
>>
>> 2.) I am ignorant about the JVM, so I'm afraid I have to ask, at a 
>> fundamental level, how does at-at work? I know that if a function calls 
>> itself recurringly, on the JVM, one eventually gets stackoverflow. So how 
>> does at-at make its magic work? 
>>
>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com 
>> 
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com 
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to clojure+u...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
> -- 
> James Reeves
> booleanknot.com
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: What do Clojure developers use for recurring functions, other than at-at

[James Reeves]

I'd use an executor:

  (ns example.main
(:import [java.util.concurrent Executors TimeUnit]))

  (def scheduler
(Executors/newScheduledThreadPool 32))

  (defn fetch-files []
(println "Fetching files...))

  (defn -main []
(.scheduleAtFixedRate scheduler ^Runnable fetch-files 15 15
TimeUnit/MINUTES))

On Mon, 17 Dec 2018 at 20:14,  wrote:

> I'm coming back to Clojure development after a year away. This is a fast
> moving community and it is hard to keep up when one is not working on it
> full time. I'm dusting off some code I wrote 2 years ago, and trying to
> bring all the dependencies up to their current versions.
>
> I have a function that fetches files from an AWS S3 bucket, every 15
> minutes. I had previously used the at-at library for this:
>
> https://github.com/overtone/at-at
>
> But at-at has not been updated in 6 years, so I assume it is abandoned. I
> have two questions about this:
>
> 1.) how else do Clojure programmers usually call recurring functionality?
>
> 2.) I am ignorant about the JVM, so I'm afraid I have to ask, at a
> fundamental level, how does at-at work? I know that if a function calls
> itself recurringly, on the JVM, one eventually gets stackoverflow. So how
> does at-at make its magic work?
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>


-- 
James Reeves
booleanknot.com

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: What do Clojure developers use for recurring functions, other than at-at

[Tim Visher]

On Mon, Dec 17, 2018 at 2:54 PM  wrote:

> But at-at has not been updated in 6 years, so I assume it is abandoned. I
> have two questions about this:
>

A common bit of wisdom here in the Clojure community is that time since
last update is not always (or even often) a sign of abandonment but instead
of stability. If there are many and recent open issues on at-at then maybe
it's been abandoned. Or it could just be stable.

Disclaimer, I don't know much about at-at.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: What do Clojure developers use for recurring functions, other than at-at

[Laurens Van Houtven]

Honestly I'd use CloudWatch Timed Events to kick off a Lambda or ECS
Fargate job (which of course you can write in Clojure) assuming you're
using AWS yourself anyway. If you don't care about batching maybe even just
attach a Lambda to the write-to-S3 bucket itself instead of checking every
15m?

If you want to do it in-process, my tool of choice is ztellman's manifold:
https://github.com/ztellman/manifold/blob/d67a8c1b9f1268c094895d70dbbf146521f5774b/src/manifold/time.clj

On Mon, Dec 17, 2018 at 1:54 PM  wrote:

> I'm coming back to Clojure development after a year away. This is a fast
> moving community and it is hard to keep up when one is not working on it
> full time. I'm dusting off some code I wrote 2 years ago, and trying to
> bring all the dependencies up to their current versions.
>
> I have a function that fetches files from an AWS S3 bucket, every 15
> minutes. I had previously used the at-at library for this:
>
> https://github.com/overtone/at-at
>
> But at-at has not been updated in 6 years, so I assume it is abandoned. I
> have two questions about this:
>
> 1.) how else do Clojure programmers usually call recurring functionality?
>
> 2.) I am ignorant about the JVM, so I'm afraid I have to ask, at a
> fundamental level, how does at-at work? I know that if a function calls
> itself recurringly, on the JVM, one eventually gets stackoverflow. So how
> does at-at make its magic work?
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

What do Clojure developers use for recurring functions, other than at-at

[lawrence . krubner]

I'm coming back to Clojure development after a year away. This is a fast 
moving community and it is hard to keep up when one is not working on it 
full time. I'm dusting off some code I wrote 2 years ago, and trying to 
bring all the dependencies up to their current versions. 

I have a function that fetches files from an AWS S3 bucket, every 15 
minutes. I had previously used the at-at library for this:

https://github.com/overtone/at-at

But at-at has not been updated in 6 years, so I assume it is abandoned. I 
have two questions about this:

1.) how else do Clojure programmers usually call recurring functionality? 

2.) I am ignorant about the JVM, so I'm afraid I have to ask, at a 
fundamental level, how does at-at work? I know that if a function calls 
itself recurringly, on the JVM, one eventually gets stackoverflow. So how 
does at-at make its magic work? 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Question: How do you organize large clojure projects?

['somewhat-functional-programmer' via Clojure]

Thank you, it does help.  I almost went the monorepo route but for some reason 
thought I'd have one project.clj file with one giant list of dependencies.  
Having a monorepo but with different dependency sets makes a ton of sense and I 
think would have been much easier to manage.

‐‐‐ Original Message ‐‐‐
On Thursday, November 15, 2018 7:12 AM, Patrik Sundberg 
 wrote:

> I've been happy using a monorepo with boot, and a build.boot with many 
> building blocks that can be mixed and matched for many deployables (uberjars 
> in my case).
>
> In my build.boot I define my internal blocks, but also defs for external deps 
> like eg postgres and grpc. That way I have one place for these groups of 
> dependencies to manage, and can merge them into many of my own building 
> blocks in a flexible and consistent way.
>
> There are no checkouts, multiple outputs and consistent usage of dependencies.
>
> If I need different release management/deployment process for different 
> deployables of my own I tend to use different release branches by deployable 
> that my ci/CD setup triggers from.
>
> Hope that helps!
>
> -
>
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with your 
> first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
>
> 
>
> You received this message because you are subscribed to the Google Groups 
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Question: How do you organize large clojure projects?

[Patrik Sundberg]

I've been happy using a monorepo with boot, and a build.boot with many building 
blocks that can be mixed and matched for many deployables (uberjars in my case).

In my build.boot I define my internal blocks, but also defs for external deps 
like eg postgres and grpc. That way I have one place for these groups of 
dependencies to manage, and can merge them into many of my own building blocks 
in a flexible and consistent way.

There are no checkouts, multiple outputs and consistent usage of dependencies. 

If I need different release management/deployment process for different 
deployables of my own I tend to use different release branches by deployable 
that my ci/CD setup triggers from. 

Hope that helps! 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Question: How do you organize large clojure projects?

['somewhat-functional-programmer' via Clojure]

This is somewhat of a retrospective -- so please bear with me.  I've had the 
privilege of working on a clojure project for a couple of years now, and have 
accumulated some 15-20k lines of clojure code.  I'm taking a little time to 
look back over what has worked for me and what hasn't in terms of code/project 
organization -- and *I'd love to know what has worked for other people (or 
hasn't)* for similarly large projects.

I knew my project was going to grow to at least as much code as it has now at 
the start, and my domain problem was fairly well-defined.  From the very 
beginning I organized my code into many (15-20) different clojure 'projects' 
using lein.  Rather than organizing code into these projects *by function 
area*, I found myself organizing code into projects by their *dependencies*.  
So any code that used libraries X,Y,Z went into a project that declared those 
dependencies -- even if a function made sense in a different namespace by name 
-- if it needed deps that I already had in another project, I moved the 
function to that project.  For example, in the Java GIS world, if you do 
anything with swing components, you can easily pull in 100s of MBs of 
dependencies.  My project involves GIS work both server-side rest-apis, but 
it's also nice to pull up a quick swing component showing data on a map for 
debugging etc.  I don't have these things in the same project even though there 
is some library overlap because the GUI deps are just too many.

I think for me on my project, the only reason to separate anything into 
'projects' is for reuse based on dependencies -- i.e., to use a function/set of 
functions I've written again, what's the minimal amount of deps to pull in.  
It's worked well for me in that sense -- I'm able to create a new 'main' 
project, include libraries that I want from my project, and not pull in 2,000 
dependencies unless I need it.  The dependencies I'm mostly concerned with here 
are Java deps and not clojure deps.  I've found a relatively small core set of 
clojure deps I almost always want available to me (specter, timbre, core.async) 
-- though even still I have a utility library project where I have a hard rule 
of zero dependencies (for basic macros like (ignore-exception body-forms)).

I've used lein's checkouts and managed dependencies fairly successfully though 
I still forget to lein install everything before I lein uberjar my final 
delivery artifacts and end up debugging old code before realizing what happened 
(and my way of installing everything is a bash for loop :-)).

I've found this approach somewhat tedious and have been wondering if there's a 
better way -- and am very curious what others do.

What I've been playing around with lately is a different concept for my own 
code organization:
  - What if all my clojure code could go in one place, or one project? (Even if 
it ended up being 20k+ lines of code)
  - What if namespaces contained their required dependencies in their metadata?
  - What if upon namespace creation, a namespace's dependencies were 
automatically added to the classpath?
  - What if functions declared in a namespace could also declare additional 
dependencies?  These would be added to the classpath upon first invocation of 
the function.  This is great for my seldom used functions that need many 
dependencies -- code could live in the namespace that is matches its function 
instead of squirreled away in a project just to match its deps.

I have written a basic library that does these things, and am currently trying 
it out on a small scale.  Concerns I've already been trying to address:
  - Dynamically adding things to the classpath is generally considered a /bad/ 
thing to do.
- My code reads all pom.xmls on the classpath to determine what libraries 
are already on the classpath -- and does not re-add libraries that are already 
on the classpath.  I think this alone takes care of a lot of issues with 
dynamically adding deps to the classpath (multiple versions of libraries on the 
classpath).
  - Production deployments should not need to calculate classpaths dynamically
- This technique should be able to be used whether the classpath is 
precomputed (i.e., for production), or dynamically during development.

Example namespace loaded deps:
(ns test
  {:dependencies 'com.taoensso/timbre "4.10.0"}
  (:require [taoensso.timbre :as timbre
 :refer [info debug error warn spy]]))
-or-
(ns test
  {:deps '{[[com.taoensso/timbre {:mvn/version "4.10.0"]]}}}
  (:require [taoensso.timbre :as timbre
 :refer [info debug error warn spy]]))

Now these namespace deps would be loaded dynamically by aliasing the ns macro 
for development with one that loads deps dynamically.  In production, the 
metadata is simply attached to the namespace per normal use of the ns form, no 
deps added dynamically.

(defn-deps test-fn-deps
 "Test Function with optional deps"
 {

Re: How do I use spec and deftype together?

[markus . agwin]

Just in case someone stumbles over this post, I found a simple solution 
which works for me:

(do
  (require '[clojure.spec.alpha :as s])
  (import '(clojure.lang ISeq))
  (defprotocol Foo (foo [this]))
  (defrecord Bar [bar] Foo (foo [this] (.bar this)))
  (deftype   Baz [bar]
Foo (foo [this] (.bar this))
ISeq (seq [this] `(~(.bar this
  (s/def ::bar number?)
  [(s/valid? (s/keys :req-un [::bar]) (->Bar 0))
   (s/valid? (s/cat :bar ::bar) (->Baz 0))]) ;[true true]

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

RE: How do I use spec and deftype together?

[Sean Corfield]

So is it best practice to implement map in the deftype to make it spec-able? 
Are there examples somewhere that show how to do this in a canonical way?

If you want a `deftype` that is spec-able as a map, why not just use 
`defrecord`? Manually implementing the various map-related interfaces is a fair 
bit of boilerplate/work.

Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood


From: clojure@googlegroups.com  on behalf of 
markus.ag...@gmail.com 
Sent: Wednesday, July 4, 2018 7:47:45 AM
To: Clojure
Subject: Re: How do I use spec and deftype together?

So is it best practice to implement map in the deftype to make it spec-able? 
Are there examples somewhere that show how to do this in a canonical way?

On Wednesday, July 4, 2018 at 12:51:29 PM UTC+2, Leon Grapenthin wrote:
Spec doesn't check object fields. The defrecord case works because defrecord 
implements map.

On Wednesday, July 4, 2018 at 12:47:26 PM UTC+2, markus...@gmail.com wrote:
The same question was asked on
https://clojurians-log.clojureverse.org/clojure-spec/2018-01-19
but I did not find an answer.

An example, where defrecord works fine but deftype fails:

(do
  (require '[clojure.spec.alpha :as s])
  (defprotocol Foo (foo [this]))
  (defrecord Bar [bar] Foo (foo [this] (.bar this)))
  (deftype   Baz [bar] Foo (foo [this] (.bar this)))
  (s/def ::bar number?)
  [(s/valid? (s/keys :req-un [::bar]) (->Bar 0))
   (s/valid? (s/keys :req-un [::bar]) (->Baz 0))]) ;[true false]

How do I have to change the last line so that it yields true as well?

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 
clojure+unsubscr...@googlegroups.com<mailto:clojure+unsubscr...@googlegroups.com>.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How do I use spec and deftype together?

[markus . agwin]

So is it best practice to implement map in the deftype to make it 
spec-able? Are there examples somewhere that show how to do this in a 
canonical way?

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How do I use spec and deftype together?

[markus . agwin]

So is it best practice to implement map in the deftype to make it 
spec-able? Are there examples somewhere that show how to do this in a 
canonical way?

On Wednesday, July 4, 2018 at 12:51:29 PM UTC+2, Leon Grapenthin wrote:
>
> Spec doesn't check object fields. The defrecord case works because 
> defrecord implements map.
>
> On Wednesday, July 4, 2018 at 12:47:26 PM UTC+2, markus...@gmail.com 
> wrote:
>>
>> The same question was asked on
>> https://clojurians-log.clojureverse.org/clojure-spec/2018-01-19
>> but I did not find an answer.
>>
>> An example, where defrecord works fine but deftype fails:
>>
>> (do
>>   (require '[clojure.spec.alpha :as s])
>>   (defprotocol Foo (foo [this]))
>>   (defrecord Bar [bar] Foo (foo [this] (.bar this)))
>>   (deftype   Baz [bar] Foo (foo [this] (.bar this)))
>>   (s/def ::bar number?)
>>   [(s/valid? (s/keys :req-un [::bar]) (->Bar 0))
>>(s/valid? (s/keys :req-un [::bar]) (->Baz 0))]) ;[true false]
>>
>>
>> How do I have to change the last line so that it yields true as well?
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How do I use spec and deftype together?

[Leon Grapenthin]

Spec doesn't check object fields. The defrecord case works because 
defrecord implements map.

On Wednesday, July 4, 2018 at 12:47:26 PM UTC+2, markus...@gmail.com wrote:
>
> The same question was asked on
> https://clojurians-log.clojureverse.org/clojure-spec/2018-01-19
> but I did not find an answer.
>
> An example, where defrecord works fine but deftype fails:
>
> (do
>   (require '[clojure.spec.alpha :as s])
>   (defprotocol Foo (foo [this]))
>   (defrecord Bar [bar] Foo (foo [this] (.bar this)))
>   (deftype   Baz [bar] Foo (foo [this] (.bar this)))
>   (s/def ::bar number?)
>   [(s/valid? (s/keys :req-un [::bar]) (->Bar 0))
>(s/valid? (s/keys :req-un [::bar]) (->Baz 0))]) ;[true false]
>
>
> How do I have to change the last line so that it yields true as well?
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

How do I use spec and deftype together?

[markus . agwin]

The same question was asked on
https://clojurians-log.clojureverse.org/clojure-spec/2018-01-19
but I did not find an answer.

An example, where defrecord works fine but deftype fails:

(do
  (require '[clojure.spec.alpha :as s])
  (defprotocol Foo (foo [this]))
  (defrecord Bar [bar] Foo (foo [this] (.bar this)))
  (deftype   Baz [bar] Foo (foo [this] (.bar this)))
  (s/def ::bar number?)
  [(s/valid? (s/keys :req-un [::bar]) (->Bar 0))
   (s/valid? (s/keys :req-un [::bar]) (->Baz 0))]) ;[true false]


How do I have to change the last line so that it yields true as well?

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: What does the ref *loaded-libs* do?

[Stephen Gilardi]

It is there to support the “:reload” and “:reload-all” features of “require” 
and to help separate the concern of loading libs from the concern of tracking 
namespaces.

--Steve

> On Jan 29, 2018, at 6:36 PM, Raymond Huang <12ay.hu...@gmail.com> wrote:
> 
> I was poking around `tools.namespace` and I found it interesting that the 
> implementation of `remove-lib` is:
> 
> ```(defn remove-lib
>   "Remove lib's namespace and remove lib from the set of loaded libs."
>   [lib]
>   (remove-ns lib)
>   (dosync (alter @#'clojure.core/*loaded-libs* disj lib)))
> ```
> 
> 
> I’m wondering if someone can enlighten me to explain why `*loaded-libs*` 
> needs to exist, as opposed to checking `clojure.lang.Namespace` directly?
> 
> This is a question carried over from slack: 
> https://clojurians.slack.com/archives/C03S1KBA2/p151721185721
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with your 
> first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

What does the ref *loaded-libs* do?

[squeegee]

It is there to support the “:reload” and “:reload-all” features of “require” 
and to help separate the concern of loading libs from the concern of tracking 
namespaces.

--Steve

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: what does future do after fn finish ?

[Jacek Grzebyta]

HI,

So things works well. In terms of memory usage anyway. I use 50 main
threads (in process-data-by-condition-set) and 20 for replications
(process-growths). However I found that after good performance in the
beginning it slows down rapid and quick. VisualVM shows the threads usage
is poor - mainly they are waiting. Only 1 claypoole thread works + some
lower level threads managed probably by Clojure, etc. CPU usage is poor as
well. I guess the reason is on the lower - Java classes level with IO
operations bottleneck. IMO they are working by blocking. That why I wanted
to separate them. However IO speed operations go down as well.
I've tried async channels but that they were flooded and some data where
lost for some reason.

Regards,
Jacek


On 3 February 2018 at 00:14, Justin Smith <noisesm...@gmail.com> wrote:

> -> is just a list transform performed after reading your code into a list
> data structure containing symbols, and before compiling to byte code - it
> doesn't do anything directly.
>
> On Fri, Feb 2, 2018 at 3:55 PM Jacek Grzebyta <grzebyta@gmail.com>
> wrote:
>
>> OK I found what makes the memory leak.
>>
>> In the project I work with I use a  java Model class which is java
>> Collection proxy/facade for a single transaction. Unfortunately it's not
>> thread safe. In a few places I passed single instance of model into several
>> threads Also I requested the instance with -> which makes new thread as
>> well. I was surprised that -> makes trouble but after thinking that might
>> be expected. Especially that internally the wrapper doesn't do simple
>> mapping - it uses some iterator, etc. Anyway the machinery is fragile. It
>> seems I need to rewrite code and replace all multithreading parts by
>> something simpler.
>>
>>
>>
>> If you want see the stacktrace just look at:
>> https://github.com/jgrzebyta/triple-loader/issues/53
>>
>> On 2 February 2018 at 11:16, Jacek Grzebyta <grzebyta@gmail.com>
>> wrote:
>>
>>>
>>> On 2 February 2018 at 08:34, Niels van Klaveren <
>>> niels.vanklave...@gmail.com> wrote:
>>>
>>>> +1 for Claypoole, it removed the needs of using agents or futures in
>>>> 95% of the cases in my code.
>>>>
>>>>
>>> Thanks a lot. I modify the code using claypoole. I imagine with-shutdown
>>> will close the pool properly after finish all tasks so there is no need to
>>> watch them?
>>>
>>> J
>>>
>>>
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: what does future do after fn finish ?

[Justin Smith]

-> is just a list transform performed after reading your code into a list
data structure containing symbols, and before compiling to byte code - it
doesn't do anything directly.

On Fri, Feb 2, 2018 at 3:55 PM Jacek Grzebyta <grzebyta@gmail.com>
wrote:

> OK I found what makes the memory leak.
>
> In the project I work with I use a  java Model class which is java
> Collection proxy/facade for a single transaction. Unfortunately it's not
> thread safe. In a few places I passed single instance of model into several
> threads Also I requested the instance with -> which makes new thread as
> well. I was surprised that -> makes trouble but after thinking that might
> be expected. Especially that internally the wrapper doesn't do simple
> mapping - it uses some iterator, etc. Anyway the machinery is fragile. It
> seems I need to rewrite code and replace all multithreading parts by
> something simpler.
>
>
>
> If you want see the stacktrace just look at:
> https://github.com/jgrzebyta/triple-loader/issues/53
>
> On 2 February 2018 at 11:16, Jacek Grzebyta <grzebyta@gmail.com>
> wrote:
>
>>
>> On 2 February 2018 at 08:34, Niels van Klaveren <
>> niels.vanklave...@gmail.com> wrote:
>>
>>> +1 for Claypoole, it removed the needs of using agents or futures in 95%
>>> of the cases in my code.
>>>
>>>
>> Thanks a lot. I modify the code using claypoole. I imagine with-shutdown
>> will close the pool properly after finish all tasks so there is no need to
>> watch them?
>>
>> J
>>
>>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: what does future do after fn finish ?

[Jacek Grzebyta]

OK I found what makes the memory leak.

In the project I work with I use a  java Model class which is java
Collection proxy/facade for a single transaction. Unfortunately it's not
thread safe. In a few places I passed single instance of model into several
threads Also I requested the instance with -> which makes new thread as
well. I was surprised that -> makes trouble but after thinking that might
be expected. Especially that internally the wrapper doesn't do simple
mapping - it uses some iterator, etc. Anyway the machinery is fragile. It
seems I need to rewrite code and replace all multithreading parts by
something simpler.



If you want see the stacktrace just look at:
https://github.com/jgrzebyta/triple-loader/issues/53

On 2 February 2018 at 11:16, Jacek Grzebyta <grzebyta@gmail.com> wrote:

>
> On 2 February 2018 at 08:34, Niels van Klaveren <
> niels.vanklave...@gmail.com> wrote:
>
>> +1 for Claypoole, it removed the needs of using agents or futures in 95%
>> of the cases in my code.
>>
>>
> Thanks a lot. I modify the code using claypoole. I imagine with-shutdown
> will close the pool properly after finish all tasks so there is no need to
> watch them?
>
> J
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: what does future do after fn finish ?

[Jacek Grzebyta]

On 2 February 2018 at 08:34, Niels van Klaveren  wrote:

> +1 for Claypoole, it removed the needs of using agents or futures in 95%
> of the cases in my code.
>
>
Thanks a lot. I modify the code using claypoole. I imagine with-shutdown
will close the pool properly after finish all tasks so there is no need to
watch them?

J

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: what does future do after fn finish ?

[Niels van Klaveren]

+1 for Claypoole, it removed the needs of using agents or futures in 95% of 
the cases in my code.

On Thursday, February 1, 2018 at 9:54:36 PM UTC+1, Alan Thompson wrote:
>
> You may find that using the Claypoole library is the easiest way to handle 
> threadpools:   https://github.com/TheClimateCorporation/claypoole
> Alan
>
> On Thu, Feb 1, 2018 at 11:16 AM, Justin Smith <noise...@gmail.com 
> > wrote:
>
>> yes, that's the idea exactly
>>
>> also, you might want more fine grained control of how much parallelism 
>> occurs (eg. if every thread is writing to the same physical device, you can 
>> often get better throughput by not parallelizing at all, or keeping the 
>> parallelism quite limited - it's worth experimenting) - there are good ways 
>> to control that using ThreadPoolExecutor directly, or using 
>> clojure.lang.PersistentQueue/EMPTY as a control construct, or core.async 
>> channels, or ztellman's manifold library, or the claypoole threading library
>>
>> On Thu, Feb 1, 2018, 03:44 Jacek Grzebyta <grzeby...@gmail.com 
>> > wrote:
>>
>>> Thanks folks. I see now! It should be a list of agents not list of 
>>> futures within agent.  Also any task sent to a agent is processed 
>>> within a thread anyway so I do not need to add future...
>>>
>>> On 1 February 2018 at 02:17, John Newman <joh...@gmail.com 
>>> > wrote:
>>>
>>>> Ah, he's using one agent, I see.
>>>>
>>>> On Jan 31, 2018 9:15 PM, "John Newman" <joh...@gmail.com > 
>>>> wrote:
>>>>
>>>>> Multiple sen-doffs to one agent will serialize it's calls, but 
>>>>> spawning agents on each new task will spawn threads on a bounded thread 
>>>>> pool, I believe.
>>>>>
>>>>> On Jan 31, 2018 8:32 PM, "Justin Smith" <noise...@gmail.com 
>>>>> > wrote:
>>>>>
>>>>>> Doing all the actions via one agent means that the actions are 
>>>>>> serialized though - you end up with no performance improvement over 
>>>>>> doing 
>>>>>> them all in a doseq in one future - the right way to do this tends to be 
>>>>>> trickier than it looks at first glance, and depends on your 
>>>>>> requirements. 
>>>>>> agents, the claypoole library, and reducers are all potentially useful. 
>>>>>> If 
>>>>>> parallelization leads to complex coordination needs, core.async can help 
>>>>>> too.
>>>>>>
>>>>>> On Wed, Jan 31, 2018 at 5:18 PM John Newman <joh...@gmail.com 
>>>>>> > wrote:
>>>>>>
>>>>>>> Agents manage a pool of threads for you. Try doing it without the 
>>>>>>> future call and see if that works (unless you're trying to do something 
>>>>>>> else).
>>>>>>>
>>>>>>> John
>>>>>>>
>>>>>>> On Wed, Jan 31, 2018 at 7:31 PM, Jacek Grzebyta <grzeby...@gmail.com 
>>>>>>> > wrote:
>>>>>>>
>>>>>>>> Thanks a lot. I will check it tomorrow. 
>>>>>>>>
>>>>>>>> J
>>>>>>>>
>>>>>>>> On 1 Feb 2018 12:12 a.m., "Justin Smith" <noise...@gmail.com 
>>>>>>>> > wrote:
>>>>>>>>
>>>>>>>>> this is exactly the kind of problem code I was describing - 
>>>>>>>>> there's no backpressure on existing future tasks to hold up the 
>>>>>>>>> launching 
>>>>>>>>> of more futures - the work done by the agent calling conj is 
>>>>>>>>> negligible. 
>>>>>>>>> You need to control the size of the pool of threads used, and you 
>>>>>>>>> need to 
>>>>>>>>> impose back-pressure.
>>>>>>>>>
>>>>>>>>> On Wed, Jan 31, 2018 at 3:46 PM Jacek Grzebyta <
>>>>>>>>> grzeby...@gmail.com > wrote:
>>>>>>>>>
>>>>>>>>>> On 31 January 2018 at 18:08, James Reeves <ja...@booleanknot.com 
>>>>>>>>>> > wrote:
>>>>>>>>>>
>>>>>>>>>>> On 31 January 2018 at 17

[job] S.F Bay Area: Enjoy using Clojure to do a bit of everyhing?

[Arthur Ulfeldt]

Yummly, Redwood City, CA

I'm looking for folks who genuinly enjoy using Clojure to do a great 
variety of things like Backend mobile APIs, Web services, Infrastructre/AWS 
automation, security scanning, and all sorts of other things. It's really a 
lot of fun.

If you're intersted please email me: art...@yummly.com or call me at 
1-219-CLOJURE

This is honestly the happiest team i have worked with (small sample size 
bias)

https://careers.jobscore.com/careers/yummly/jobs/generalist-software-engineer-dnDs0w99Kr57z9dG1ZS6tF?sid=161

and in general about working at yummly:
https://www.yummly.com/careers

most of us work remotly some of the time and everyone is in the office much 
of the time as well. It's a really nice office and friendly pet friendly. 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: what does future do after fn finish ?

[Alan Thompson]

You may find that using the Claypoole library is the easiest way to handle
threadpools:   https://github.com/TheClimateCorporation/claypoole
Alan

On Thu, Feb 1, 2018 at 11:16 AM, Justin Smith <noisesm...@gmail.com> wrote:

> yes, that's the idea exactly
>
> also, you might want more fine grained control of how much parallelism
> occurs (eg. if every thread is writing to the same physical device, you can
> often get better throughput by not parallelizing at all, or keeping the
> parallelism quite limited - it's worth experimenting) - there are good ways
> to control that using ThreadPoolExecutor directly, or using
> clojure.lang.PersistentQueue/EMPTY as a control construct, or core.async
> channels, or ztellman's manifold library, or the claypoole threading library
>
> On Thu, Feb 1, 2018, 03:44 Jacek Grzebyta <grzebyta@gmail.com> wrote:
>
>> Thanks folks. I see now! It should be a list of agents not list of
>> futures within agent.  Also any task sent to a agent is processed
>> within a thread anyway so I do not need to add future...
>>
>> On 1 February 2018 at 02:17, John Newman <john...@gmail.com> wrote:
>>
>>> Ah, he's using one agent, I see.
>>>
>>> On Jan 31, 2018 9:15 PM, "John Newman" <john...@gmail.com> wrote:
>>>
>>>> Multiple sen-doffs to one agent will serialize it's calls, but spawning
>>>> agents on each new task will spawn threads on a bounded thread pool, I
>>>> believe.
>>>>
>>>> On Jan 31, 2018 8:32 PM, "Justin Smith" <noisesm...@gmail.com> wrote:
>>>>
>>>>> Doing all the actions via one agent means that the actions are
>>>>> serialized though - you end up with no performance improvement over doing
>>>>> them all in a doseq in one future - the right way to do this tends to be
>>>>> trickier than it looks at first glance, and depends on your requirements.
>>>>> agents, the claypoole library, and reducers are all potentially useful. If
>>>>> parallelization leads to complex coordination needs, core.async can help
>>>>> too.
>>>>>
>>>>> On Wed, Jan 31, 2018 at 5:18 PM John Newman <john...@gmail.com> wrote:
>>>>>
>>>>>> Agents manage a pool of threads for you. Try doing it without the
>>>>>> future call and see if that works (unless you're trying to do something
>>>>>> else).
>>>>>>
>>>>>> John
>>>>>>
>>>>>> On Wed, Jan 31, 2018 at 7:31 PM, Jacek Grzebyta <
>>>>>> grzebyta@gmail.com> wrote:
>>>>>>
>>>>>>> Thanks a lot. I will check it tomorrow.
>>>>>>>
>>>>>>> J
>>>>>>>
>>>>>>> On 1 Feb 2018 12:12 a.m., "Justin Smith" <noisesm...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> this is exactly the kind of problem code I was describing - there's
>>>>>>>> no backpressure on existing future tasks to hold up the launching of 
>>>>>>>> more
>>>>>>>> futures - the work done by the agent calling conj is negligible. You 
>>>>>>>> need
>>>>>>>> to control the size of the pool of threads used, and you need to impose
>>>>>>>> back-pressure.
>>>>>>>>
>>>>>>>> On Wed, Jan 31, 2018 at 3:46 PM Jacek Grzebyta <
>>>>>>>> grzebyta@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> On 31 January 2018 at 18:08, James Reeves <ja...@booleanknot.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> On 31 January 2018 at 17:59, Jacek Grzebyta <
>>>>>>>>>> grzebyta@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> I have application with quite intense tripe store populating
>>>>>>>>>>> ~30/40 k records per chunk (139 portions). The data are wrapped 
>>>>>>>>>>> within the
>>>>>>>>>>> future:
>>>>>>>>>>>
>>>>>>>>>>> (conj agent (future (apply task args)))
>>>>>>>>>>>
>>>>>>>>>>>  and that all together is send-off into (agent 

Re: what does future do after fn finish ?

[Justin Smith]

yes, that's the idea exactly

also, you might want more fine grained control of how much parallelism
occurs (eg. if every thread is writing to the same physical device, you can
often get better throughput by not parallelizing at all, or keeping the
parallelism quite limited - it's worth experimenting) - there are good ways
to control that using ThreadPoolExecutor directly, or using
clojure.lang.PersistentQueue/EMPTY as a control construct, or core.async
channels, or ztellman's manifold library, or the claypoole threading library

On Thu, Feb 1, 2018, 03:44 Jacek Grzebyta <grzebyta@gmail.com> wrote:

> Thanks folks. I see now! It should be a list of agents not list of futures
> within agent.  Also any task sent to a agent is processed within a
> thread anyway so I do not need to add future...
>
> On 1 February 2018 at 02:17, John Newman <john...@gmail.com> wrote:
>
>> Ah, he's using one agent, I see.
>>
>> On Jan 31, 2018 9:15 PM, "John Newman" <john...@gmail.com> wrote:
>>
>>> Multiple sen-doffs to one agent will serialize it's calls, but spawning
>>> agents on each new task will spawn threads on a bounded thread pool, I
>>> believe.
>>>
>>> On Jan 31, 2018 8:32 PM, "Justin Smith" <noisesm...@gmail.com> wrote:
>>>
>>>> Doing all the actions via one agent means that the actions are
>>>> serialized though - you end up with no performance improvement over doing
>>>> them all in a doseq in one future - the right way to do this tends to be
>>>> trickier than it looks at first glance, and depends on your requirements.
>>>> agents, the claypoole library, and reducers are all potentially useful. If
>>>> parallelization leads to complex coordination needs, core.async can help
>>>> too.
>>>>
>>>> On Wed, Jan 31, 2018 at 5:18 PM John Newman <john...@gmail.com> wrote:
>>>>
>>>>> Agents manage a pool of threads for you. Try doing it without the
>>>>> future call and see if that works (unless you're trying to do something
>>>>> else).
>>>>>
>>>>> John
>>>>>
>>>>> On Wed, Jan 31, 2018 at 7:31 PM, Jacek Grzebyta <
>>>>> grzebyta@gmail.com> wrote:
>>>>>
>>>>>> Thanks a lot. I will check it tomorrow.
>>>>>>
>>>>>> J
>>>>>>
>>>>>> On 1 Feb 2018 12:12 a.m., "Justin Smith" <noisesm...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> this is exactly the kind of problem code I was describing - there's
>>>>>>> no backpressure on existing future tasks to hold up the launching of 
>>>>>>> more
>>>>>>> futures - the work done by the agent calling conj is negligible. You 
>>>>>>> need
>>>>>>> to control the size of the pool of threads used, and you need to impose
>>>>>>> back-pressure.
>>>>>>>
>>>>>>> On Wed, Jan 31, 2018 at 3:46 PM Jacek Grzebyta <
>>>>>>> grzebyta@gmail.com> wrote:
>>>>>>>
>>>>>>>> On 31 January 2018 at 18:08, James Reeves <ja...@booleanknot.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> On 31 January 2018 at 17:59, Jacek Grzebyta <
>>>>>>>>> grzebyta@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> I have application with quite intense tripe store populating
>>>>>>>>>> ~30/40 k records per chunk (139 portions). The data are wrapped 
>>>>>>>>>> within the
>>>>>>>>>> future:
>>>>>>>>>>
>>>>>>>>>> (conj agent (future (apply task args)))
>>>>>>>>>>
>>>>>>>>>>  and that all together is send-off into (agent []).
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> What is "agent"? The first line of code indicates that it's a
>>>>>>>>> local collection shadowing the code function, while the second code 
>>>>>>>>> snippet
>>>>>>>>> indicates that you're using the core agent function.
>>>>>>>>>
>>>>>>>>> Also why are you sending off to an agent?
>>>

Re: what does future do after fn finish ?

[Jacek Grzebyta]

Thanks folks. I see now! It should be a list of agents not list of futures
within agent.  Also any task sent to a agent is processed within a
thread anyway so I do not need to add future...

On 1 February 2018 at 02:17, John Newman <john...@gmail.com> wrote:

> Ah, he's using one agent, I see.
>
> On Jan 31, 2018 9:15 PM, "John Newman" <john...@gmail.com> wrote:
>
>> Multiple sen-doffs to one agent will serialize it's calls, but spawning
>> agents on each new task will spawn threads on a bounded thread pool, I
>> believe.
>>
>> On Jan 31, 2018 8:32 PM, "Justin Smith" <noisesm...@gmail.com> wrote:
>>
>>> Doing all the actions via one agent means that the actions are
>>> serialized though - you end up with no performance improvement over doing
>>> them all in a doseq in one future - the right way to do this tends to be
>>> trickier than it looks at first glance, and depends on your requirements.
>>> agents, the claypoole library, and reducers are all potentially useful. If
>>> parallelization leads to complex coordination needs, core.async can help
>>> too.
>>>
>>> On Wed, Jan 31, 2018 at 5:18 PM John Newman <john...@gmail.com> wrote:
>>>
>>>> Agents manage a pool of threads for you. Try doing it without the
>>>> future call and see if that works (unless you're trying to do something
>>>> else).
>>>>
>>>> John
>>>>
>>>> On Wed, Jan 31, 2018 at 7:31 PM, Jacek Grzebyta <grzebyta@gmail.com
>>>> > wrote:
>>>>
>>>>> Thanks a lot. I will check it tomorrow.
>>>>>
>>>>> J
>>>>>
>>>>> On 1 Feb 2018 12:12 a.m., "Justin Smith" <noisesm...@gmail.com> wrote:
>>>>>
>>>>>> this is exactly the kind of problem code I was describing - there's
>>>>>> no backpressure on existing future tasks to hold up the launching of more
>>>>>> futures - the work done by the agent calling conj is negligible. You need
>>>>>> to control the size of the pool of threads used, and you need to impose
>>>>>> back-pressure.
>>>>>>
>>>>>> On Wed, Jan 31, 2018 at 3:46 PM Jacek Grzebyta <
>>>>>> grzebyta@gmail.com> wrote:
>>>>>>
>>>>>>> On 31 January 2018 at 18:08, James Reeves <ja...@booleanknot.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> On 31 January 2018 at 17:59, Jacek Grzebyta <grzebyta@gmail.com
>>>>>>>> > wrote:
>>>>>>>>
>>>>>>>>> I have application with quite intense tripe store populating
>>>>>>>>> ~30/40 k records per chunk (139 portions). The data are wrapped 
>>>>>>>>> within the
>>>>>>>>> future:
>>>>>>>>>
>>>>>>>>> (conj agent (future (apply task args)))
>>>>>>>>>
>>>>>>>>>  and that all together is send-off into (agent []).
>>>>>>>>>
>>>>>>>>
>>>>>>>> What is "agent"? The first line of code indicates that it's a local
>>>>>>>> collection shadowing the code function, while the second code snippet
>>>>>>>> indicates that you're using the core agent function.
>>>>>>>>
>>>>>>>> Also why are you sending off to an agent?
>>>>>>>>
>>>>>>>
>>>>>>> I have ~8sec computing task for each input dataset which generates
>>>>>>> those records. After that I write it into disk (in software-specific
>>>>>>> transaction). I just wanted to separate hard computing and io 
>>>>>>> operations. I
>>>>>>> created a side-effect method which is injected together with the dataset
>>>>>>> into a future. The futures are async collected within a list wrapped in
>>>>>>> agent. After the computing the main thread is waiting until all io tasks
>>>>>>> will be finished.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> At the end of the main thread function I just use await-for and
>>>>>>>>> after that:
>>>>>>>>>
>>>>>

Re: what does future do after fn finish ?

[John Newman]

Ah, he's using one agent, I see.

On Jan 31, 2018 9:15 PM, "John Newman" <john...@gmail.com> wrote:

> Multiple sen-doffs to one agent will serialize it's calls, but spawning
> agents on each new task will spawn threads on a bounded thread pool, I
> believe.
>
> On Jan 31, 2018 8:32 PM, "Justin Smith" <noisesm...@gmail.com> wrote:
>
>> Doing all the actions via one agent means that the actions are serialized
>> though - you end up with no performance improvement over doing them all in
>> a doseq in one future - the right way to do this tends to be trickier than
>> it looks at first glance, and depends on your requirements. agents, the
>> claypoole library, and reducers are all potentially useful. If
>> parallelization leads to complex coordination needs, core.async can help
>> too.
>>
>> On Wed, Jan 31, 2018 at 5:18 PM John Newman <john...@gmail.com> wrote:
>>
>>> Agents manage a pool of threads for you. Try doing it without the future
>>> call and see if that works (unless you're trying to do something else).
>>>
>>> John
>>>
>>> On Wed, Jan 31, 2018 at 7:31 PM, Jacek Grzebyta <grzebyta@gmail.com>
>>> wrote:
>>>
>>>> Thanks a lot. I will check it tomorrow.
>>>>
>>>> J
>>>>
>>>> On 1 Feb 2018 12:12 a.m., "Justin Smith" <noisesm...@gmail.com> wrote:
>>>>
>>>>> this is exactly the kind of problem code I was describing - there's no
>>>>> backpressure on existing future tasks to hold up the launching of more
>>>>> futures - the work done by the agent calling conj is negligible. You need
>>>>> to control the size of the pool of threads used, and you need to impose
>>>>> back-pressure.
>>>>>
>>>>> On Wed, Jan 31, 2018 at 3:46 PM Jacek Grzebyta <grzebyta@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> On 31 January 2018 at 18:08, James Reeves <ja...@booleanknot.com>
>>>>>> wrote:
>>>>>>
>>>>>>> On 31 January 2018 at 17:59, Jacek Grzebyta <grzebyta@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I have application with quite intense tripe store populating ~30/40
>>>>>>>> k records per chunk (139 portions). The data are wrapped within the 
>>>>>>>> future:
>>>>>>>>
>>>>>>>> (conj agent (future (apply task args)))
>>>>>>>>
>>>>>>>>  and that all together is send-off into (agent []).
>>>>>>>>
>>>>>>>
>>>>>>> What is "agent"? The first line of code indicates that it's a local
>>>>>>> collection shadowing the code function, while the second code snippet
>>>>>>> indicates that you're using the core agent function.
>>>>>>>
>>>>>>> Also why are you sending off to an agent?
>>>>>>>
>>>>>>
>>>>>> I have ~8sec computing task for each input dataset which generates
>>>>>> those records. After that I write it into disk (in software-specific
>>>>>> transaction). I just wanted to separate hard computing and io 
>>>>>> operations. I
>>>>>> created a side-effect method which is injected together with the dataset
>>>>>> into a future. The futures are async collected within a list wrapped in
>>>>>> agent. After the computing the main thread is waiting until all io tasks
>>>>>> will be finished.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> At the end of the main thread function I just use await-for and
>>>>>>>> after that:
>>>>>>>>
>>>>>>>> (reduce + (map #(deref %) @data-loading-tasks))
>>>>>>>>
>>>>>>>
>>>>>> As a control, tasks return number of written records.
>>>>>>
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>> For some reason I see the happy collecting (see attached screenshot
>>>>>>>> of jconsole).
>>>>>>>>
>>>>>>>
>>>>>>> "happy" = "heap"?
>>>>>>>
>>>>>>

Re: what does future do after fn finish ?

[John Newman]

Multiple sen-doffs to one agent will serialize it's calls, but spawning
agents on each new task will spawn threads on a bounded thread pool, I
believe.

On Jan 31, 2018 8:32 PM, "Justin Smith" <noisesm...@gmail.com> wrote:

> Doing all the actions via one agent means that the actions are serialized
> though - you end up with no performance improvement over doing them all in
> a doseq in one future - the right way to do this tends to be trickier than
> it looks at first glance, and depends on your requirements. agents, the
> claypoole library, and reducers are all potentially useful. If
> parallelization leads to complex coordination needs, core.async can help
> too.
>
> On Wed, Jan 31, 2018 at 5:18 PM John Newman <john...@gmail.com> wrote:
>
>> Agents manage a pool of threads for you. Try doing it without the future
>> call and see if that works (unless you're trying to do something else).
>>
>> John
>>
>> On Wed, Jan 31, 2018 at 7:31 PM, Jacek Grzebyta <grzebyta@gmail.com>
>> wrote:
>>
>>> Thanks a lot. I will check it tomorrow.
>>>
>>> J
>>>
>>> On 1 Feb 2018 12:12 a.m., "Justin Smith" <noisesm...@gmail.com> wrote:
>>>
>>>> this is exactly the kind of problem code I was describing - there's no
>>>> backpressure on existing future tasks to hold up the launching of more
>>>> futures - the work done by the agent calling conj is negligible. You need
>>>> to control the size of the pool of threads used, and you need to impose
>>>> back-pressure.
>>>>
>>>> On Wed, Jan 31, 2018 at 3:46 PM Jacek Grzebyta <grzebyta@gmail.com>
>>>> wrote:
>>>>
>>>>> On 31 January 2018 at 18:08, James Reeves <ja...@booleanknot.com>
>>>>> wrote:
>>>>>
>>>>>> On 31 January 2018 at 17:59, Jacek Grzebyta <grzebyta@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I have application with quite intense tripe store populating ~30/40
>>>>>>> k records per chunk (139 portions). The data are wrapped within the 
>>>>>>> future:
>>>>>>>
>>>>>>> (conj agent (future (apply task args)))
>>>>>>>
>>>>>>>  and that all together is send-off into (agent []).
>>>>>>>
>>>>>>
>>>>>> What is "agent"? The first line of code indicates that it's a local
>>>>>> collection shadowing the code function, while the second code snippet
>>>>>> indicates that you're using the core agent function.
>>>>>>
>>>>>> Also why are you sending off to an agent?
>>>>>>
>>>>>
>>>>> I have ~8sec computing task for each input dataset which generates
>>>>> those records. After that I write it into disk (in software-specific
>>>>> transaction). I just wanted to separate hard computing and io operations. 
>>>>> I
>>>>> created a side-effect method which is injected together with the dataset
>>>>> into a future. The futures are async collected within a list wrapped in
>>>>> agent. After the computing the main thread is waiting until all io tasks
>>>>> will be finished.
>>>>>
>>>>>
>>>>>>
>>>>>> At the end of the main thread function I just use await-for and after
>>>>>>> that:
>>>>>>>
>>>>>>> (reduce + (map #(deref %) @data-loading-tasks))
>>>>>>>
>>>>>>
>>>>> As a control, tasks return number of written records.
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>> For some reason I see the happy collecting (see attached screenshot
>>>>>>> of jconsole).
>>>>>>>
>>>>>>
>>>>>> "happy" = "heap"?
>>>>>>
>>>>>
>>>>> Both. As you can see on attached screenshot the heap usage grows easy
>>>>> until aver. ~2 1/4 G than keep that  for a few minutes. In that moment I
>>>>> stopped. After that starts grow till ~4G with tendency to do jumps a bit
>>>>> more that 4G.
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>>> After seeing the source code of future I suppose that t

Re: what does future do after fn finish ?

[Justin Smith]

Doing all the actions via one agent means that the actions are serialized
though - you end up with no performance improvement over doing them all in
a doseq in one future - the right way to do this tends to be trickier than
it looks at first glance, and depends on your requirements. agents, the
claypoole library, and reducers are all potentially useful. If
parallelization leads to complex coordination needs, core.async can help
too.

On Wed, Jan 31, 2018 at 5:18 PM John Newman <john...@gmail.com> wrote:

> Agents manage a pool of threads for you. Try doing it without the future
> call and see if that works (unless you're trying to do something else).
>
> John
>
> On Wed, Jan 31, 2018 at 7:31 PM, Jacek Grzebyta <grzebyta@gmail.com>
> wrote:
>
>> Thanks a lot. I will check it tomorrow.
>>
>> J
>>
>> On 1 Feb 2018 12:12 a.m., "Justin Smith" <noisesm...@gmail.com> wrote:
>>
>>> this is exactly the kind of problem code I was describing - there's no
>>> backpressure on existing future tasks to hold up the launching of more
>>> futures - the work done by the agent calling conj is negligible. You need
>>> to control the size of the pool of threads used, and you need to impose
>>> back-pressure.
>>>
>>> On Wed, Jan 31, 2018 at 3:46 PM Jacek Grzebyta <grzebyta@gmail.com>
>>> wrote:
>>>
>>>> On 31 January 2018 at 18:08, James Reeves <ja...@booleanknot.com>
>>>> wrote:
>>>>
>>>>> On 31 January 2018 at 17:59, Jacek Grzebyta <grzebyta@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I have application with quite intense tripe store populating ~30/40 k
>>>>>> records per chunk (139 portions). The data are wrapped within the future:
>>>>>>
>>>>>> (conj agent (future (apply task args)))
>>>>>>
>>>>>>  and that all together is send-off into (agent []).
>>>>>>
>>>>>
>>>>> What is "agent"? The first line of code indicates that it's a local
>>>>> collection shadowing the code function, while the second code snippet
>>>>> indicates that you're using the core agent function.
>>>>>
>>>>> Also why are you sending off to an agent?
>>>>>
>>>>
>>>> I have ~8sec computing task for each input dataset which generates
>>>> those records. After that I write it into disk (in software-specific
>>>> transaction). I just wanted to separate hard computing and io operations. I
>>>> created a side-effect method which is injected together with the dataset
>>>> into a future. The futures are async collected within a list wrapped in
>>>> agent. After the computing the main thread is waiting until all io tasks
>>>> will be finished.
>>>>
>>>>
>>>>>
>>>>> At the end of the main thread function I just use await-for and after
>>>>>> that:
>>>>>>
>>>>>> (reduce + (map #(deref %) @data-loading-tasks))
>>>>>>
>>>>>
>>>> As a control, tasks return number of written records.
>>>>
>>>>
>>>>
>>>>>
>>>>>> For some reason I see the happy collecting (see attached screenshot
>>>>>> of jconsole).
>>>>>>
>>>>>
>>>>> "happy" = "heap"?
>>>>>
>>>>
>>>> Both. As you can see on attached screenshot the heap usage grows easy
>>>> until aver. ~2 1/4 G than keep that  for a few minutes. In that moment I
>>>> stopped. After that starts grow till ~4G with tendency to do jumps a bit
>>>> more that 4G.
>>>>
>>>>
>>>>>
>>>>>
>>>>>> After seeing the source code of future I suppose that the memory
>>>>>> (data are kept as #{} set) is not released. The task returns only integer
>>>>>> so I do not think that might cause the problem.
>>>>>>
>>>>>
>>>>> Can you provide more detail? You keep alluding to things that you
>>>>> don't provide code for, such as the sets of data.
>>>>>
>>>>
>>>>
>>>> The code is attached. However the important code is
>>>>
>>>> L123 .
>>>>   (let [;; keeps all data loading futures.
>>>> ;; waiting until all 

Re: what does future do after fn finish ?

[John Newman]

Agents manage a pool of threads for you. Try doing it without the future
call and see if that works (unless you're trying to do something else).

John

On Wed, Jan 31, 2018 at 7:31 PM, Jacek Grzebyta <grzebyta@gmail.com>
wrote:

> Thanks a lot. I will check it tomorrow.
>
> J
>
> On 1 Feb 2018 12:12 a.m., "Justin Smith" <noisesm...@gmail.com> wrote:
>
>> this is exactly the kind of problem code I was describing - there's no
>> backpressure on existing future tasks to hold up the launching of more
>> futures - the work done by the agent calling conj is negligible. You need
>> to control the size of the pool of threads used, and you need to impose
>> back-pressure.
>>
>> On Wed, Jan 31, 2018 at 3:46 PM Jacek Grzebyta <grzebyta@gmail.com>
>> wrote:
>>
>>> On 31 January 2018 at 18:08, James Reeves <ja...@booleanknot.com> wrote:
>>>
>>>> On 31 January 2018 at 17:59, Jacek Grzebyta <grzebyta@gmail.com>
>>>> wrote:
>>>>
>>>>> I have application with quite intense tripe store populating ~30/40 k
>>>>> records per chunk (139 portions). The data are wrapped within the future:
>>>>>
>>>>> (conj agent (future (apply task args)))
>>>>>
>>>>>  and that all together is send-off into (agent []).
>>>>>
>>>>
>>>> What is "agent"? The first line of code indicates that it's a local
>>>> collection shadowing the code function, while the second code snippet
>>>> indicates that you're using the core agent function.
>>>>
>>>> Also why are you sending off to an agent?
>>>>
>>>
>>> I have ~8sec computing task for each input dataset which generates those
>>> records. After that I write it into disk (in software-specific
>>> transaction). I just wanted to separate hard computing and io operations. I
>>> created a side-effect method which is injected together with the dataset
>>> into a future. The futures are async collected within a list wrapped in
>>> agent. After the computing the main thread is waiting until all io tasks
>>> will be finished.
>>>
>>>
>>>>
>>>> At the end of the main thread function I just use await-for and after
>>>>> that:
>>>>>
>>>>> (reduce + (map #(deref %) @data-loading-tasks))
>>>>>
>>>>
>>> As a control, tasks return number of written records.
>>>
>>>
>>>
>>>>
>>>>> For some reason I see the happy collecting (see attached screenshot of
>>>>> jconsole).
>>>>>
>>>>
>>>> "happy" = "heap"?
>>>>
>>>
>>> Both. As you can see on attached screenshot the heap usage grows easy
>>> until aver. ~2 1/4 G than keep that  for a few minutes. In that moment I
>>> stopped. After that starts grow till ~4G with tendency to do jumps a bit
>>> more that 4G.
>>>
>>>
>>>>
>>>>
>>>>> After seeing the source code of future I suppose that the memory (data
>>>>> are kept as #{} set) is not released. The task returns only integer so I 
>>>>> do
>>>>> not think that might cause the problem.
>>>>>
>>>>
>>>> Can you provide more detail? You keep alluding to things that you don't
>>>> provide code for, such as the sets of data.
>>>>
>>>
>>>
>>> The code is attached. However the important code is
>>>
>>> L123 .
>>>   (let [;; keeps all data loading futures.
>>> ;; waiting until all futures are finished
>>> ;; should be done outside the main loop
>>> data-loading-tasks (agent [])]
>>>
>>> L128
>>> (doseq
>>>  (let [r1 (long operation)]   L133
>>>  (doseq
>>> (let [r2 (v.v. long)]   L155
>>>
>>>   L163   (send-off data-loading-task conj-task)
>>>
>>>  )
>>>  )
>>> )
>>> )
>>>
>>>
>>> I guess first I will move data-loading-tasks list into one of inner
>>> lets. Also I will create within an injecting function a separate abstract
>>> function let inside. The task will populate tmp variable which will be
>>> returned as a future result:
>>>
>>>
>>> L114 (conj agent (future (apply (fn []

Re: what does future do after fn finish ?

[Jacek Grzebyta]

Thanks a lot. I will check it tomorrow.

J

On 1 Feb 2018 12:12 a.m., "Justin Smith" <noisesm...@gmail.com> wrote:

> this is exactly the kind of problem code I was describing - there's no
> backpressure on existing future tasks to hold up the launching of more
> futures - the work done by the agent calling conj is negligible. You need
> to control the size of the pool of threads used, and you need to impose
> back-pressure.
>
> On Wed, Jan 31, 2018 at 3:46 PM Jacek Grzebyta <grzebyta@gmail.com>
> wrote:
>
>> On 31 January 2018 at 18:08, James Reeves <ja...@booleanknot.com> wrote:
>>
>>> On 31 January 2018 at 17:59, Jacek Grzebyta <grzebyta@gmail.com>
>>> wrote:
>>>
>>>> I have application with quite intense tripe store populating ~30/40 k
>>>> records per chunk (139 portions). The data are wrapped within the future:
>>>>
>>>> (conj agent (future (apply task args)))
>>>>
>>>>  and that all together is send-off into (agent []).
>>>>
>>>
>>> What is "agent"? The first line of code indicates that it's a local
>>> collection shadowing the code function, while the second code snippet
>>> indicates that you're using the core agent function.
>>>
>>> Also why are you sending off to an agent?
>>>
>>
>> I have ~8sec computing task for each input dataset which generates those
>> records. After that I write it into disk (in software-specific
>> transaction). I just wanted to separate hard computing and io operations. I
>> created a side-effect method which is injected together with the dataset
>> into a future. The futures are async collected within a list wrapped in
>> agent. After the computing the main thread is waiting until all io tasks
>> will be finished.
>>
>>
>>>
>>> At the end of the main thread function I just use await-for and after
>>>> that:
>>>>
>>>> (reduce + (map #(deref %) @data-loading-tasks))
>>>>
>>>
>> As a control, tasks return number of written records.
>>
>>
>>
>>>
>>>> For some reason I see the happy collecting (see attached screenshot of
>>>> jconsole).
>>>>
>>>
>>> "happy" = "heap"?
>>>
>>
>> Both. As you can see on attached screenshot the heap usage grows easy
>> until aver. ~2 1/4 G than keep that  for a few minutes. In that moment I
>> stopped. After that starts grow till ~4G with tendency to do jumps a bit
>> more that 4G.
>>
>>
>>>
>>>
>>>> After seeing the source code of future I suppose that the memory (data
>>>> are kept as #{} set) is not released. The task returns only integer so I do
>>>> not think that might cause the problem.
>>>>
>>>
>>> Can you provide more detail? You keep alluding to things that you don't
>>> provide code for, such as the sets of data.
>>>
>>
>>
>> The code is attached. However the important code is
>>
>> L123 .
>>   (let [;; keeps all data loading futures.
>> ;; waiting until all futures are finished
>> ;; should be done outside the main loop
>> data-loading-tasks (agent [])]
>>
>> L128
>> (doseq
>>  (let [r1 (long operation)]   L133
>>  (doseq
>> (let [r2 (v.v. long)]   L155
>>
>>   L163   (send-off data-loading-task conj-task)
>>
>>  )
>>  )
>> )
>> )
>>
>>
>> I guess first I will move data-loading-tasks list into one of inner lets.
>> Also I will create within an injecting function a separate abstract
>> function let inside. The task will populate tmp variable which will be
>> returned as a future result:
>>
>>
>> L114 (conj agent (future (apply (fn [] (let [result (apply task args)]
>> result)
>>
>>
>>>
>>> --
>>> James Reeves
>>> booleanknot.com
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to clojure@googlegroups.com
>>> Note that posts from new members are moderated - please be patient with
>>> your first post.
>>> To unsubscribe from this group, send email to
>>> clojure+unsubscr...@googlegroups.com
>>> For more options, visit this group at
>>> http://groups.google.com/group/

Re: what does future do after fn finish ?

[Justin Smith]

this is exactly the kind of problem code I was describing - there's no
backpressure on existing future tasks to hold up the launching of more
futures - the work done by the agent calling conj is negligible. You need
to control the size of the pool of threads used, and you need to impose
back-pressure.

On Wed, Jan 31, 2018 at 3:46 PM Jacek Grzebyta <grzebyta@gmail.com>
wrote:

> On 31 January 2018 at 18:08, James Reeves <ja...@booleanknot.com> wrote:
>
>> On 31 January 2018 at 17:59, Jacek Grzebyta <grzebyta@gmail.com>
>> wrote:
>>
>>> I have application with quite intense tripe store populating ~30/40 k
>>> records per chunk (139 portions). The data are wrapped within the future:
>>>
>>> (conj agent (future (apply task args)))
>>>
>>>  and that all together is send-off into (agent []).
>>>
>>
>> What is "agent"? The first line of code indicates that it's a local
>> collection shadowing the code function, while the second code snippet
>> indicates that you're using the core agent function.
>>
>> Also why are you sending off to an agent?
>>
>
> I have ~8sec computing task for each input dataset which generates those
> records. After that I write it into disk (in software-specific
> transaction). I just wanted to separate hard computing and io operations. I
> created a side-effect method which is injected together with the dataset
> into a future. The futures are async collected within a list wrapped in
> agent. After the computing the main thread is waiting until all io tasks
> will be finished.
>
>
>>
>> At the end of the main thread function I just use await-for and after
>>> that:
>>>
>>> (reduce + (map #(deref %) @data-loading-tasks))
>>>
>>
> As a control, tasks return number of written records.
>
>
>
>>
>>> For some reason I see the happy collecting (see attached screenshot of
>>> jconsole).
>>>
>>
>> "happy" = "heap"?
>>
>
> Both. As you can see on attached screenshot the heap usage grows easy
> until aver. ~2 1/4 G than keep that  for a few minutes. In that moment I
> stopped. After that starts grow till ~4G with tendency to do jumps a bit
> more that 4G.
>
>
>>
>>
>>> After seeing the source code of future I suppose that the memory (data
>>> are kept as #{} set) is not released. The task returns only integer so I do
>>> not think that might cause the problem.
>>>
>>
>> Can you provide more detail? You keep alluding to things that you don't
>> provide code for, such as the sets of data.
>>
>
>
> The code is attached. However the important code is
>
> L123 .
>   (let [;; keeps all data loading futures.
> ;; waiting until all futures are finished
> ;; should be done outside the main loop
> data-loading-tasks (agent [])]
>
> L128
> (doseq
>  (let [r1 (long operation)]   L133
>  (doseq
> (let [r2 (v.v. long)]   L155
>
>   L163   (send-off data-loading-task conj-task)
>
>  )
>  )
> )
> )
>
>
> I guess first I will move data-loading-tasks list into one of inner lets.
> Also I will create within an injecting function a separate abstract
> function let inside. The task will populate tmp variable which will be
> returned as a future result:
>
>
> L114 (conj agent (future (apply (fn [] (let [result (apply task args)]
> result)
>
>
>>
>> --
>> James Reeves
>> booleanknot.com
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with

Re: what does future do after fn finish ?

[Jacek Grzebyta]

On 31 January 2018 at 18:08, James Reeves <ja...@booleanknot.com> wrote:

> On 31 January 2018 at 17:59, Jacek Grzebyta <grzebyta@gmail.com>
> wrote:
>
>> I have application with quite intense tripe store populating ~30/40 k
>> records per chunk (139 portions). The data are wrapped within the future:
>>
>> (conj agent (future (apply task args)))
>>
>>  and that all together is send-off into (agent []).
>>
>
> What is "agent"? The first line of code indicates that it's a local
> collection shadowing the code function, while the second code snippet
> indicates that you're using the core agent function.
>
> Also why are you sending off to an agent?
>

I have ~8sec computing task for each input dataset which generates those
records. After that I write it into disk (in software-specific
transaction). I just wanted to separate hard computing and io operations. I
created a side-effect method which is injected together with the dataset
into a future. The futures are async collected within a list wrapped in
agent. After the computing the main thread is waiting until all io tasks
will be finished.


>
> At the end of the main thread function I just use await-for and after that:
>>
>> (reduce + (map #(deref %) @data-loading-tasks))
>>
>
As a control, tasks return number of written records.



>
>> For some reason I see the happy collecting (see attached screenshot of
>> jconsole).
>>
>
> "happy" = "heap"?
>

Both. As you can see on attached screenshot the heap usage grows easy until
aver. ~2 1/4 G than keep that  for a few minutes. In that moment I stopped.
After that starts grow till ~4G with tendency to do jumps a bit more that
4G.


>
>
>> After seeing the source code of future I suppose that the memory (data
>> are kept as #{} set) is not released. The task returns only integer so I do
>> not think that might cause the problem.
>>
>
> Can you provide more detail? You keep alluding to things that you don't
> provide code for, such as the sets of data.
>


The code is attached. However the important code is

L123 .
  (let [;; keeps all data loading futures.
;; waiting until all futures are finished
;; should be done outside the main loop
data-loading-tasks (agent [])]

L128
(doseq
 (let [r1 (long operation)]   L133
 (doseq
(let [r2 (v.v. long)]   L155

  L163   (send-off data-loading-task conj-task)

 )
 )
)
)


I guess first I will move data-loading-tasks list into one of inner lets.
Also I will create within an injecting function a separate abstract
function let inside. The task will populate tmp variable which will be
returned as a future result:


L114 (conj agent (future (apply (fn [] (let [result (apply task args)]
result)


>
> --
> James Reeves
> booleanknot.com
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


experiment_result.clj
Description: Binary data

Re: what does future do after fn finish ?

[Justin Smith]

As a shot in the dark, a common problem with memory usage and futures that
I have seen is the antipattern of launching a future for each piece of data
in a collection. The problem that occurs is that the code works for small
input collections and a small load of running tasks / requests, but for a
larger input and more requests it uses up the heap easily. Then, a dev
assumes that somehow the future itself is leaking or otherwise taking up
space it shouldn't, when the true problem is that the undbounded creation
of futures happens to exhaust your available memory space. If you aren't
doing such a thing feel free to disregard.

On Wed, Jan 31, 2018 at 10:09 AM James Reeves <ja...@booleanknot.com> wrote:

> On 31 January 2018 at 17:59, Jacek Grzebyta <grzebyta@gmail.com>
> wrote:
>
>> I have application with quite intense tripe store populating ~30/40 k
>> records per chunk (139 portions). The data are wrapped within the future:
>>
>> (conj agent (future (apply task args)))
>>
>>  and that all together is send-off into (agent []).
>>
>
> What is "agent"? The first line of code indicates that it's a local
> collection shadowing the code function, while the second code snippet
> indicates that you're using the core agent function.
>
> Also why are you sending off to an agent?
>
> At the end of the main thread function I just use await-for and after that:
>>
>> (reduce + (map #(deref %) @data-loading-tasks))
>>
>> For some reason I see the happy collecting (see attached screenshot of
>> jconsole).
>>
>
> "happy" = "heap"?
>
>
>> After seeing the source code of future I suppose that the memory (data
>> are kept as #{} set) is not released. The task returns only integer so I do
>> not think that might cause the problem.
>>
>
> Can you provide more detail? You keep alluding to things that you don't
> provide code for, such as the sets of data.
>
> --
> James Reeves
> booleanknot.com
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: what does future do after fn finish ?

[James Reeves]

On 31 January 2018 at 17:59, Jacek Grzebyta <grzebyta@gmail.com> wrote:

> I have application with quite intense tripe store populating ~30/40 k
> records per chunk (139 portions). The data are wrapped within the future:
>
> (conj agent (future (apply task args)))
>
>  and that all together is send-off into (agent []).
>

What is "agent"? The first line of code indicates that it's a local
collection shadowing the code function, while the second code snippet
indicates that you're using the core agent function.

Also why are you sending off to an agent?

At the end of the main thread function I just use await-for and after that:
>
> (reduce + (map #(deref %) @data-loading-tasks))
>
> For some reason I see the happy collecting (see attached screenshot of
> jconsole).
>

"happy" = "heap"?


> After seeing the source code of future I suppose that the memory (data are
> kept as #{} set) is not released. The task returns only integer so I do not
> think that might cause the problem.
>

Can you provide more detail? You keep alluding to things that you don't
provide code for, such as the sets of data.

-- 
James Reeves
booleanknot.com

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

what does future do after fn finish ?

[Jacek Grzebyta]

Hi,

I have application with quite intense tripe store populating ~30/40 k
records per chunk (139 portions). The data are wrapped within the future:

(conj agent (future (apply task args)))

 and that all together is send-off into (agent []).
At the end of the main thread function I just use await-for and after that:

(reduce + (map #(deref %) @data-loading-tasks))

For some reason I see the happy collecting (see attached screenshot of
jconsole). After seeing the source code of future I suppose that the memory
(data are kept as #{} set) is not released. The task returns only integer
so I do not think that might cause the problem.

Thanks a lot,
Jacek

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

What does the ref *loaded-libs* do?

[Raymond Huang]

I was poking around `tools.namespace` and I found it interesting that the 
implementation of `remove-lib` is:

```(defn remove-lib
  "Remove lib's namespace and remove lib from the set of loaded libs."
  [lib]
  (remove-ns lib)
  (dosync (alter @#'clojure.core/*loaded-libs* disj lib)))
```


I’m wondering if someone can enlighten me to explain why `*loaded-libs*` 
needs to exist, as opposed to checking `clojure.lang.Namespace` directly?

This is a question carried over from 
slack: https://clojurians.slack.com/archives/C03S1KBA2/p151721185721

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Port graphs: What would Rich Hickey do?

[Ben Kovitz]

On Monday, January 8, 2018 at 12:14:59 AM UTC-5, James Gatannah wrote:
 

> FWIW, I think https://leanpub.com/specter looks extremely interesting. 
> (Or it may be awful...I haven't had a chance to read it yet, much less work 
> through the exercises).
>

Actually, I worked through that ebook last week. (Or at least the version 
of the ebook in Google's cache. Leanpub's web site appears to be down.) 
It's excellent! It didn't go all that far, but it got across the main ideas 
very effectively.

I've been continuing to look at Specter, and the more I've learned, the 
better it's seemed. I have yet to try it myself on anything beyond toy 
examples, though. Currently I'm stumped on how to make a navigator for a 
data structure that doesn't work with 'get' and 'assoc'. There's probably 
still something elementary that I haven't understood yet. I just posted a 
question on the github site:
https://github.com/nathanmarz/specter/issues/241

--
Ben Kovitz
http://pages.iu.edu/~bkovitz/

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Port graphs: What would Rich Hickey do?

[James Gatannah]

On Thursday, January 4, 2018 at 5:51:27 PM UTC-6, Ben Kovitz wrote:
>
>
> Well! So far, Specter appears to have taken the "path map" idea that I'd 
> been toying with to its ultimate logical conclusion—far better than I could 
> ever do it. Nathan Marz even says that needing to manipulate tricky graphs 
> "forced" him to come up with Specter. If I understand it correctly, Specter 
> would even allow me to map these peculiar port graphs to simplicial graphs 
> without adding any performance overhead.
>
> It looks like I'd need to write a custom navigator or two. I haven't yet 
> found a good tutorial for getting beyond the most elementary concepts of 
> Specter, though. Can you recommend one?
>

FWIW, I think https://leanpub.com/specter looks extremely interesting. (Or 
it may be awful...I haven't had a chance to read it yet, much less work 
through the exercises).

Regards,
James


> --
> Ben Kovitz
> http://pages.iu.edu/~bkovitz/
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Port graphs: What would Rich Hickey do?

[Ben Kovitz]

I replied to Gary Verhaegen:

If you’re shopping around for ideas as to how to define data manipulation 
>> API, it may be worth taking some time looking at Specter too.
>>
>
> Thanks for this suggestion, too. I'm now checking it out.
>

Well! So far, Specter appears to have taken the "path map" idea that I'd 
been toying with to its ultimate logical conclusion—far better than I could 
ever do it. Nathan Marz even says that needing to manipulate tricky graphs 
"forced" him to come up with Specter. If I understand it correctly, Specter 
would even allow me to map these peculiar port graphs to simplicial graphs 
without adding any performance overhead.

It looks like I'd need to write a custom navigator or two. I haven't yet 
found a good tutorial for getting beyond the most elementary concepts of 
Specter, though. Can you recommend one?

--
Ben Kovitz
http://pages.iu.edu/~bkovitz/

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Port graphs: What would Rich Hickey do?

[Christopher Small]

Yes; Datomic really is great. There's plenty of wonderful stuff I haven't
even mentioned, like database as a value, persistent history, snapshots at
a timepoint, transaction metadata, smart peer replication etc. It's kind of
the best thing since sliced bread. But if you don't need all that jazz and
are fine with keeping things in memory for your problem, you may wish to
consider DataScript. It's free and open source and has remarkably good
coverage of the entity, pull and q APIs, and can run in cljs or clj.

One final note: with respect to "type headache" and "chain[ing] anything to
anything", you're right; this is a major benefit over traditional SQL.
Attribute are fundamentally polymorphic, which can be super helpful with
tricky modelling problems.

Chris


On Thu, Jan 4, 2018 at 10:48 AM, Ben Kovitz  wrote:

> On Tuesday, January 2, 2018 at 3:11:45 PM UTC-5, Christopher Small wrote:
>
>
>> http://docs.datomic.com/entities.html
>>
>
>
>> http://docs.datomic.com/pull.html
>>
>
> Thanks—this is really good stuff! Not that I expected anything less, but
> it's a happy surprise that it applies so well to graphs. Now I'm wondering
> if I should just use Datomic rather than borrow ideas from it.
>
> One thing I'm convinced of now is that it's OK to assign ids to
> everything. I was cringing at that idea before, because it seems like an
> unnecessary layer of indirection and one more thing that could go wrong.
> But clearly Datomic's approach of assigning every "entity" an id enables
> some wonderful simplicity, since it enables one to very conveniently chain
> entities together. And when basically everything is an entity, that means
> you can chain anything to anything—which addresses the "type headache" I
> was trying to avoid.
>
> --
> Ben Kovitz
> http://pages.iu.edu/~bkovitz/
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/clojure/uSwY475pbGA/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Port graphs: What would Rich Hickey do?

[Gary Verhaegen]

Unfortunately most of the contexts were fairly specific and a few
years ago, so I don't remember all of the details, and mostly in the
context of private research that I'm not at liberty to share. Some of
the results have been published (though not my specific work) by a
then-colleague of mine; you can find his papers by googling "amine
ghrab graph olap".

On 4 January 2018 at 18:19, Ben Kovitz <bkov...@gmail.com> wrote:
> On Thursday, January 4, 2018 at 4:01:41 AM UTC-5, Gary Verhaegen wrote:
>
>> Have you considered encoding your port graph into a normal graph?
>
>
> That was the first idea I had, and I quickly rejected it as adding
> complication. The consideration that led me to port graphs in the first
> place was that when, say, I have a node for "plus", and it has neighbor
> nodes for "operand" and "sum", which other nodes connect to, an easy bug is
> that when the "plus" is deleted, its "operand" and "sum" nodes might not get
> deleted—leaving the graph in an invalid state. I figured it would be best to
> just absorb those connection nodes into the "plus" node in the form of a
> list of "ports", and include the port labels in the edges. Then deleting the
> "plus" automatically deletes the ports, you can easily query for "what nodes
> are connected to the plus" (where you want to skip connection nodes), and
> generally there are fewer ways to make mistakes in the code.
>
> However, following Christopher Small's suggestion to check out the Datomic
> APIs, I'm now thinking that a layer to map the port graphs to simplicial
> ("normal", undirected) graphs might actually be the simplest thing. Nodes,
> ports, and edges are easily represented by nodes in a simplicial graph. This
> escapes the "type headache" that led me to start this thread, since all
> edges in the port graph are just plain nodes in the simplicial graph,
> whether they're port-to-port edges, port-to-edge edges, or edge-to-edge
> edges. The layer of indirection might cost some performance—and performance
> does matter here, since this program is extremely CPU-bound—but it's more
> important to preëmpt bugs by keeping code simple.
>
> And now you make this interesting point, which I hadn't previously
> considered:
>
>> That would require you to define a mapping, but would then allow you to
>> fairly easily reuse theorems, algorithms and libraries by having them
>> operate on the encoded graph, with a hopefully thin layer of translation.
>
>
> I could even use ubergraph to represent the simplicial graph, and thereby
> get access to all code that follows the loom protocols.
>
>> I have seen this approach of encoding graphs into other graphs work out
>> pretty well in other contexts.
>
>
> Do you remember any of those other contexts? Maybe I could check one out and
> learn something from it.
>
>> If you’re shopping around for ideas as to how to define data manipulation
>> API, it may be worth taking some time looking at Specter too.
>
>
> Thanks for this suggestion, too. I'm now checking it out.
>
> --
> Ben Kovitz
> http://pages.iu.edu/~bkovitz/
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with your
> first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Port graphs: What would Rich Hickey do?

[Ben Kovitz]

On Thursday, January 4, 2018 at 4:01:41 AM UTC-5, Gary Verhaegen wrote:

Have you considered encoding your port graph into a normal graph?


That was the first idea I had, and I quickly rejected it as adding 
complication. The consideration that led me to port graphs in the first 
place was that when, say, I have a node for "plus", and it has neighbor 
nodes for "operand" and "sum", which other nodes connect to, an easy bug is 
that when the "plus" is deleted, its "operand" and "sum" nodes might not 
get deleted—leaving the graph in an invalid state. I figured it would be 
best to just absorb those connection nodes into the "plus" node in the form 
of a list of "ports", and include the port labels in the edges. Then 
deleting the "plus" automatically deletes the ports, you can easily query 
for "what nodes are connected to the plus" (where you want to skip 
connection nodes), and generally there are fewer ways to make mistakes in 
the code.

However, following Christopher Small's suggestion to check out the Datomic 
APIs, I'm now thinking that a layer to map the port graphs to simplicial 
("normal", undirected) graphs might actually be the simplest thing. Nodes, 
ports, and edges are easily represented by nodes in a simplicial graph. 
This escapes the "type headache" that led me to start this thread, since 
all edges in the port graph are just plain nodes in the simplicial graph, 
whether they're port-to-port edges, port-to-edge edges, or edge-to-edge 
edges. The layer of indirection might cost some performance—and performance 
does matter here, since this program is extremely CPU-bound—but it's more 
important to preëmpt bugs by keeping code simple.

And now you make this interesting point, which I hadn't previously 
considered:

That would require you to define a mapping, but would then allow you to 
> fairly easily reuse theorems, algorithms and libraries by having them 
> operate on the encoded graph, with a hopefully thin layer of translation.


I could even use ubergraph to represent the simplicial graph, and thereby 
get access to all code that follows the loom protocols. 

I have seen this approach of encoding graphs into other graphs work out 
> pretty well in other contexts.


Do you remember any of those other contexts? Maybe I could check one out 
and learn something from it.

If you’re shopping around for ideas as to how to define data manipulation 
> API, it may be worth taking some time looking at Specter too.
>

Thanks for this suggestion, too. I'm now checking it out.
 
--
Ben Kovitz
http://pages.iu.edu/~bkovitz/

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Port graphs: What would Rich Hickey do?

[Ben Kovitz]

On Tuesday, January 2, 2018 at 3:11:45 PM UTC-5, Christopher Small wrote:
 

> http://docs.datomic.com/entities.html
>
 

> http://docs.datomic.com/pull.html
>

Thanks—this is really good stuff! Not that I expected anything less, but 
it's a happy surprise that it applies so well to graphs. Now I'm wondering 
if I should just use Datomic rather than borrow ideas from it.

One thing I'm convinced of now is that it's OK to assign ids to everything. 
I was cringing at that idea before, because it seems like an unnecessary 
layer of indirection and one more thing that could go wrong. But clearly 
Datomic's approach of assigning every "entity" an id enables some wonderful 
simplicity, since it enables one to very conveniently chain entities 
together. And when basically everything is an entity, that means you can 
chain anything to anything—which addresses the "type headache" I was trying 
to avoid.

--
Ben Kovitz
http://pages.iu.edu/~bkovitz/

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Port graphs: What would Rich Hickey do?

[Gary Verhaegen]

If you’re shopping around for ideas as to how to define data manipulation API, 
it may be worth taking some time looking at Specter too.

Have you considered encoding your port graph into a normal graph? That would 
require you to define a mapping, but would then allow you to fairly easily 
reuse theorems, algorithms and libraries by having them operate on the encoded 
graph, with a hopefully thin layer of translation.

One possible encoding based on what you’ve described could be to turn nodes and 
edges in the port graph into nodes in the encoded graph, and ports in the port 
graph into edges in the encoded graph. It’s hard to say how well that specific 
transform would work in your case without more details about your type of graph 
and somewhat more considerate thinking than I’ve given this, but I have seen 
this approach of encoding graphs into other graphs work out pretty well in 
other contexts.

> On 2 Jan 2018, at 20:11, Christopher Small <metasoar...@gmail.com> wrote:
> 
> > ...noticed that the attribute names in the datoms are a lot like the port 
> > labels in port graphs. A port label is essentially an attribute key whose 
> > value is the edges incident to it. And when you chain data patterns in 
> > Datalog queries, that's a lot like navigating through a graph.
> 
> That's very much in line with what I was thinking you might want to do in 
> this setting.
> 
> You should also take a look at the entity api. It let's you traverse the 
> graph via these entity objects which behave more or less like a dictionary of 
> the av pairs associated with a particular entity in the database. Importantly 
> though, when an attribute points to another entity (a reference attribute), 
> it is returned as yet another of these entity objects, letting you traverse 
> the graph using get, in a pretty similar fashion to what you're describing.
> 
> http://docs.datomic.com/entities.html
> 
> You may also check out the pull api. It let's you pull facts out of the graph 
> as regular old maps by specifying what attributes & paths you wish to pull 
> out of the graph. The result is that you can read from the underlying graph 
> database as though it was a document store, but where the documents are 
> dynamically computed/queried from the graph based on the pull expression you 
> provide. You can also use pull expressions inside the `:find` clause of 
> either a DataScript of Datomic datalog query, which is often a convenient way 
> of specifying both a set of entities (via a datalog query) and statement of 
> facts to pull about them.
> 
> http://docs.datomic.com/pull.html
> 
> Collectively, these three interfaces give you quite a lot of expressiveness 
> and flexibility in how you query/traverse your data.
> 
> 
> 
>> On Tue, Jan 2, 2018 at 12:31 PM, Ben Kovitz <bkov...@gmail.com> wrote:
>> Christopher Small,
>> 
>> Your suggestion to look into Datomic definitely turned up a new idea. I went 
>> through the Datalog tutorial at http://www.learndatalogtoday.org (which is 
>> excellent, by the way), and noticed that the attribute names in the datoms 
>> are a lot like the port labels in port graphs. A port label is essentially 
>> an attribute key whose value is the edges incident to it. And when you chain 
>> data patterns in Datalog queries, that's a lot like navigating through a 
>> graph.
>> 
>> The new, hopefully simple, easy-to-use, and not error-prone idea is to use 
>> the clojure.lang.IPersistentMap  interface to query and update graphs. 
>> Nodes, edge, attribute names, port labels, and a few "reserved keywords" 
>> like :adj-nodes and :edge, could navigate the graph as if it were an 
>> ordinary nested map with get-in, assoc-in, and the like, without ever 
>> needing to wrap anything inside a record to tell what it is. Here are some 
>> sample queries and what they could return:
>> 
>> (get-in g [node port-label :adj-nodes])
>>   a coll of all the nodes with an edge to [node port-label]
>> (get-in g [node port-label :adj-node])
>>   one node with an edge to [node port-label], or nil
>> (get-in g [node port-label])
>>   true or false, according to whether node has a port named port-label
>> (get-in g [node k])
>>   value of node's attr k
>> (get-in g [edge k])
>>   value of edge's attr k
>> (get g node-or-edge)
>>   the entire map of attrs associated with node-or-edge, or nil if it does 
>> not exist
>> (get-in g [node k1 k2 k3])
>>   treat node's attrs as a nested map
>> (get-in [node port-label :edges])
>>   a coll of all the edges that link to [node port-label]
>> (get-in [node port-label :edge])
>>   a single edge that links t

Re: Port graphs: What would Rich Hickey do?

[Christopher Small]

> ...noticed that the attribute names in the datoms are a lot like the port
labels in port graphs. A port label is essentially an attribute key whose
value is the edges incident to it. And when you chain data patterns in
Datalog queries, that's a lot like navigating through a graph.

That's very much in line with what I was thinking you might want to do in
this setting.

You should also take a look at the entity api. It let's you traverse the
graph via these entity objects which behave more or less like a dictionary
of the av pairs associated with a particular entity in the database.
Importantly though, when an attribute points to another entity (a reference
attribute), it is returned as yet another of these entity objects, letting
you traverse the graph using get, in a pretty similar fashion to what
you're describing.

http://docs.datomic.com/entities.html

You may also check out the pull api. It let's you pull facts out of the
graph as regular old maps by specifying what attributes & paths you wish to
pull out of the graph. The result is that you can read from the underlying
graph database as though it was a document store, but where the documents
are dynamically computed/queried from the graph based on the pull
expression you provide. You can also use pull expressions inside the
`:find` clause of either a DataScript of Datomic datalog query, which is
often a convenient way of specifying both a set of entities (via a datalog
query) and statement of facts to pull about them.

http://docs.datomic.com/pull.html

Collectively, these three interfaces give you quite a lot of expressiveness
and flexibility in how you query/traverse your data.



On Tue, Jan 2, 2018 at 12:31 PM, Ben Kovitz <bkov...@gmail.com> wrote:

> Christopher Small,
>
> Your suggestion to look into Datomic definitely turned up a new idea. I
> went through the Datalog tutorial at http://www.learndatalogtoday.org
> (which is excellent, by the way), and noticed that the attribute names in
> the datoms are a lot like the port labels in port graphs. A port label is
> essentially an attribute key whose value is the edges incident to it. And
> when you chain data patterns in Datalog queries, that's a lot like
> navigating through a graph.
>
> The new, hopefully simple, easy-to-use, and not error-prone idea is to use
> the clojure.lang.IPersistentMap  interface to query and update graphs.
> Nodes, edge, attribute names, port labels, and a few "reserved keywords"
> like :adj-nodes and :edge, could navigate the graph as if it were an
> ordinary nested map with get-in, assoc-in, and the like, without ever
> needing to wrap anything inside a record to tell what it is. Here are some
> sample queries and what they could return:
>
> (get-in g [node port-label :adj-nodes])
>   a coll of all the nodes with an edge to [node port-label]
> (get-in g [node port-label :adj-node])
>   one node with an edge to [node port-label], or nil
> (get-in g [node port-label])
>   true or false, according to whether node has a port named port-label
> (get-in g [node k])
>   value of node's attr k
> (get-in g [edge k])
>   value of edge's attr k
> (get g node-or-edge)
>   the entire map of attrs associated with node-or-edge, or nil if it does
> not exist
> (get-in g [node k1 k2 k3])
>   treat node's attrs as a nested map
> (get-in [node port-label :edges])
>   a coll of all the edges that link to [node port-label]
> (get-in [node port-label :edge])
>   a single edge that links to port-label, or nil
> (get-in [node port-label1 port-label2])
>   coll of all nodes that link to [node port-label1] from a port named
> port-label2
>
> And here are some function calls that would return a modified graph:
>
> (assoc-in g [node k] v)
>   sets node attr k to v
> (assoc-in g [edge k] v)
>   sets edge attr k to v  (and similarly for multiple keys)
> (assoc-in g [node1 port-label1 port-label2 node2] :edge)
>   makes an edge between [node1 port-label1] and [node2 port-label2]
> (assoc-in g [node1 port-label1 port-label2 :edge k] v)
>   sets value of edge attr k to v
>
> I haven't yet thought this through completely, and I suspect that some
> parts are still error-prone. For example, notice in the last function call
> that it's not clear whether you should include node2. The approach might be
> "too cute"—that is, brittle because it tries to impose one kind of
> structure (nested maps) onto one that isn't fully isomorphic (graphs).
>
> But I like the basic idea of having a "little language" for designating
> whatever you want, and the little language is nothing but a sequence that
> tells how to navigate through the graph to find (or make) what you're
> interested in.
>
> This would solve the fundamental problem of how to designate an edge
> without imposin

Re: Port graphs: What would Rich Hickey do?

[Ben Kovitz]

Christopher Small,

Your suggestion to look into Datomic definitely turned up a new idea. I 
went through the Datalog tutorial at http://www.learndatalogtoday.org 
(which is excellent, by the way), and noticed that the attribute names in 
the datoms are a lot like the port labels in port graphs. A port label is 
essentially an attribute key whose value is the edges incident to it. And 
when you chain data patterns in Datalog queries, that's a lot like 
navigating through a graph.

The new, hopefully simple, easy-to-use, and not error-prone idea is to use 
the clojure.lang.IPersistentMap  interface to query and update graphs. 
Nodes, edge, attribute names, port labels, and a few "reserved keywords" 
like :adj-nodes and :edge, could navigate the graph as if it were an 
ordinary nested map with get-in, assoc-in, and the like, without ever 
needing to wrap anything inside a record to tell what it is. Here are some 
sample queries and what they could return:

(get-in g [node port-label :adj-nodes])
  a coll of all the nodes with an edge to [node port-label]
(get-in g [node port-label :adj-node])
  one node with an edge to [node port-label], or nil
(get-in g [node port-label])
  true or false, according to whether node has a port named port-label
(get-in g [node k])
  value of node's attr k
(get-in g [edge k])
  value of edge's attr k
(get g node-or-edge)
  the entire map of attrs associated with node-or-edge, or nil if it does 
not exist
(get-in g [node k1 k2 k3])
  treat node's attrs as a nested map
(get-in [node port-label :edges])
  a coll of all the edges that link to [node port-label]
(get-in [node port-label :edge])
  a single edge that links to port-label, or nil
(get-in [node port-label1 port-label2])
  coll of all nodes that link to [node port-label1] from a port named 
port-label2

And here are some function calls that would return a modified graph:

(assoc-in g [node k] v)
  sets node attr k to v
(assoc-in g [edge k] v)
  sets edge attr k to v  (and similarly for multiple keys)
(assoc-in g [node1 port-label1 port-label2 node2] :edge)
  makes an edge between [node1 port-label1] and [node2 port-label2]
(assoc-in g [node1 port-label1 port-label2 :edge k] v)
  sets value of edge attr k to v

I haven't yet thought this through completely, and I suspect that some 
parts are still error-prone. For example, notice in the last function call 
that it's not clear whether you should include node2. The approach might be 
"too cute"—that is, brittle because it tries to impose one kind of 
structure (nested maps) onto one that isn't fully isomorphic (graphs).

But I like the basic idea of having a "little language" for designating 
whatever you want, and the little language is nothing but a sequence that 
tells how to navigate through the graph to find (or make) what you're 
interested in. 

This would solve the fundamental problem of how to designate an edge 
without imposing type restrictions. For each connection point, you just 
have a sequence that describes a path to that point. If the sequence ends 
on a port label, the connection point is a port. If it ends on :edge, the 
connection point is an edge. So, instead of type restrictions, there are 
just a few reserved words in the tiny DSL for navigating the graph.

This hopefully makes simple editing and querying of port graphs easy, but 
it does not even try to solve the problem of conveniently entering a graph. 
I figure that a "shorthand", i.e. a separate tiny DSL, is the solution for 
that. I found with my current implementation of port graphs that it really 
helped to use a custom shorthand for each specific type of graph. For 
example, there's a obvious and very nice shorthand for specifying graphs 
that represent mathematical expressions, but it's terrible for graphs of 
social networks. Graphs of social networks, though, can be conveniently 
described by their own custom sexprs.

I figure that implementing it will force me to think this idea through the 
rest of the way. Here goes…

--
Ben Kovitz
http://pages.iu.edu/~bkovitz/


-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Port graphs: What would Rich Hickey do?

[Ben Kovitz]

On Monday, January 1, 2018 at 7:57:11 PM UTC-5, Christopher Small wrote:

…you might want to look at Datomic (also authored by Rich Hickey), 
> DataScript (an in-memory approximation) or RDF (data language of the 
> Semantic Web). They model graphs as `(entity, attribute, value)` triples, 
> where `attribute`s can act as labeled, directional arrows between entities, 
> and can themselves show up in the `entity` position of other triples, thus 
> allowing you to have "meta-attributes" (attributes pointing to/from other 
> attributes).


Datomic might be just the dose of simplicity I was looking for! I haven't 
yet used Datomic, so now is probably a very good time to try it out. I had 
thought that Datomic might make a good underlying implementation rather 
than the API, but now I wonder if the Datomic API has ideas I could use. 
The entity-attribute-value approach sounds to me like one of the best ideas 
ever (long predating computers, even). Thanks! I'll have a closer look. 
This might even be what Rich Hickey would do! :)

Do you need to be able to attach data to specific edges instances?
>

Yes. Every edge needs to be able to have an arbitrary map of attributes.

Or would it be sufficient to be able to have different types of edges, and 
> be able to associate data with those abstract edge types?
>

I'd like to be able to treat all edges the same.
 

> This is for a research project, so I need to be able to try out new ideas 
quickly to see how they perform. I've been using an implementation of port 
graphs modeled on ubergraph with some success, where all nodes and edges 
can have arbitrary attribute maps associated with them. Where I've needed 
edges that link to edges, I've used a couple hacks, like making a separate 
graph where some of the nodes are edges from the main graph. The nuisance 
of using those hacks has been steering me away from trying out promising 
ideas that freely make use of edge-to-edge edges. So, I figure that it's 
time to just implement this right and be done with it.

The fact that it's a research project makes minimal error-proneness 
especially important. It's often hard to know in advance what the correct 
behavior looks like. The reason for writing the program is to find out.
 

> Obviously, this is a little bit different from your "ports" idea (I 
> haven't read about this; is it published on anywhere?), but in some ways a 
> helpful restriction.
>

If you google for "port graphs", you'll find some stuff. It's not clear to 
me why, but port graphs seem to be popular for "graph-rewriting systems" 
(also googlable). I'm working on something like a graph rewriting system. I 
hit on the idea as a way to make the graph simpler, and only later found 
out that there's already a standard name for these graphs and even some 
good research literature. There's *some* sort of natural fit here.
 

> For example, `mom` vs `child` is isomorphic to an edge labeled 
> `isMotherOf`. But simpler, because there's less to specify, and in 
> particular, less to get wrong with respect to which kinds of ports can 
> connect to which kinds of other ports.
>

Something I've liked about the port graphs up until now is that it's easy 
to add an explicit representation of rules for which kinds of ports can 
connect and which can't. The current implementation (the one without 
edge-to-edge edges) has some "shorthand" functions that exploit those rules 
so that you can specify some edges very simply, often without explicitly 
providing the port labels, since the system can (in some cases) deduce 
them. Most of my previous experiences with programming involving graphs 
have involved a lot of hard-to-read code and a lot of sweating and gritting 
my teeth while trying to avoid bugs. The ports-rules-and-shorthand approach 
has enabled to me keep the code mostly readable. In fact, much of the code 
is just static data that specifies small subgraphs—in a pretty readable way.

The rules are also nicely separable from the generic aspects of the graph. 
I added a couple protocols beyond the loom protocols: one for creating new 
nodes of a given 'class', so they can get ids and attributes assigned to 
them automatically; and one for consulting the rules for legal connections.

But it's clearly time for me to look at this in a new way. I like your 
observation that (in effect) edge classes of the form (port-label1, 
port-label2) are isomorphic to edges between ports. That and Datomic might 
be exactly the whack(s) on the side of the head that I needed!

--
Ben Kovitz
http://pages.iu.edu/~bkovitz/


-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+uns

Re: Port graphs: What would Rich Hickey do?

[Christopher Small]

Do you need to be able to attach data to specific edges instances? Or would 
it be sufficient to be able to have different types of edges, and be able 
to associate data with those abstract edge types?

If the latter, you might want to look at Datomic (also authored by Rich 
Hickey), DataScript (an in-memory approximation) or RDF (data language of 
the Semantic Web). They model graphs as `(entity, attribute, value)` 
triples, where `attribute`s can act as labeled, directional arrows between 
entities, and can themselves show up in the `entity` position of other 
triples, thus allowing you to have "meta-attributes" (attributes pointing 
to/from other attributes).

Obviously, this is a little bit different from your "ports" idea (I haven't 
read about this; is it published on anywhere?), but in some ways a helpful 
restriction. For example, `mom` vs `child` is isomorphic to an edge labeled 
`isMotherOf`. But simpler, because there's less to specify, and in 
particular, less to get wrong with respect to which kinds of ports can 
connect to which kinds of other ports. So IMHO, simple labeled directional 
edges are best when semantically sufficient.

If either data needs to be attached to edges, or there are semantics of the 
modeling of things in terms of ports which are advantageous, you can still 
use EAV to model this data. You would just have entities modeling the edges 
and/or ports, and connect them using triples along the lines of `[[edge 
:my.graph.edge/from node-or-port-1] [edge :my.graph.edge/to node-or-port-2] 
[edge :my.graph.edge/label "a really cool edge"]]`. With Datomic or 
DataScript, you'd then be able to put together a set of Datalog rules which 
represent whatever higher level graph relations that might be useful. 
(Datalog is awesome, BTW)

Interested to hear what you think of this idea :-)

Chris



On Monday, January 1, 2018 at 3:07:28 PM UTC-7, Ben Kovitz wrote:
>
> I'm making a little library for myself to represent a kind of graph. I 
> want to
> make it simple, the way Rich Hickey designed much of Clojure. My designs so
> far have lacked that kind of plainness and ease of use. I keep wondering,
> "What would Rich Hickey do?" Even if you're not Rich Hickey, maybe you can
> suggest something. I'd love to hear some ideas about how to design this.
>
> This kind of graph is different from most graphs in two ways:
>
> (1) Edges connect to "ports" rather than simply to nodes. A port is a tuple
> (node, port-label). So, for example, if you had nodes :elizabeth and 
> :charles,
> you might have an edge connecting :elizabeth's "child" port to :charles's
> "mother" port, rather than simply an edge connecting :elizabeth and 
> :charles.
>
> (2) Edges can connect to edges as well as to ports. For example, the
> aforementioned edge might itself be connected by an edge to the "attests" 
> port
> of a node :royal-document-22416.
>
> These differences from ordinary graphs preclude the use of the loom 
> protocols
> or implementations such as ubergraph. I would like also to allow any 
> arbitrary
> map to be associated with any node or edge, just as ubergraph allows. By 
> the
> way, if you know of a standard name for this kind of graph, please tell me.
> It's not quite the same as a hypergraph. Note also that directed graphs 
> are a
> special case of this kind of graph, since the port labels can imply 
> asymmetry
> if you like. For example, if you only use labels :in and :out for ports, 
> then
> you have an ordinary directed graph.
>
> The fact that edges can connect to edges seems to be the main source of my
> difficulties. I would like to allow nodes and port labels to be of any data
> type: keywords, strings, vectors, anything, even graphs and edges from 
> graphs.
> The library should make no assumptions about the data types of the nodes 
> and
> port labels. So, when passing arguments to functions in the library, you 
> can't
> have a convention like "a 2-element vector represents a port" or "a 
> 2-element
> vector of 2-element vectors represents an edge".
>
> 
>
> Here are the main design ideas that I've entertained so far:
>
> 1. I could wrap nodes, ports, and edges in records, like this:
>
>   (defrecord Node [nodeid])
>   (defrecord Port [nodeid port-label])
>   (defrecord Edge [connpt1 connpt2])
>
> where connpt1 and connpt2 are "connection points": either a port or an 
> edge.
> This completely disambiguates everything. Library functions never look 
> inside
> a nodeid or port-label, but they can look inside a connection point as 
> needed.
>
> To add a node or edge to a graph g, you could call:
>

Re: Port graphs: What would Rich Hickey do?

[Raoul Duke]

€0.02 i like option #3, i think it would be possibly nice for edges to be
named based on the ports they connect.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Port graphs: What would Rich Hickey do?

[Ben Kovitz]

I'm making a little library for myself to represent a kind of graph. I want 
to
make it simple, the way Rich Hickey designed much of Clojure. My designs so
far have lacked that kind of plainness and ease of use. I keep wondering,
"What would Rich Hickey do?" Even if you're not Rich Hickey, maybe you can
suggest something. I'd love to hear some ideas about how to design this.

This kind of graph is different from most graphs in two ways:

(1) Edges connect to "ports" rather than simply to nodes. A port is a tuple
(node, port-label). So, for example, if you had nodes :elizabeth and 
:charles,
you might have an edge connecting :elizabeth's "child" port to :charles's
"mother" port, rather than simply an edge connecting :elizabeth and 
:charles.

(2) Edges can connect to edges as well as to ports. For example, the
aforementioned edge might itself be connected by an edge to the "attests" 
port
of a node :royal-document-22416.

These differences from ordinary graphs preclude the use of the loom 
protocols
or implementations such as ubergraph. I would like also to allow any 
arbitrary
map to be associated with any node or edge, just as ubergraph allows. By the
way, if you know of a standard name for this kind of graph, please tell me.
It's not quite the same as a hypergraph. Note also that directed graphs are 
a
special case of this kind of graph, since the port labels can imply 
asymmetry
if you like. For example, if you only use labels :in and :out for ports, 
then
you have an ordinary directed graph.

The fact that edges can connect to edges seems to be the main source of my
difficulties. I would like to allow nodes and port labels to be of any data
type: keywords, strings, vectors, anything, even graphs and edges from 
graphs.
The library should make no assumptions about the data types of the nodes and
port labels. So, when passing arguments to functions in the library, you 
can't
have a convention like "a 2-element vector represents a port" or "a 
2-element
vector of 2-element vectors represents an edge".



Here are the main design ideas that I've entertained so far:

1. I could wrap nodes, ports, and edges in records, like this:

  (defrecord Node [nodeid])
  (defrecord Port [nodeid port-label])
  (defrecord Edge [connpt1 connpt2])

where connpt1 and connpt2 are "connection points": either a port or an edge.
This completely disambiguates everything. Library functions never look 
inside
a nodeid or port-label, but they can look inside a connection point as 
needed.

To add a node or edge to a graph g, you could call:

  (defn add [g elem] ...)

where elem must be a Node or Edge record.

A problem with this is that it's a pain to use. Every time you call a 
function
like 'add', you have to wrap all the nodes. Wrapping edges is slightly more 
of
a nuisance.

Another problem is that some decisions seem unclear and arbitrary, hence
error-prone for a caller. For example, what should the 'nodes' function
return: nodes wrapped in Node records, or unwrapped nodes? Each choice seems
reasonable—hence you can easily assume the wrong choice when writing code 
that
calls the library.

2. I could put two kinds of function in the library: some with wrapped
arguments and return values, and equivalents with unwrapped values. So, for
example, there could be:

  (defn add [g elem] ...)  ;same as above
  (defn add-nodes [g & nodeids] ...)
  (defn add-edges [g & edges] ...)  ;each edge is, uh, I'm not sure
  (defn has? [g elem] ...)  ;elem must be wrapped
  (defn has-node? [g nodeid] ...)  ;unwrapped
  (defn has-edge? [g connpt1 connpt2) ...)  ;uh...
  (defn nodes [g] ...)  ;returns Node records
  (defn nodes-unwrapped [g] ...)  ;returns nodeids

It's still not fully clear how to pass edges to 'add-edges' and 'has-edge?'.
You can't assume a convention like [nodeid1 port-label1 nodeid2 port-label2]
because edges are allowed as connection points. Maybe edges would still 
always
need to be wrapped.

3. Inspired by this hypergraph library by bdesham:
https://github.com/bdesham/hitting-set/blob/master/README.md
I could give ids to edges as well as to nodes. However, most edges that I
anticipate working with will not have a meaningful id other than "the edge
between these connection points."

4. I could have a few "shorthand" functions for describing graphs with a
little DSL, in addition to the core API which would require all graph
elements to be wrapped. The shorthand DSL could be written to make the most
common kinds of graphs and subgraphs very convenient to describe, maybe even
sacrificing full generality.



OK, that's enough. I'm probably missing something obvious here—obvious, at
least, to Rich Hickey or someone with more Clojure experience. Would you 
care
to smack me on

Re: Datomic evals clojure subset in datalog - how can I do this?

[Karsten Schmidt]

I can only point you to an example for your 2nd question, using a
similar usage context (works in both clj/cljs):

https://github.com/thi-ng/fabric/blob/master/fabric-facts/src/dsl.org#examples

Implementation details:

https://github.com/thi-ng/fabric/blob/master/fabric-facts/src/dsl.org#expressions

Hth!

On 1 September 2017 at 14:50, Dustin Getz <dustin.g...@gmail.com> wrote:
> Datomic allows clojure.core fns to be embedded inside datalog queries,
> except eval (http://docs.datomic.com/query.html)
>
> 1. Do we know how secure is this - what if the datalog comes from untrusted
> user input?
>
> 2. What is the best way to implement restricted-eval like this on jvm,
> compared to on js?
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with your
> first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



-- 
Karsten Schmidt
http://postspectacular.com | http://thi.ng | http://toxiclibs.org

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Datomic evals clojure subset in datalog - how can I do this?

[Dustin Getz]

Datomic allows clojure.core fns to be embedded inside datalog queries, 
except eval (http://docs.datomic.com/query.html)

1. Do we know how secure is this - what if the datalog comes from untrusted 
user input? 

2. What is the best way to implement restricted-eval like this on jvm, 
compared to on js?

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How do I just use AOT compiled class files over clj(c) source with a leiningen uberjar?

[lvh]

Thanks, Alex! That helps a lot.

Is my understanding of how uberjars with aot complication work otherwise
correct? I.e. apart from the dynamically loaded

One thing I particularly care about is compile-time precomputation. My app
uses source JSON data (that I don't control) that's pretty big, but the
stuff it produces from that data is pretty small. In ClojureScript, I did
this by just using macros, since ClojureScript conveniently guarantees
those will be evaluated on the JVM and compile time. So, for example, when
I was using ClojureScript to write a GreaseMonkey script/WebExtension
(where there are good reasons to keep everything in a single file) and i
wanted to include e.g. some literal CSS in the source, I'd just use a macro.

Does a similar trick work with Clojure and AOT compilation + uberjar, or
should I use a resource as an intermediate? I'm fine with it loading e.g.
instant18.clj (although it'd certainly be neater if I didn't have to :)))

thanks in advance,
lvh


On Mon, Apr 3, 2017 at 9:24 AM, Alex Miller <a...@puredanger.com> wrote:

> core_instant18.clj is conditionally loaded if you are using >= Java 1.8 to
> extend the Inst protocol to java.time.Instant (which didn't exist in prior
> JDKs). This is set up at:
>
> https://github.com/clojure/clojure/blob/7aad2f7dbb3a66019e5c
> ef3726d52d721e9c60df/src/clj/clojure/core.clj#L6661-L6665
>
> There is some similar conditional loading for clojure.core.reducers
> (dealing with whether using JDK 1.6 with jsr-166 jar or JDK 1.7+) in:
>
> https://github.com/clojure/clojure/blob/f572a60262852af68cdb
> 561784a517143a5847cf/src/clj/clojure/core/reducers.clj#L24
>
> Those are the two things like that that I know of in Clojure. There are a
> handful of other namespaces that are not AOT compiled - clojure.parallel
> (deprecated, also has forkjoin dependencies), clojure.instant, and
> clojure.uuid. I think the last two may just be an oversight.
>
>
> On Saturday, April 1, 2017 at 1:49:22 PM UTC-5, lvh ‌ wrote:
>>
>> Hi,
>>
>>
>> Context: I'm using Proguard to try to minimize an uberjar produced with
>> lein uberjar. This is failing, because when I run the resulting jar with
>> java -jar minimized.jar, I get errors about missing classes while trying to
>> load/compile .clj files.  proguard is renaming classes and removing unused
>> ones. In this case, the class is still in the jar, but under another name.
>> As long as the _class_ files are loaded, this should be fine (it's
>> proguard's job to rename all references). But proguard doesn't know about
>> Clojure, so the Clojure files don't get the same treatment. I understand
>> that Clojure generally imports source files as resources, but I was hoping
>> with an :aot :all uberjar, that wouldn't be the (primary) case. Removing
>> clj[csx]? files from the jar (via :uberjar-exclusions) (which I was hoping
>> would force it to try the class files), breaks:
>>
>> Caused by: java.io.FileNotFoundException: Could not locate
>> clojure/core_instant18__init.class or clojure/core_instant18.clj on
>> classpath. Please check that namespaces with dashes use underscores in the
>> Clojure file name.
>>
>> This appears to be due to this source file: https://github.com/cloju
>> re/clojure/blob/master/src/clj/clojure/core_instant18.clj ; and the lack
>> of matching class file can be explained because it is in the clojure.core
>> namespace.
>>
>> Do I misunderstand how :aot :all works? Are clj files always going to
>> exist in the resulting uberjar if I want Clojure to work?
>>
>>
>> lvh
>>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How do I just use AOT compiled class files over clj(c) source with a leiningen uberjar?

[Alex Miller]

On Saturday, April 1, 2017 at 2:24:30 PM UTC-5, Daniel Compton wrote:
>
> Zach Tellman talked 
> <https://groups.google.com/forum/#!topic/clojure-dev/HT5NnY2ROUg> about 
> making Clojure work better with Proguard in August 2016. I couldn’t find 
> any open JIRA tickets about this though.
>

Zach and I actually talked about this again at Clojure/west this week and I 
think he's going to do some work on it. I can't really comment as I don't 
know what is being requested.


-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How do I just use AOT compiled class files over clj(c) source with a leiningen uberjar?

[Alex Miller]

core_instant18.clj is conditionally loaded if you are using >= Java 1.8 to 
extend the Inst protocol to java.time.Instant (which didn't exist in prior 
JDKs). This is set up at:

https://github.com/clojure/clojure/blob/7aad2f7dbb3a66019e5cef3726d52d721e9c60df/src/clj/clojure/core.clj#L6661-L6665

There is some similar conditional loading for clojure.core.reducers 
(dealing with whether using JDK 1.6 with jsr-166 jar or JDK 1.7+) in:

https://github.com/clojure/clojure/blob/f572a60262852af68cdb561784a517143a5847cf/src/clj/clojure/core/reducers.clj#L24

Those are the two things like that that I know of in Clojure. There are a 
handful of other namespaces that are not AOT compiled - clojure.parallel 
(deprecated, also has forkjoin dependencies), clojure.instant, and 
clojure.uuid. I think the last two may just be an oversight.


On Saturday, April 1, 2017 at 1:49:22 PM UTC-5, lvh ‌ wrote:
>
> Hi,
>
>
> Context: I'm using Proguard to try to minimize an uberjar produced with 
> lein uberjar. This is failing, because when I run the resulting jar with 
> java -jar minimized.jar, I get errors about missing classes while trying to 
> load/compile .clj files.  proguard is renaming classes and removing unused 
> ones. In this case, the class is still in the jar, but under another name. 
> As long as the _class_ files are loaded, this should be fine (it's 
> proguard's job to rename all references). But proguard doesn't know about 
> Clojure, so the Clojure files don't get the same treatment. I understand 
> that Clojure generally imports source files as resources, but I was hoping 
> with an :aot :all uberjar, that wouldn't be the (primary) case. Removing 
> clj[csx]? files from the jar (via :uberjar-exclusions) (which I was hoping 
> would force it to try the class files), breaks:
>
> Caused by: java.io.FileNotFoundException: Could not locate 
> clojure/core_instant18__init.class or clojure/core_instant18.clj on 
> classpath. Please check that namespaces with dashes use underscores in the 
> Clojure file name.
>
> This appears to be due to this source file: 
> https://github.com/clojure/clojure/blob/master/src/clj/clojure/core_instant18.clj
>  
> ; and the lack of matching class file can be explained because it is in the 
> clojure.core namespace.
>
> Do I misunderstand how :aot :all works? Are clj files always going to 
> exist in the resulting uberjar if I want Clojure to work?
>  
>
> lvh
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How do I just use AOT compiled class files over clj(c) source with a leiningen uberjar?

[lvh]

Hi Daniel,


Thanks, I didn't know about Zach's talk. FWIW: that instant18 clj does not
occur in the _uberjar_ (before proguard gets its grubby mitts on it).


lvh


On Sat, Apr 1, 2017 at 2:24 PM, Daniel Compton <
daniel.compton.li...@gmail.com> wrote:

> -keep class clojure.core_instant18__init { public static void load(); }
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How do I just use AOT compiled class files over clj(c) source with a leiningen uberjar?

[Daniel Compton]

Zach Tellman talked
<https://groups.google.com/forum/#!topic/clojure-dev/HT5NnY2ROUg> about
making Clojure work better with Proguard in August 2016. I couldn’t find
any open JIRA tickets about this though.

https://www.deepbluelambda.org/programming/clojure/creating-android-applications-with-clojure--slimming-things-down-with-proguard
 and
https://github.com/sattvik/Clojure-Android-Examples/blob/master/slimmed/proguard.cfg
are
from an old post about Proguard on Android which looks like it might have
what you need to get started, though I’m not sure about core_instant18.clj.
Possibly something like this might help?

-keep class clojure.core_instant18__init { public static void load(); }

On Sun, Apr 2, 2017 at 6:49 AM lvh ‌ <_...@lvh.cc> wrote:

> Hi,
>
>
> Context: I'm using Proguard to try to minimize an uberjar produced with
> lein uberjar. This is failing, because when I run the resulting jar with
> java -jar minimized.jar, I get errors about missing classes while trying to
> load/compile .clj files.  proguard is renaming classes and removing unused
> ones. In this case, the class is still in the jar, but under another name.
> As long as the _class_ files are loaded, this should be fine (it's
> proguard's job to rename all references). But proguard doesn't know about
> Clojure, so the Clojure files don't get the same treatment. I understand
> that Clojure generally imports source files as resources, but I was hoping
> with an :aot :all uberjar, that wouldn't be the (primary) case. Removing
> clj[csx]? files from the jar (via :uberjar-exclusions) (which I was hoping
> would force it to try the class files), breaks:
>
> Caused by: java.io.FileNotFoundException: Could not locate
> clojure/core_instant18__init.class or clojure/core_instant18.clj on
> classpath. Please check that namespaces with dashes use underscores in the
> Clojure file name.
>
> This appears to be due to this source file:
> https://github.com/clojure/clojure/blob/master/src/clj/clojure/core_instant18.clj
> ; and the lack of matching class file can be explained because it is in the
> clojure.core namespace.
>
> Do I misunderstand how :aot :all works? Are clj files always going to
> exist in the resulting uberjar if I want Clojure to work?
>
>
> lvh
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
-- 

Daniel

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

How do I just use AOT compiled class files over clj(c) source with a leiningen uberjar?

[lvh ‌]

Hi,


Context: I'm using Proguard to try to minimize an uberjar produced with 
lein uberjar. This is failing, because when I run the resulting jar with 
java -jar minimized.jar, I get errors about missing classes while trying to 
load/compile .clj files.  proguard is renaming classes and removing unused 
ones. In this case, the class is still in the jar, but under another name. 
As long as the _class_ files are loaded, this should be fine (it's 
proguard's job to rename all references). But proguard doesn't know about 
Clojure, so the Clojure files don't get the same treatment. I understand 
that Clojure generally imports source files as resources, but I was hoping 
with an :aot :all uberjar, that wouldn't be the (primary) case. Removing 
clj[csx]? files from the jar (via :uberjar-exclusions) (which I was hoping 
would force it to try the class files), breaks:

Caused by: java.io.FileNotFoundException: Could not locate 
clojure/core_instant18__init.class or clojure/core_instant18.clj on 
classpath. Please check that namespaces with dashes use underscores in the 
Clojure file name.

This appears to be due to this source 
file: 
https://github.com/clojure/clojure/blob/master/src/clj/clojure/core_instant18.clj
 
; and the lack of matching class file can be explained because it is in the 
clojure.core namespace.

Do I misunderstand how :aot :all works? Are clj files always going to exist 
in the resulting uberjar if I want Clojure to work?
 

lvh

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

How do I spec a nullary function?

[Alex Miller]

(s/fdef :args (s/cat) ...

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

How do I spec a nullary function?

[Didier]

I'm trying to fdef a function of no arguments, how would I go about it? I 
only want a spec for the return value basically?

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Elegant way to do lazy infinite list with custom step

[Ghadi Shayban]

If you really want to have _exactly_ what Haskell does (derives the range 
from an example) -- yes, you could do a macro.  Clojure's new spec facility 
(in 1.9-alphas) makes the macro arguments checked at load time too, which 
is extra nice:

(set! *print-length* 5) ;; avoid printing infinite lists

(defmacro haskell-like-range [lower next dots]
  `(range ~lower Long/MAX_VALUE ~(- next lower)))

(require '[clojure.spec :as s])
(s/fdef haskell-like-range :args (s/cat :lower int? :next int? :dots 
'#{...}))

;; these will fail at load time with a really nice error message
(haskell-like-range :foo 42 ...)
(haskell-like-range 4 6 ..) ;; needs three dots according to the spec

;; this will succeed
(haskell-like-range 2 6 ...)


On Wednesday, December 14, 2016 at 1:02:38 AM UTC-5, tbc++ wrote:
>
> Ah! I knew there was a way to do it via iterate, but it was 9:30pm at the 
> time and I was tired. Nice example though.
>
> On Tue, Dec 13, 2016 at 10:03 PM, Ghadi Shayban <gsha...@gmail.com 
> > wrote:
>
>> A common way to do it in Clojure is `iterate`, no macros necessary. As of 
>> Clojure 1.7 this doesn't allocate a list at all if you reduce/fold over it:
>> (iterate #(+ % 2) 4)
>>
>>
>> On Tuesday, December 13, 2016 at 11:27:28 PM UTC-5, tbc++ wrote:
>>>
>>> I'm not aware of such a construct, but it's trivial enough to write 
>>> something like this using `range` or perhaps write a function that will 
>>> yield a lazy seq:
>>>
>>> (defn inf-list
>>>   ([x y]
>>> (cons x (cons y (inf-list x y 2
>>>   ([x y c]
>>> (cons (+ x (* c (- y x)))
>>>   (lazy-seq (inf-list x y (inc c))
>>>
>>> No macros required
>>>
>>> On Tue, Dec 13, 2016 at 9:14 PM, bill nom nom <hellomot...@gmail.com> 
>>> wrote:
>>>
>>>> In Haskell, I can get an infinite lazy list that is incremented by two 
>>>> by, starting at 4
>>>> [4,6..]
>>>>
>>>> which yields the result
>>>> [4,6,8,10...]
>>>>
>>>> the more general form
>>>> [x,y..] produces [x, x + (y-x), x + 2 * (y-x), x + 3 * (y-x)...]
>>>>
>>>> Is there a way to make a macro of this in clojure, or is there 
>>>> something already as elegant? 
>>>>
>>>> -- 
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Clojure" group.
>>>> To post to this group, send email to clo...@googlegroups.com
>>>> Note that posts from new members are moderated - please be patient with 
>>>> your first post.
>>>> To unsubscribe from this group, send email to
>>>> clojure+u...@googlegroups.com
>>>> For more options, visit this group at
>>>> http://groups.google.com/group/clojure?hl=en
>>>> --- 
>>>> You received this message because you are subscribed to the Google 
>>>> Groups "Clojure" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send 
>>>> an email to clojure+u...@googlegroups.com.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>>
>>>
>>> -- 
>>> “One of the main causes of the fall of the Roman Empire was that–lacking 
>>> zero–they had no way to indicate successful termination of their C 
>>> programs.”
>>> (Robert Firth) 
>>>
>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com 
>> 
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com 
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to clojure+u...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
> “One of the main causes of the fall of the Roman Empire was that–lacking 
> zero–they had no way to indicate successful termination of their C 
> programs.”
> (Robert Firth) 
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Elegant way to do lazy infinite list with custom step

[Timothy Baldridge]

Ah! I knew there was a way to do it via iterate, but it was 9:30pm at the
time and I was tired. Nice example though.

On Tue, Dec 13, 2016 at 10:03 PM, Ghadi Shayban <gshay...@gmail.com> wrote:

> A common way to do it in Clojure is `iterate`, no macros necessary. As of
> Clojure 1.7 this doesn't allocate a list at all if you reduce/fold over it:
> (iterate #(+ % 2) 4)
>
>
> On Tuesday, December 13, 2016 at 11:27:28 PM UTC-5, tbc++ wrote:
>>
>> I'm not aware of such a construct, but it's trivial enough to write
>> something like this using `range` or perhaps write a function that will
>> yield a lazy seq:
>>
>> (defn inf-list
>>   ([x y]
>> (cons x (cons y (inf-list x y 2
>>   ([x y c]
>> (cons (+ x (* c (- y x)))
>>   (lazy-seq (inf-list x y (inc c))
>>
>> No macros required
>>
>> On Tue, Dec 13, 2016 at 9:14 PM, bill nom nom <hellomot...@gmail.com>
>> wrote:
>>
>>> In Haskell, I can get an infinite lazy list that is incremented by two
>>> by, starting at 4
>>> [4,6..]
>>>
>>> which yields the result
>>> [4,6,8,10...]
>>>
>>> the more general form
>>> [x,y..] produces [x, x + (y-x), x + 2 * (y-x), x + 3 * (y-x)...]
>>>
>>> Is there a way to make a macro of this in clojure, or is there something
>>> already as elegant?
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to clo...@googlegroups.com
>>> Note that posts from new members are moderated - please be patient with
>>> your first post.
>>> To unsubscribe from this group, send email to
>>> clojure+u...@googlegroups.com
>>> For more options, visit this group at
>>> http://groups.google.com/group/clojure?hl=en
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to clojure+u...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>>
>> --
>> “One of the main causes of the fall of the Roman Empire was that–lacking
>> zero–they had no way to indicate successful termination of their C
>> programs.”
>> (Robert Firth)
>>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
“One of the main causes of the fall of the Roman Empire was that–lacking
zero–they had no way to indicate successful termination of their C
programs.”
(Robert Firth)

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Elegant way to do lazy infinite list with custom step

[Ghadi Shayban]

A common way to do it in Clojure is `iterate`, no macros necessary. As of 
Clojure 1.7 this doesn't allocate a list at all if you reduce/fold over it:
(iterate #(+ % 2) 4)


On Tuesday, December 13, 2016 at 11:27:28 PM UTC-5, tbc++ wrote:
>
> I'm not aware of such a construct, but it's trivial enough to write 
> something like this using `range` or perhaps write a function that will 
> yield a lazy seq:
>
> (defn inf-list
>   ([x y]
> (cons x (cons y (inf-list x y 2
>   ([x y c]
> (cons (+ x (* c (- y x)))
>   (lazy-seq (inf-list x y (inc c))
>
> No macros required
>
> On Tue, Dec 13, 2016 at 9:14 PM, bill nom nom <hellomot...@gmail.com 
> > wrote:
>
>> In Haskell, I can get an infinite lazy list that is incremented by two 
>> by, starting at 4
>> [4,6..]
>>
>> which yields the result
>> [4,6,8,10...]
>>
>> the more general form
>> [x,y..] produces [x, x + (y-x), x + 2 * (y-x), x + 3 * (y-x)...]
>>
>> Is there a way to make a macro of this in clojure, or is there something 
>> already as elegant? 
>>
>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com 
>> 
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com 
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to clojure+u...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
> “One of the main causes of the fall of the Roman Empire was that–lacking 
> zero–they had no way to indicate successful termination of their C 
> programs.”
> (Robert Firth) 
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Elegant way to do lazy infinite list with custom step

[Alex Miller]

(range 4 Long/MAX_VALUE 2)

On Tuesday, December 13, 2016 at 10:14:00 PM UTC-6, bill nom nom wrote:
>
> In Haskell, I can get an infinite lazy list that is incremented by two by, 
> starting at 4
> [4,6..]
>
> which yields the result
> [4,6,8,10...]
>
> the more general form
> [x,y..] produces [x, x + (y-x), x + 2 * (y-x), x + 3 * (y-x)...]
>
> Is there a way to make a macro of this in clojure, or is there something 
> already as elegant? 
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Elegant way to do lazy infinite list with custom step

[Timothy Baldridge]

I'm not aware of such a construct, but it's trivial enough to write
something like this using `range` or perhaps write a function that will
yield a lazy seq:

(defn inf-list
  ([x y]
(cons x (cons y (inf-list x y 2
  ([x y c]
(cons (+ x (* c (- y x)))
  (lazy-seq (inf-list x y (inc c))

No macros required

On Tue, Dec 13, 2016 at 9:14 PM, bill nom nom 
wrote:

> In Haskell, I can get an infinite lazy list that is incremented by two by,
> starting at 4
> [4,6..]
>
> which yields the result
> [4,6,8,10...]
>
> the more general form
> [x,y..] produces [x, x + (y-x), x + 2 * (y-x), x + 3 * (y-x)...]
>
> Is there a way to make a macro of this in clojure, or is there something
> already as elegant?
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
“One of the main causes of the fall of the Roman Empire was that–lacking
zero–they had no way to indicate successful termination of their C
programs.”
(Robert Firth)

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Elegant way to do lazy infinite list with custom step

[bill nom nom]

In Haskell, I can get an infinite lazy list that is incremented by two by, 
starting at 4
[4,6..]

which yields the result
[4,6,8,10...]

the more general form
[x,y..] produces [x, x + (y-x), x + 2 * (y-x), x + 3 * (y-x)...]

Is there a way to make a macro of this in clojure, or is there something 
already as elegant? 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: what does (seq) in zipmap do?

[Alex Miller]

keys and vals could be any kind of collection. Calling seq will return 
either nil or a seq with at least one value. By forcing empty collections 
to nil allows the (and ks vs) to work.

That is if keys is [] and vals is [1], then:

(and keys vals) is [1] (a truthy value)

But (seq []) is nil and (seq [1]) is (1) and (and nil (1)) is nil (a falsey 
value).

There are a lot of ways zipmap's performance can be improved (particularly 
if keys and vals are reducible) and there is a ticket where some ideas have 
been discussed: http://dev.clojure.org/jira/browse/CLJ-1005

On Tuesday, November 29, 2016 at 9:42:48 PM UTC-6, larry google groups 
wrote:
>
> I apologize for this question, because I think it has been asked before, 
> and yet I can not find the answer. 
>
> In the definition of zipmap, what do these 2 lines do?
>
> ks (seq keys)
> vs (seq vals)
>
> In particular, what is (seq) doing? Is this to ensure that ks is false if 
> "keys" is empty? And vs is false if "vals" is empty? Because an empty list 
> (or vector) is truthy but we want it to be falsey in this situation?  
>
> Is there any other reason to use (seq), other than to set the 
> truthy/falsey values of ks and vs? 
>
>
> (defn zipmap
> "Returns a map with the keys mapped to the corresponding vals."
> {:added "1.0"
> :static true}
> [keys vals]
> (loop [map {}
> ks (seq keys)
> vs (seq vals)]
> (if (and ks vs)
> (recur (assoc map (first ks) (first vs))
> (next ks)
> (next vs))
> map))) 
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: what does (seq) in zipmap do?

[Andy Fingerhut]

I can't answer for the author, but the answer you gave would be a good
sufficient reason to call seq in those places, yes?

If you create a function zipmap2 that is identical to zipmap, except it
does not have any calls to seq, you get these results:

user=> (zipmap [] [1])

{}

user=> (zipmap2 [] [1])

{nil 1}


I definitely think the former result makes more sense than the latter does.

If you are thinking that it would be better to have the calls to seq in the
if condition, that would be at least a little bit slower than the current
implementation, as it would have 2 additional calls to seq for every
iteration through the loop.

Andy

On Tue, Nov 29, 2016 at 7:42 PM, larry google groups <
lawrencecloj...@gmail.com> wrote:

> I apologize for this question, because I think it has been asked before,
> and yet I can not find the answer.
>
> In the definition of zipmap, what do these 2 lines do?
>
> ks (seq keys)
> vs (seq vals)
>
> In particular, what is (seq) doing? Is this to ensure that ks is false if
> "keys" is empty? And vs is false if "vals" is empty? Because an empty list
> (or vector) is truthy but we want it to be falsey in this situation?
>
> Is there any other reason to use (seq), other than to set the
> truthy/falsey values of ks and vs?
>
>
> (defn zipmap
> "Returns a map with the keys mapped to the corresponding vals."
> {:added "1.0"
> :static true}
> [keys vals]
> (loop [map {}
> ks (seq keys)
> vs (seq vals)]
> (if (and ks vs)
> (recur (assoc map (first ks) (first vs))
> (next ks)
> (next vs))
> map)))
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

what does (seq) in zipmap do?

[larry google groups]

I apologize for this question, because I think it has been asked before, 
and yet I can not find the answer. 

In the definition of zipmap, what do these 2 lines do?

ks (seq keys)
vs (seq vals)

In particular, what is (seq) doing? Is this to ensure that ks is false if 
"keys" is empty? And vs is false if "vals" is empty? Because an empty list 
(or vector) is truthy but we want it to be falsey in this situation?  

Is there any other reason to use (seq), other than to set the truthy/falsey 
values of ks and vs? 


(defn zipmap
"Returns a map with the keys mapped to the corresponding vals."
{:added "1.0"
:static true}
[keys vals]
(loop [map {}
ks (seq keys)
vs (seq vals)]
(if (and ks vs)
(recur (assoc map (first ks) (first vs))
(next ks)
(next vs))
map))) 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How do I call a function in the catch block of try/catch?

[larry google groups]

Sorry, I am an idiot. I had added a try/catch to the first "convert" when I 
was debugging. 

Ignore all this. 



On Friday, November 25, 2016 at 6:41:57 PM UTC-5, larry google groups wrote:
>
> I know some of you will tell me that I shouldn't use try/catch for control 
> flow. That is fine. I might redo this. But for now, I'd like to simply know 
> why this doesn't work. I have this function: 
>
> (defn start []
>   (let [filenames-as-seq-of-strings (.list (io/file 
> "/home/ec2-user/uploads/"))]
> (doseq [f filenames-as-seq-of-strings]
>   (println "in file_converter.start, the file is " (str f))
>   (spit  (str "/home/ec2-user/converted_csv_files/" f ".csv") " ")
>   (let [input-file (java.io.File. (str "/home/ec2-user/uploads/" f))
> 
> output-file (java.io.File. (str 
> "/home/ec2-user/converted_csv_files/" f ".csv"))]
> (try 
>   (xls/convert input-file output-file)
>   (catch Exception e
> (println e)
> (xlsx/convert input-file output-file)))
>
> Given some Microsoft Excel files, I first try to parse them as xls files, 
> and if that doesn't work, then in the catch block I would like to try to 
> parse them xlsx files. 
>
> If I run this app in the terminal with "java -jar" I see the exception as 
> I expect. But this line never seems to get called:
>
> (xlsx/convert input-file output-file)
>
> This convert function starts with: 
>
> (defn convert
>   "Stealing code from 
> http://stackoverflow.com/questions/31873931/java-lang-outofmemoryerror-gc-overhead-limit-exceeded-when-loading-an-xlsx-file
> "
>   [^java.io.File input-file ^java.io.File output-file]
>(println "at the start of xlsx")
>
> But I never see that println in the terminal. 
>
> Why not?
>
>
>
>
>
>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How do I call a function in the catch block of try/catch?

[larry google groups]

When the Exception is thrown, I get: 



  
 java.lang.Thread.run   Thread.java:  745

 java.util.concurrent.ThreadPoolExecutor$Worker.run   
ThreadPoolExecutor.java:  615
  
java.util.concurrent.ThreadPoolExecutor.runWorker   
ThreadPoolExecutor.java: 1145
  
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run 
 ScheduledThreadPoolExecutor.java:  293
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301 
 ScheduledThreadPoolExecutor.java:  178

java.util.concurrent.FutureTask.runAndReset   
FutureTask.java:  304

java.util.concurrent.Executors$RunnableAdapter.call   
 Executors.java:  471

...

excel-to-csv.start/convert-excel/fn start.clj:   64
  
excel-to-csv.file-converter/startfile_converter.clj:   30
  
 excel-to-csv.file-converter/start/fnfile_converter.clj:   
31
  
 excel-to-csv.xls/convert   xls.clj:   69
  
org.apache.poi.hssf.usermodel.HSSFWorkbook. 
HSSFWorkbook.java:  303
  
org.apache.poi.hssf.usermodel.HSSFWorkbook. 
HSSFWorkbook.java:  322

 org.apache.poi.poifs.filesystem.POIFSFileSystem. 
 POIFSFileSystem.java:  138

org.apache.poi.poifs.storage.HeaderBlock. 
 HeaderBlock.java:  104

org.apache.poi.poifs.storage.HeaderBlock. 
 HeaderBlock.java:  131
org.apache.poi.poifs.filesystem.OfficeXmlFileException: The supplied data 
appears to be in the Office 2007+ XML. You are calling the part of POI that 
deals with OLE2 Office Documents. You need to call a different part of POI 
to process this data (eg XSSF instead of HSSF)


Which is what I am expecting. But it seems like my try/catch does not catch 
this. How is that possible? 




On Friday, November 25, 2016 at 6:41:57 PM UTC-5, larry google groups wrote:
>
> I know some of you will tell me that I shouldn't use try/catch for control 
> flow. That is fine. I might redo this. But for now, I'd like to simply know 
> why this doesn't work. I have this function: 
>
> (defn start []
>   (let [filenames-as-seq-of-strings (.list (io/file 
> "/home/ec2-user/uploads/"))]
> (doseq [f filenames-as-seq-of-strings]
>   (println "in file_converter.start, the file is " (str f))
>   (spit  (str "/home/ec2-user/converted_csv_files/" f ".csv") " ")
>   (let [input-file (java.io.File. (str "/home/ec2-user/uploads/" f))
> 
> output-file (java.io.File. (str 
> "/home/ec2-user/converted_csv_files/" f ".csv"))]
> (try 
>   (xls/convert input-file output-file)
>   (catch Exception e
> (println e)
> (xlsx/convert input-file output-file)))
>
> Given some Microsoft Excel files, I first try to parse them as xls files, 
> and if that doesn't work, then in the catch block I would like to try to 
> parse them xlsx files. 
>
> If I run this app in the terminal with "java -jar" I see the exception as 
> I expect. But this line never seems to get called:
>
> (xlsx/convert input-file output-file)
>
> This convert function starts with: 
>
> (defn convert
>   "Stealing code from 
> http://stackoverflow.com/questions/31873931/java-lang-outofmemoryerror-gc-overhead-limit-exceeded-when-loading-an-xlsx-file
> "
>   [^java.io.File input-file ^java.io.File output-file]
>(println "at the start of xlsx")
>
> But I never see that println in the terminal. 
>
> Why not?
>
>
>
>
>
>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

How do I call a function in the catch block of try/catch?

[larry google groups]

I know some of you will tell me that I shouldn't use try/catch for control 
flow. That is fine. I might redo this. But for now, I'd like to simply know 
why this doesn't work. I have this function: 

(defn start []
  (let [filenames-as-seq-of-strings (.list (io/file 
"/home/ec2-user/uploads/"))]
(doseq [f filenames-as-seq-of-strings]
  (println "in file_converter.start, the file is " (str f))
  (spit  (str "/home/ec2-user/converted_csv_files/" f ".csv") " ")
  (let [input-file (java.io.File. (str "/home/ec2-user/uploads/" f))

output-file (java.io.File. (str 
"/home/ec2-user/converted_csv_files/" f ".csv"))]
(try 
  (xls/convert input-file output-file)
  (catch Exception e
(println e)
(xlsx/convert input-file output-file)))

Given some Microsoft Excel files, I first try to parse them as xls files, 
and if that doesn't work, then in the catch block I would like to try to 
parse them xlsx files. 

If I run this app in the terminal with "java -jar" I see the exception as I 
expect. But this line never seems to get called:

(xlsx/convert input-file output-file)

This convert function starts with: 

(defn convert
  "Stealing code from 
http://stackoverflow.com/questions/31873931/java-lang-outofmemoryerror-gc-overhead-limit-exceeded-when-loading-an-xlsx-file;
  [^java.io.File input-file ^java.io.File output-file]
   (println "at the start of xlsx")

But I never see that println in the terminal. 

Why not?







-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How do I set up database in the context of a web application ?

[Alan Thompson]

You can find a nice overview in the Clojure Cookbook

.




On Wed, Nov 2, 2016 at 2:58 PM, 'George' via Clojure <
clojure@googlegroups.com> wrote:

> I'd like to use datomic, but my question applies to any database.
>
> When I used a framework in another language it was easy, the config file
> and ORM did setup the connection. I only installed an adapter and wrote
> the URI of the database..
> Now as I want to use a minimal framework or to totally get rid of the
> framework,  I dont know what is the righ way to set up the connection.
>
> I tried to write the connection details (such as URI, conn and db) in the
> route-handler file and it works.
> But I know that I'm doing something wrong, because I've read that some
> setup is done by leiningen,
> for example in the examples I found about SQL. I searched and couldn't
> find examples for datomic and luminus.
>
> I guess my routes handler file should only require something to import and
> than in each function
>  queries using datomic.api. But datomic.api/q needs a db value, to be
> imported from somewhere.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

How do I set up database in the context of a web application ?

['George' via Clojure]

I'd like to use datomic, but my question applies to any database.

When I used a framework in another language it was easy, the config file 
and ORM did setup the connection. I only installed an adapter and wrote the 
URI of the database..
Now as I want to use a minimal framework or to totally get rid of the 
framework,  I dont know what is the righ way to set up the connection.

I tried to write the connection details (such as URI, conn and db) in the 
route-handler file and it works.
But I know that I'm doing something wrong, because I've read that some 
setup is done by leiningen, 
for example in the examples I found about SQL. I searched and couldn't find 
examples for datomic and luminus.

I guess my routes handler file should only require something to import and 
than in each function
 queries using datomic.api. But datomic.api/q needs a db value, to be 
imported from somewhere.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Is there a standard name for this transducer? Or a more idiomatic way to do it?

[Kevin Downey]

looks like a variation of partition-by

On 09/14/2016 07:16 PM, Marshall handheld Flax wrote:
> This looks like a standard thing you might want to do with a transducer:
> accept a stream of inputs and statefully group them into a stream of
> varying-length vectors.  A typical example might be to accept a series
> of bowling throws and emit them grouped into bowling frames.  So you
> have a generic "vectorTransducer" which is defined by the rule as to
> when one frame is full and the next begins.  Does such a thing have a
> standard name?
> 
> Thanks!
> 
> Marshall
> 
> (defn vectorTransducer [timeForNextVector]
>   (fn [rf]
> (let [pending (volatile! [])]
>   (fn
> ([] (rf))
> ([result] (if (not-empty @pending)
> (rf result @pending)
> (rf result)))
> ([result input]
>  (let [appended (conj @pending input)]
>(if (timeForNextVector appended)
>  (do (vreset! pending [])
>  (rf result appended))
>  (do (vreset! pending appended)
>  result
> 
> (defn bowlingTransducer[]
>   (vectorTransducer
>(fn ([v]
> (or (<= 2 (count v))
> (<= 10 (reduce + v)))
> 
> (into [] (bowlingTransducer)
>   (concat (range 11) (range 11)))
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to clojure+unsubscr...@googlegroups.com
> <mailto:clojure+unsubscr...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.


-- 
And what is good, Phaedrus,
And what is not good—
Need we ask anyone to tell us these things?

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Is there a standard name for this transducer? Or a more idiomatic way to do it?

[Marshall handheld Flax]

This looks like a standard thing you might want to do with a transducer: 
accept a stream of inputs and statefully group them into a stream of 
varying-length vectors.  A typical example might be to accept a series of 
bowling throws and emit them grouped into bowling frames.  So you have a 
generic "vectorTransducer" which is defined by the rule as to when one 
frame is full and the next begins.  Does such a thing have a standard name?

Thanks!

Marshall

(defn vectorTransducer [timeForNextVector]
  (fn [rf]
(let [pending (volatile! [])]
  (fn
([] (rf))
([result] (if (not-empty @pending)
(rf result @pending)
(rf result)))
([result input]
 (let [appended (conj @pending input)]
   (if (timeForNextVector appended)
     (do (vreset! pending [])
 (rf result appended))
     (do (vreset! pending appended)
 result

(defn bowlingTransducer[]
  (vectorTransducer
   (fn ([v]
(or (<= 2 (count v))
(<= 10 (reduce + v)))

(into [] (bowlingTransducer)
  (concat (range 11) (range 11)))

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to do functional programming

[John Newman]

ts are passed through the call stack from function to
> function. In practice, because stack sizes are limited, most values passed
> between functions will be boxed references pointing to memory locations on
> the heap. However, as far as the functional programmer is concerned, there
> is no heap and there is no mutable state. Immutable values simply flow
> seamlessly from one function to the next until the program has finished.
> The driving principle is that computation is accomplished by transforming
> data through mapping a set of immutable inputs to an immutable output.
> Think f(x,y) = z.
>
> In order to accomplish this stateless pipeline of data transformations,
> functions much possess a property called "referential transparency". This
> means that a function must be able to calculate its outputs
> deterministically using only its inputs AND without modifying any of its
> inputs in the process. This property is preserved even when global
> variables are referenced within the body of a function as long as the
> values associated with those global variables are constants throughout the
> lifetime of the program.
>
> In practice, very few languages (outside of Haskell) actually try to be
> completely pure and allow no mutation of state whatsoever. Clojure opts to
> make all data immutable by default, but provides some special language
> features (refs, atoms, agents, volatiles) that you can use if you really do
> want to write an algorithm that involves some mutable state. Unless there
> is a performance bottleneck (as in numerical computing) or no other
> sensible way to model the domain (as in some web applications), making use
> of these features for mutable state is generally frowned upon. When they
> are really necessary and valuable, however, Clojure's language tools for
> accomplishing mutation are wonderful because they carefully protect it
> against concurrency clashes in multi-threated environments.
>
> To approach writing a functional program, first think about how to model
> the computation as a series of data transformations. Then build your
> program from the bottom up (prototyping and testing it in the REPL) by
> writing small, referentially transparent functions that describe the lower
> level operations that you will need. Then build higher level functions on
> top of those that build up your abstractions in a layer cake style until
> you reach your entry point function that either takes input data from the
> outside world or generates it internally and then starts the execution of
> the function call tree.
>
> This is, of course, my attempt at summarizing the mindset that I
> frequently use when trying to write a new functional program, and I welcome
> additions or corrections from other folks in this thread to further flesh
> it out. Best of luck in your functional programming adventures and with
> Clojure in particular. The FP way of thinking can be a bit tricky to wrap
> your mind around when coming from a traditional OOP background, but once
> you grok it, there is a great feeling of freedom and simplicity that
> emerges from what I suspect many of my fellow Clojurians would agree is a
> far more elegant and powerful programming paradigm.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to do functional programming

[Gary Johnson]

A shell script written in a procedural style (e.g. with Bash or equivalent 
shell language) will frequently start out by declaring some global 
variables, then perform some conditional checks (if then else), throw in a 
few loops (for, while), and ultimately end up with some new values in those 
initially declared variables that you use as your program's output. If you 
are feeling particularly intrepid, you might factor out some repetitive 
operations into separate subroutines defined higher up in the file and then 
call them as necessary in those aforementioned conditional blocks and loops.

The mental model behind this type of programming is the Universal Turing 
Machine. Your variables are some internal state that the program 
instructions are reading and writing, reading and writing, writing and 
reading until you get to the last instruction that returns some subset of 
the final variable values. The driving principle is that computation is 
accomplished by the free mutation of state.

A program written in a functional style (e.g. with any Lisp, one of the 
MLs, Haskell, Clean, etc) begins with a chunk of data, which may either be 
hard-coded, input from the outside world, or generated internally with a 
function like "range" or "rand". This piece of data may or may not be 
stored in one or more global variables. However (and this is HUGE however), 
these are not generally mutated over the life of the program. That is to 
say, they are constants. More often than not, you won't even store the 
initial data in a global variable but will just feed it into a function 
that processes it and spits out some new data, which is then fed to another 
function that performs some other processing operation and again spits out 
some new data. Ultimately, the data that you produce is passed through an 
arbitrarily long pipeline of functions until the final result is produced 
and returned by the program. In practice, these function calls are rarely 
linear and are much more likely to form a branching call tree. But 
ultimately, the relevant branches of this tree will be traversed and 
executed in a depth first manner (unless lazy evaluation inserts its magic 
to reorder some of that computation behind the scenes) and you still get to 
a final output returned by the last function fall evaluated.

The mental model behind this type of programming is Lambda Calculus. There 
is no mutable state anywhere in a pure functional program, and instead the 
intermediate results are passed through the call stack from function to 
function. In practice, because stack sizes are limited, most values passed 
between functions will be boxed references pointing to memory locations on 
the heap. However, as far as the functional programmer is concerned, there 
is no heap and there is no mutable state. Immutable values simply flow 
seamlessly from one function to the next until the program has finished. 
The driving principle is that computation is accomplished by transforming 
data through mapping a set of immutable inputs to an immutable output. 
Think f(x,y) = z.

In order to accomplish this stateless pipeline of data transformations, 
functions much possess a property called "referential transparency". This 
means that a function must be able to calculate its outputs 
deterministically using only its inputs AND without modifying any of its 
inputs in the process. This property is preserved even when global 
variables are referenced within the body of a function as long as the 
values associated with those global variables are constants throughout the 
lifetime of the program.

In practice, very few languages (outside of Haskell) actually try to be 
completely pure and allow no mutation of state whatsoever. Clojure opts to 
make all data immutable by default, but provides some special language 
features (refs, atoms, agents, volatiles) that you can use if you really do 
want to write an algorithm that involves some mutable state. Unless there 
is a performance bottleneck (as in numerical computing) or no other 
sensible way to model the domain (as in some web applications), making use 
of these features for mutable state is generally frowned upon. When they 
are really necessary and valuable, however, Clojure's language tools for 
accomplishing mutation are wonderful because they carefully protect it 
against concurrency clashes in multi-threated environments.

To approach writing a functional program, first think about how to model 
the computation as a series of data transformations. Then build your 
program from the bottom up (prototyping and testing it in the REPL) by 
writing small, referentially transparent functions that describe the lower 
level operations that you will need. Then build higher level functions on 
top of those that build up your abstractions in a layer cake style until 
you reach your entry point function that either takes input data from the 
outside world or generates

Re: How to do functional programming

[Piyush Katariya]

www.slideshare.net/*piyush*katariya/*thinking*-*functionally*

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Do you use any Clojars data feeds?

[Toby Crawley]

Clojars provides a few static data feeds[1]:

- a list of poms in the repo (all-poms.txt.gz)
- a list of all releases in the repo (all-jars.clj.gz)
- an expanded list of releases, including more metadata from the poms
(feed.clj.gz)

We're evaluating our bandwidth usage in anticipation of moving the
repository to a CDN to improve availability, and noticed some odd
request patterns for these feeds.

There are sites/services out there that make (sometimes heavy) use of
these feeds, so I want to point out that they are only updated hourly,
so requesting them more often gives you no benefit. Even though it's
only 1.4MB in size, feed.clj.gz alone accounted for ~20% (~200GB) of
Clojars' bandwidth usage for the past month, with one client
requesting it 30 times *per minute*.

So if you (or someone you love) uses one of these feeds, can you
confirm that you are caching a copy for at least an hour before
retrieving the latest version?

Thanks!
Your Friendly Neighborhood Clojars Staff

[1]: https://github.com/clojars/clojars-web/wiki/Data

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: clojure.spec merge is excellent, but do you consider dissoc spec for keys?

[Alex Miller]

Well first I'd say it's actually more important here that you are reusing 
the attribute specs for ::fname ::lname etc across front and back.

And second, perhaps you should be breaking out the common parts into a spec 
you can reuse instead:

(s/def ::person-shared-spec (s/keys ::req-un [::fname ::lname]))
(s/def ::person-spec (s/merge ::person-shared-spec (s/keys ::req-un 
[::id])))
(s/def ::person-ui-spec (s/merge ::person-shared-spec (s/keys ::req-un 
[::channel])))

Or maybe you really want to combine both of these into a single spec:

(s/def ::person-spec (s/keys :req-un [(or ::id ::channel) ::fname ::lname]))

Or you could potentially use multi-spec, but I doubt you need an open spec 
for this.

We're not going to add anything like dissoc afaik.

On Tuesday, July 19, 2016 at 11:54:52 AM UTC-5, Mamun wrote:
>
> Hi 
>
> Without dissoc in keys, how to avoid duplicate spec between backend and 
> frontend layer?
>
> As an example 
>
> I have backend service where I defined spec like this 
>
> (s/def ::person-spec (s/keys ::req-un [::id ::fname ::lname]))
>
> Here Id is mandatory for some purpose. 
>
> Now application layer I would like to reuse that backend spec but only id. 
> Look like now it is not possible as there is no dissoc 
>
> As it is application layer 
>
> (s/def ::person-ui-spec (s/merge (s/keys ::req-un [::channel])
>::person-spec
>) )
>
> merge is excellent, as I could reuse exiting spec. But how I dissoc id 
> from exiting spec.
>
> Only way is now to do is define again in application layer.
>
> (s/def ::person-ui-spec (s/keys ::req-un [::channel ::fname ::lname]))
>
>
> Do you consider dissoc in spec for keys?
>
>
>
> Br,
> Mamun
> bnp paribas groups
>
>
>
>
>
>
>
>
>
>
>  
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.