records alias with local vars
In this example (defrecord x [y]) (defn b [x] (.getBytes ^String x)) The compiler fails to resolve .getBytes. It emits reflection code, saying x is a class. It is resolved by deleting the defrecord, or renaming the parameter. Is this expected, or documented? Are there other cases like this? -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/clojure/38933dc0-1b8e-452a-ae01-feec77739623%40googlegroups.com.
Re: avoiding casts with aget & friends
Is there any way to inspect what the jit does to it?
On Wednesday, January 30, 2019 at 10:07:38 PM UTC-8, Alex Miller wrote:
>
>
>
> On Wed, Jan 30, 2019 at 11:07 PM Brian Craft > wrote:
>
>> With much experimentation, I ended up with this:
>>
>> (let [a 1
>> b (.longValue ^Long (:foo {:foo 3}))
>> c (if (< a b) a b)])
>>
>> which seems to avoid the longCast call:
>>
>> Object o;
>> if (_thunk__0__ == (o = _thunk__0__.get(const__3))) {
>> o = (__thunk__0__ =
>> __site__0__.fault(const__3)).get(const__3);
>> }
>> final long b = (long)o;
>> final long c = (a < b) ? a : b;
>>
>> I don't know if this is advisable. Does anyone do this?
>>
>
> No, I wouldn't do this. `long` can inline so it's going to be better (it's
> also more likely to jit well as it's used other places and is likely hotter
> in the jit).
>
> Going back to the original...
>
> (let [a 1
> b (:foo {:foo 3})
> c (if (< a b) a b)])
>
> let will track primitives if possible.
> - a is going to be a primitive long.
> - (:foo {:foo 3}) is going to (always) return an Object and it's best to
> recognize that and make an explicit cast to a primitive long.
> - if a and b are both primitives, then < is able to inline a
> long-primitive comparison (via Numbers.lt())
> - the if is going to return an Object though, so again you'll probably
> want to type hint or cast c, but it's hard to know for sure without seeing
> more code
>
> Without other info, I would probably start with
>
> (let [a 1
> b (long (:foo {:foo 3}))
> c (if (< a b) a b)])
>
> Or alternately, it might be better to just type hint b in the comparison
> to avoid reboxing b, but hard to know without more context:
>
> (let [a 1
> b (:foo {:foo 3})
> c (if (< a ^long b) a b)])
>
> Comparing the bytecode for these (skipping everything up through the
> keyword lookup, which is same for all):
>
> Original: Option 1:
>Option 2:
> 45: astore_2 45: invokestatic #41 45:
> astore_2
> 46: lload_0 48: lstore_2
>46: lload_0
> 47: aload_249: lload_0
> 47: aload_2
> 48: invokestatic #41 50: lload_2
> 48: checkcast #37
> 51: ifeq 62 51: lcmp
> 51: invokestatic #43
> 54: lload_0 52: ifge 60
> 54: lcmp
> 55: invokestatic #45 55: lload_0
> 55: ifge 66
> 58: goto 6556: goto 61
> 58: lload_0
> 61: pop 59: pop
> 59: invokestatic #49
> 62: aload_260: lload_2
> 62: goto 69
> 63: aconst_null 61: lstore4
> 65: pop
> 64: astore_2 63: lload 4
>66: aload_2
> 65: astore_3 65: invokestatic #47 67:
> aconst_null
> 66: aload_3
> 68: astore_2
> 67: aconst_null
>69: astore_3
> 68: astore_3
> 70: aload_3
>
> 71: aconst_null
>
> 72: astore_3
>
> Option 1 does an lstore/lload (long) instead of an astore/lstore (object).
> Both options use lcmp which is likely the fastest path to compare longs.
> I've omitted some info here to make these fit, but Original line 48 will
> invoke Numbers.lt:(JLjava/lang/Object;)Z which is the Numbers.lt(long,
> Object) - lcmp is definitely going to be preferable to this. Also of
> importance is that in Option 1, both a and b are stored as longs and loaded
> as longs so if there is subsequent stuff to do on them, they can avoid
> boxing (this is also betrayed by the shorter length from fewer casts).
>
> Your longValue one is like Option 1, but starts:
>
> 45: checkcast #37 // class java/lang/Long
> 48: inv
Re: avoiding casts with aget & friends
With much experimentation, I ended up with this:
(let [a 1
b (.longValue ^Long (:foo {:foo 3}))
c (if (< a b) a b)])
which seems to avoid the longCast call:
Object o;
if (_thunk__0__ == (o = _thunk__0__.get(const__3))) {
o = (__thunk__0__ = __site__0__.fault(const__3)).get(const__3);
}
final long b = (long)o;
final long c = (a < b) ? a : b;
I don't know if this is advisable. Does anyone do this?
Also just noted this is another case where explicitly calling something
seems to make it disappear. :-p
On Wednesday, January 30, 2019 at 8:41:49 PM UTC-8, Brian Craft wrote:
>
> The full context is large. But, for example, in this code:
>
> (let [a 1
> b (:foo {:foo 3})
> c (if (< a b) a b)])
>
> b and c are Object (if the disassembly is to be believed) which leads to
> casts where c is used later. Also, the compare is calling Numbers.lt, which
> is going to do a bunch of casting & dispatch.
>
> Adding a :long hint on b, b is still Object, and the compare becomes
>
> if (a < RT.longCast(b)) {
> num = Numbers.num(a);
> }
>
> with a long cast that doesn't seem necessary. Also, c is still Object.
>
> Casting the lookup to long, like (long (:foo {:foo 3})), b and c are now
> long, but there's now a cast on the return of the lookup
>
> Object x;
> if (_thunk__0__ == (x = _thunk__0__.get(const__4))) {
> x = (__thunk__0__ = __site__0__.fault(const__4)).get(const__4);
> }
> final long b = RT.longCast(x);
> final long c = (a < b) ? a : b;
>
> which is going to hit the RT.longCast(Object) method, and start probing
> for the class so it can pick a method.
>
>
> On Wednesday, January 30, 2019 at 6:58:07 PM UTC-8, Alex Miller wrote:
>>
>> It would really help to see a full function of code context. From that I
>> could probably talk a little more about what I would expect the compiler to
>> understand and how you might be able to influence it.
>>
>> On Wed, Jan 30, 2019 at 8:50 PM Brian Craft wrote:
>>
>>> If there is unnecessary casting or boxing, how do you avoid it? hinting
>>> and casting affect it, but not in ways I understand, or can predict.
>>>
>>> On Wednesday, January 30, 2019 at 6:06:37 PM UTC-8, Alex Miller wrote:
>>>>
>>>> Sometimes the insertion of profiling instrumentation magnifies the cost
>>>> of things in a hot loop or provides misleading hot spot info. If you're
>>>> using a tracing profiler, you might try sampling instead as it's less
>>>> prone
>>>> to this.
>>>>
>>>> Or, this sounds silly, but you can manually sample by just doing a few
>>>> thread dumps while it's running (either ctrl-\ or externally with kill -3)
>>>> and see what's at the top. If there really is a hot spot, this is a
>>>> surprisingly effective way to see it.
>>>>
>>>> For this kind of code, there is no substitute for actually reading the
>>>> bytecode and thinking carefully about where there is unnecessary casting
>>>> or
>>>> boxing.
>>>>
>>>>
>>>>
>>>> On Wed, Jan 30, 2019 at 6:55 PM Brian Craft wrote:
>>>>
>>>>> I haven't tried much. I'm getting the java
>>>>> via clj-java-decompiler.core 'decompile' macro.
>>>>>
>>>>> A long cast does drop the cast (which is really counter-intuitive:
>>>>> explicitly invoke 'long', which calls longCast, in order to avoid calling
>>>>> longCast).
>>>>>
>>>>> Amusingly this doesn't reduce the total run-time, though longCast
>>>>> drops out of the hotspot list. :-p There must be some other limiting step
>>>>> that I'm missing in the profiler.
>>>>>
>>>>> I'm calling it around 1.2M times, so hopefully that engages the jit.
>>>>>
>>>>> On Wednesday, January 30, 2019 at 3:39:41 PM UTC-8, Alex Miller wrote:
>>>>>>
>>>>>> What have you tried? And how are you getting that Java? I would
>>>>>> prefer to look at bytecode (via javap) to verify what you're saying.
>>>>>>
>>>>>> Have you tried an explicit long cast?
>>>>>>
>>>>>> (aget flat-dict (bit-and 0xff (long (aget arr j
>>>>>>
>>>>>> Are you running this hot enough for the JIT to kick in? Usually this
>>>>>> is the kind of th
Re: avoiding casts with aget & friends
The full context is large. But, for example, in this code:
(let [a 1
b (:foo {:foo 3})
c (if (< a b) a b)])
b and c are Object (if the disassembly is to be believed) which leads to
casts where c is used later. Also, the compare is calling Numbers.lt, which
is going to do a bunch of casting & dispatch.
Adding a :long hint on b, b is still Object, and the compare becomes
if (a < RT.longCast(b)) {
num = Numbers.num(a);
}
with a long cast that doesn't seem necessary. Also, c is still Object.
Casting the lookup to long, like (long (:foo {:foo 3})), b and c are now
long, but there's now a cast on the return of the lookup
Object x;
if (_thunk__0__ == (x = _thunk__0__.get(const__4))) {
x = (__thunk__0__ = __site__0__.fault(const__4)).get(const__4);
}
final long b = RT.longCast(x);
final long c = (a < b) ? a : b;
which is going to hit the RT.longCast(Object) method, and start probing for
the class so it can pick a method.
On Wednesday, January 30, 2019 at 6:58:07 PM UTC-8, Alex Miller wrote:
>
> It would really help to see a full function of code context. From that I
> could probably talk a little more about what I would expect the compiler to
> understand and how you might be able to influence it.
>
> On Wed, Jan 30, 2019 at 8:50 PM Brian Craft > wrote:
>
>> If there is unnecessary casting or boxing, how do you avoid it? hinting
>> and casting affect it, but not in ways I understand, or can predict.
>>
>> On Wednesday, January 30, 2019 at 6:06:37 PM UTC-8, Alex Miller wrote:
>>>
>>> Sometimes the insertion of profiling instrumentation magnifies the cost
>>> of things in a hot loop or provides misleading hot spot info. If you're
>>> using a tracing profiler, you might try sampling instead as it's less prone
>>> to this.
>>>
>>> Or, this sounds silly, but you can manually sample by just doing a few
>>> thread dumps while it's running (either ctrl-\ or externally with kill -3)
>>> and see what's at the top. If there really is a hot spot, this is a
>>> surprisingly effective way to see it.
>>>
>>> For this kind of code, there is no substitute for actually reading the
>>> bytecode and thinking carefully about where there is unnecessary casting or
>>> boxing.
>>>
>>>
>>>
>>> On Wed, Jan 30, 2019 at 6:55 PM Brian Craft wrote:
>>>
>>>> I haven't tried much. I'm getting the java via clj-java-decompiler.core
>>>> 'decompile' macro.
>>>>
>>>> A long cast does drop the cast (which is really counter-intuitive:
>>>> explicitly invoke 'long', which calls longCast, in order to avoid calling
>>>> longCast).
>>>>
>>>> Amusingly this doesn't reduce the total run-time, though longCast drops
>>>> out of the hotspot list. :-p There must be some other limiting step that
>>>> I'm missing in the profiler.
>>>>
>>>> I'm calling it around 1.2M times, so hopefully that engages the jit.
>>>>
>>>> On Wednesday, January 30, 2019 at 3:39:41 PM UTC-8, Alex Miller wrote:
>>>>>
>>>>> What have you tried? And how are you getting that Java? I would prefer
>>>>> to look at bytecode (via javap) to verify what you're saying.
>>>>>
>>>>> Have you tried an explicit long cast?
>>>>>
>>>>> (aget flat-dict (bit-and 0xff (long (aget arr j
>>>>>
>>>>> Are you running this hot enough for the JIT to kick in? Usually this
>>>>> is the kind of thing it's good at, but it might take 10k invocations
>>>>> before
>>>>> it does.
>>>>>
>>>>>
>>>>> On Wednesday, January 30, 2019 at 4:03:43 PM UTC-6, Brian Craft wrote:
>>>>>>
>>>>>> Profiling is showing a lot of time spent in RT.longCast, in places
>>>>>> like this:
>>>>>>
>>>>>> (aget flat-dict (bit-and 0xff (aget arr j)))
>>>>>>
>>>>>> arr is hinted as ^bytes, and flat-dict as ^objects.
>>>>>>
>>>>>> which compiles to this:
>>>>>>
>>>>>> Object code2 = RT.aget((Object[])flat_dict, RT.intCast(0xFFL &
>>>>>> RT.longCast((Object)RT.aget((byte[])arr2, RT.intCast(k)
>>>>>>
>>>>>> Is there any way to avoid that RT.longCast? There is an aget method
>>>>>> in RT.java that returns a byte, and a lo
Re: avoiding casts with aget & friends
If there is unnecessary casting or boxing, how do you avoid it? hinting and casting affect it, but not in ways I understand, or can predict. On Wednesday, January 30, 2019 at 6:06:37 PM UTC-8, Alex Miller wrote: > > Sometimes the insertion of profiling instrumentation magnifies the cost of > things in a hot loop or provides misleading hot spot info. If you're using > a tracing profiler, you might try sampling instead as it's less prone to > this. > > Or, this sounds silly, but you can manually sample by just doing a few > thread dumps while it's running (either ctrl-\ or externally with kill -3) > and see what's at the top. If there really is a hot spot, this is a > surprisingly effective way to see it. > > For this kind of code, there is no substitute for actually reading the > bytecode and thinking carefully about where there is unnecessary casting or > boxing. > > > > On Wed, Jan 30, 2019 at 6:55 PM Brian Craft > wrote: > >> I haven't tried much. I'm getting the java via clj-java-decompiler.core >> 'decompile' macro. >> >> A long cast does drop the cast (which is really counter-intuitive: >> explicitly invoke 'long', which calls longCast, in order to avoid calling >> longCast). >> >> Amusingly this doesn't reduce the total run-time, though longCast drops >> out of the hotspot list. :-p There must be some other limiting step that >> I'm missing in the profiler. >> >> I'm calling it around 1.2M times, so hopefully that engages the jit. >> >> On Wednesday, January 30, 2019 at 3:39:41 PM UTC-8, Alex Miller wrote: >>> >>> What have you tried? And how are you getting that Java? I would prefer >>> to look at bytecode (via javap) to verify what you're saying. >>> >>> Have you tried an explicit long cast? >>> >>> (aget flat-dict (bit-and 0xff (long (aget arr j >>> >>> Are you running this hot enough for the JIT to kick in? Usually this is >>> the kind of thing it's good at, but it might take 10k invocations before it >>> does. >>> >>> >>> On Wednesday, January 30, 2019 at 4:03:43 PM UTC-6, Brian Craft wrote: >>>> >>>> Profiling is showing a lot of time spent in RT.longCast, in places like >>>> this: >>>> >>>> (aget flat-dict (bit-and 0xff (aget arr j))) >>>> >>>> arr is hinted as ^bytes, and flat-dict as ^objects. >>>> >>>> which compiles to this: >>>> >>>> Object code2 = RT.aget((Object[])flat_dict, RT.intCast(0xFFL & >>>> RT.longCast((Object)RT.aget((byte[])arr2, RT.intCast(k) >>>> >>>> Is there any way to avoid that RT.longCast? There is an aget method in >>>> RT.java that returns a byte, and a longCast for byte, but I suspect the >>>> cast to Object is causing it to hit the longCast for Object, which does a >>>> bunch of reflection. >>>> >>> -- >> You received this message because you are subscribed to the Google >> Groups "Clojure" group. >> To post to this group, send email to clo...@googlegroups.com >> >> Note that posts from new members are moderated - please be patient with >> your first post. >> To unsubscribe from this group, send email to >> clojure+u...@googlegroups.com >> For more options, visit this group at >> http://groups.google.com/group/clojure?hl=en >> --- >> You received this message because you are subscribed to a topic in the >> Google Groups "Clojure" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/clojure/vXJFuOujTaw/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> clojure+u...@googlegroups.com . >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: avoiding casts with aget & friends
I haven't tried much. I'm getting the java via clj-java-decompiler.core 'decompile' macro. A long cast does drop the cast (which is really counter-intuitive: explicitly invoke 'long', which calls longCast, in order to avoid calling longCast). Amusingly this doesn't reduce the total run-time, though longCast drops out of the hotspot list. :-p There must be some other limiting step that I'm missing in the profiler. I'm calling it around 1.2M times, so hopefully that engages the jit. On Wednesday, January 30, 2019 at 3:39:41 PM UTC-8, Alex Miller wrote: > > What have you tried? And how are you getting that Java? I would prefer to > look at bytecode (via javap) to verify what you're saying. > > Have you tried an explicit long cast? > > (aget flat-dict (bit-and 0xff (long (aget arr j > > Are you running this hot enough for the JIT to kick in? Usually this is > the kind of thing it's good at, but it might take 10k invocations before it > does. > > > On Wednesday, January 30, 2019 at 4:03:43 PM UTC-6, Brian Craft wrote: >> >> Profiling is showing a lot of time spent in RT.longCast, in places like >> this: >> >> (aget flat-dict (bit-and 0xff (aget arr j))) >> >> arr is hinted as ^bytes, and flat-dict as ^objects. >> >> which compiles to this: >> >> Object code2 = RT.aget((Object[])flat_dict, RT.intCast(0xFFL & >> RT.longCast((Object)RT.aget((byte[])arr2, RT.intCast(k) >> >> Is there any way to avoid that RT.longCast? There is an aget method in >> RT.java that returns a byte, and a longCast for byte, but I suspect the >> cast to Object is causing it to hit the longCast for Object, which does a >> bunch of reflection. >> > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
avoiding casts with aget & friends
Profiling is showing a lot of time spent in RT.longCast, in places like this: (aget flat-dict (bit-and 0xff (aget arr j))) arr is hinted as ^bytes, and flat-dict as ^objects. which compiles to this: Object code2 = RT.aget((Object[])flat_dict, RT.intCast(0xFFL & RT.longCast((Object)RT.aget((byte[])arr2, RT.intCast(k) Is there any way to avoid that RT.longCast? There is an aget method in RT.java that returns a byte, and a longCast for byte, but I suspect the cast to Object is causing it to hit the longCast for Object, which does a bunch of reflection. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
partition & fold
Still trying to understand reducers & transducers. Is the difference between r/folder and r/reducer the parallelization? Anyone know what this error is about? (r/foldcat (r/map #(apply + %) (r/folder (into [] (range 1000)) (partition-all 5 ArrayIndexOutOfBoundsException 47427 java.util.ArrayList.add (ArrayList.java:459) Is this error to do with partition-all being stateful? This works as expected, with reducer instead of folder: (r/foldcat (r/map #(apply + %) (r/reducer (into [] (range 1000)) (partition-all 5 ; XXX large output Is there some way to express "map over partitions, in parallel, without creating an intermediate collection" with reducers and transducers? -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: r/fold combinef and reducef init values
hey Sean -- The ones on the reducers reference page:
https://clojure.org/reference/reducers
On Friday, January 25, 2019 at 4:05:12 PM UTC-8, Sean Corfield wrote:
> Which docs are you reading? The docstring for r/fold says this – with no
> indication of calling (reducef) with no arguments (well, unless you do not
> supply combinef – in which case reducef will be used for that, so (reducef)
> would be called to seed the reductions):
>
>
>
> "Reduces a collection using a (potentially parallel) reduce-combine
>
> strategy. The collection is partitioned into groups of approximately
>
> n (default 512), each of which is reduced with reducef (with a seed
>
> value obtained by calling (combinef) with no arguments). The results
>
> of these reductions are then reduced with combinef (default
>
> reducef). combinef must be associative, and, when called with no
>
> arguments, (combinef) must produce its identity element. These
>
> operations may be performed in parallel, but the results will
>
> preserve order."
>
>
>
> Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
> An Architect's View -- http://corfield.org/
>
> "If you're not annoying somebody, you're not really alive."
> -- Margaret Atwood
>
>
>
> *From: *Brian Craft
> *Sent: *Friday, January 25, 2019 3:36 PM
> *Subject: *r/fold combinef and reducef init values
>
>
>
> From the docs:
>
>
>
> r/fold takes a reducible collection and partitions it into groups of
> approximately n (default 512) elements. Each group is reduced using the
> reducef function. The reducef function will be called with no arguments to
> produce an identity value in each partition. The results of those
> reductions are then reduced with the combinef (defaults to reducef)
> function. When called with no arguments, (combinef) must produce its
> identity element - this will be called multiple times. Operations may be
> performed in parallel. Results will preserve order.
>
>
>
> So, this seems to say r/fold will partition the collection and reduce each
> partition using the (reducef) as the init value.
>
>
>
> Then, all these intermediate results will be reduced with combinef, using
> (combinef) as the init value.
>
>
>
> However, in test it seems (reducef) is never called, and (combinef) is
> used as the init value for calls to reducef.
>
>
>
> (defn combinef
>
> ([] {:combine :f})
>
> ([acc v] acc))
>
>
>
> (defn reducef
>
> ([] {:reduce :f})
>
> ([acc v]
>
> (println "acc" acc "v" v)
>
> v))
>
>
>
> (clojure.core.reducers/fold combinef reducef ["foo" "bar"])
>
>
>
> ; outputs:
>
> acc {:combine :f} v foo
>
> acc foo v bar
>
> "bar"
>
>
>
> The accumulator in reducef is the init value from combinef, not the init
> value from reducef.
>
>
>
> What's going on?
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+u...@googlegroups.com .
> For more options, visit https://groups.google.com/d/optout.
>
>
>
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: r/fold combinef and reducef init values
Looks like it's something that's changed over different clojure releases.
On Friday, January 25, 2019 at 3:35:58 PM UTC-8, Brian Craft wrote:
>
> From the docs:
>
> r/fold takes a reducible collection and partitions it into groups of
> approximately n (default 512) elements. Each group is reduced using the
> reducef function. The reducef function will be called with no arguments to
> produce an identity value in each partition. The results of those
> reductions are then reduced with the combinef (defaults to reducef)
> function. When called with no arguments, (combinef) must produce its
> identity element - this will be called multiple times. Operations may be
> performed in parallel. Results will preserve order.
>
> So, this seems to say r/fold will partition the collection and reduce each
> partition using the (reducef) as the init value.
>
> Then, all these intermediate results will be reduced with combinef, using
> (combinef) as the init value.
>
> However, in test it seems (reducef) is never called, and (combinef) is
> used as the init value for calls to reducef.
>
> (defn combinef
> ([] {:combine :f})
> ([acc v] acc))
>
> (defn reducef
> ([] {:reduce :f})
> ([acc v]
> (println "acc" acc "v" v)
> v))
>
> (clojure.core.reducers/fold combinef reducef ["foo" "bar"])
>
> ; outputs:
> acc {:combine :f} v foo
> acc foo v bar
> "bar"
>
> The accumulator in reducef is the init value from combinef, not the init
> value from reducef.
>
> What's going on?
>
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
r/fold combinef and reducef init values
>From the docs:
r/fold takes a reducible collection and partitions it into groups of
approximately n (default 512) elements. Each group is reduced using the
reducef function. The reducef function will be called with no arguments to
produce an identity value in each partition. The results of those
reductions are then reduced with the combinef (defaults to reducef)
function. When called with no arguments, (combinef) must produce its
identity element - this will be called multiple times. Operations may be
performed in parallel. Results will preserve order.
So, this seems to say r/fold will partition the collection and reduce each
partition using the (reducef) as the init value.
Then, all these intermediate results will be reduced with combinef, using
(combinef) as the init value.
However, in test it seems (reducef) is never called, and (combinef) is used
as the init value for calls to reducef.
(defn combinef
([] {:combine :f})
([acc v] acc))
(defn reducef
([] {:reduce :f})
([acc v]
(println "acc" acc "v" v)
v))
(clojure.core.reducers/fold combinef reducef ["foo" "bar"])
; outputs:
acc {:combine :f} v foo
acc foo v bar
"bar"
The accumulator in reducef is the init value from combinef, not the init
value from reducef.
What's going on?
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
transducer parallelism
Are there any docs on transducer parallelism? I had the impression, from various sources, that they could operate in parallel, but in doing some benchmarks over a largish collection (counting character frequencies in 1.3M strings), transduce never uses more than one thread. Is this expected? If not, how would one debug it? -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
undocumented one-argument call of reducer
The transducers doc suggests transduce works with standard reducing functions, but then transduce makes a one-argument call to the function. The code docs for transduce say the reducing function must support a one-argument call, but don't give any information about what that call should do. It could be helpful to document this requirement, especially since the examples are all written with "+", which is rather misleading since it's not a standard reducing function with respect to its arity, and there is this unusual requirement on the arity of the function. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
removing trailing zeros in floating point formatting
Probably more of a java question, but I'm finding that floating point
formats with (format) are inflating data sizes enormously with meaningless
trailing zeros, because the underlying java conversion methods don't work
as they do on other platforms.
E.g., in C
printf("%.6g %.6g %.6g %.6g\n", 0.01, 100.1, 10.0, 100.0)
gives
1e-06 100 10 1e+06
in python:
>>> "%.6g %.6g %.6g %.6g" % (0.01, 100.1, 10.0, 100.0)
'1e-06 100 10 1e+06'
while clojure will produce this:
> (format "%.6g %.6g %.6g %.6g" 0.01, 100.1, 10.0, 100.0)
"1.0e-06 100.000 10 1.0e+06"
The desired behavior is to set the max significant digits & get the minimal
decimal representation. I've been through docs for Formatter, NumberFormat,
and others, but can't find anything that both 1) supports setting
significant digits (vs. fractional digits), and 2) doesn't pad meaningless
zeros on the end.
Is there a standard solution?
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: with-open pattern
This looks like a partial solution:
https://github.com/pjstadig/scopes
perhaps inspired by this discussion:
https://dev.clojure.org/display/design/Resource+Scopes
On Friday, May 5, 2017 at 5:10:27 AM UTC-7, Herwig Hochleitner wrote:
>
> 2017-05-04 19:35 GMT+02:00 Brian Craft <craft...@gmail.com >:
>>
>> If there's only one with-open it can be reasonably simple to pass the
>> consumer into that context (though from ring it's pretty convoluted, due to
>> needing to pass control to ring first).
>>
>
> Creating streamed ring responses can be kind of tricky. In my experience,
> there are two patterns, that work well:
>
> 1) Feeding a PipedInputStream from a new thread:
>
> (let [os (PipedOutputStream.)
> is (PipedInputStream. os)]
> (future (with-open [rs ...]
> (write-to! os rs)))
> {:body is})
>
> 2) Use a ring adapter that allows some form of asynchronous response. E.g.
> with the new ring 1.6 model
>
> {:body (reify ring.core.protocols/StreamableResponseBody
> (write-body-to-stream [_ _ output-stream]
> (with-open [rs ...
> os output-stream]
> (write-to! os rs}
>
>
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: with-open pattern
It's definitely the same problem, but I don't think it helps me. This part, in particular: "If you treat this object like a sequence, it will fully consume the input stream and fully realize the decoded data in memory." I'm specifically trying to avoid realizing the full collection in memory, because it won't fit. On Thursday, May 4, 2017 at 11:22:36 AM UTC-7, Josh Tilles wrote: > > I think the “reducible streams” approach described by Paul Stadig here > <http://paul.stadig.name/2016/08/reducible-streams.html> has potential. > It might not cover all of the scenarios you’re thinking of, though. > > On Thursday, May 4, 2017 at 1:35:48 PM UTC-4, Brian Craft wrote: >> >> The with-open style is used a lot in the jdbc lib, and elsewhere. It's >> pretty simple when data is very small, as you can just evaluate the entire >> result with doall, etc. >> >> How do you deal with larger data, where you need to evaluate iteratively? >> If there's only one with-open it can be reasonably simple to pass the >> consumer into that context (though from ring it's pretty convoluted, due to >> needing to pass control to ring first). But if there are multiple with-open >> you have to nest them, with a continuation passing style, or middleware >> pattern, or something, which quickly becomes onerous as it affects all the >> code surrounding the with-open. >> >> Is there some simpler pattern? >> > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
with-open pattern
The with-open style is used a lot in the jdbc lib, and elsewhere. It's pretty simple when data is very small, as you can just evaluate the entire result with doall, etc. How do you deal with larger data, where you need to evaluate iteratively? If there's only one with-open it can be reasonably simple to pass the consumer into that context (though from ring it's pretty convoluted, due to needing to pass control to ring first). But if there are multiple with-open you have to nest them, with a continuation passing style, or middleware pattern, or something, which quickly becomes onerous as it affects all the code surrounding the with-open. Is there some simpler pattern? -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: escape characters to make a string literal
The specification seems ambiguous on this point. The definition of strings doesn't mention hex or octal escapes, but the definition of characters does. No relationship between characters and strings is specified. Characters are unicode, apparently; strings are unspecified. On Thursday, March 30, 2017 at 2:13:23 AM UTC-7, Christophe Grand wrote: > > A funny thing is that octal or hexadecimal escapes are not part of EDN > (even if most readers recognize them). > > On Wed, Mar 29, 2017 at 8:49 PM, Brian Craft <craft...@gmail.com > > wrote: > >> A funny hole in the meta-programming story. I don't think the String or >> Character classes have methods that do this. >> >> Also interesting, clojure's print methods will handle some escapes, but >> not others: >> >> cavm.core=> (println (pr-str "foo\tbar")) >> >> "foo\tbar" >> >> nil >> >> cavm.core=> (println (pr-str "foo\001bar")) >> >> "foobar" >> >> nil >> >> >> Apparently org.apache.commons.lang3.StringEscapeUtils.escapeJava is one >> solution. >> >> >> On Wednesday, March 29, 2017 at 11:02:40 AM UTC-7, Alex Miller wrote: >>> >>> Clojure leans on Java to read that literal. There is no Clojure function >>> to forcibly print it that way again, but you can probably use the Java >>> methods on String or Character to get the String representation of a >>> character somehow. >> >> -- >> You received this message because you are subscribed to the Google >> Groups "Clojure" group. >> To post to this group, send email to clo...@googlegroups.com >> >> Note that posts from new members are moderated - please be patient with >> your first post. >> To unsubscribe from this group, send email to >> clojure+u...@googlegroups.com >> For more options, visit this group at >> http://groups.google.com/group/clojure?hl=en >> --- >> You received this message because you are subscribed to the Google Groups >> "Clojure" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to clojure+u...@googlegroups.com . >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > On Clojure http://clj-me.cgrand.net/ > Clojure Programming http://clojurebook.com > Training, Consulting & Contracting http://lambdanext.eu/ > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: escape characters to make a string literal
A funny hole in the meta-programming story. I don't think the String or Character classes have methods that do this. Also interesting, clojure's print methods will handle some escapes, but not others: cavm.core=> (println (pr-str "foo\tbar")) "foo\tbar" nil cavm.core=> (println (pr-str "foo\001bar")) "foobar" nil Apparently org.apache.commons.lang3.StringEscapeUtils.escapeJava is one solution. On Wednesday, March 29, 2017 at 11:02:40 AM UTC-7, Alex Miller wrote: > > Clojure leans on Java to read that literal. There is no Clojure function > to forcibly print it that way again, but you can probably use the Java > methods on String or Character to get the String representation of a > character somehow. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
escape characters to make a string literal
Is there an easy way to display a string as a string literal that can be read by clojure? E.g. > (let [x "\001"] (what-goes-here x)) "\001" -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: instrument code only in dev, with lein
So, this seems to work pretty easily, though I'm relying on 'require' resolving references in the same order as the directories in :source-paths. For :source-paths ["src-dev", "src"], it will find src-dev/foo/instrument.clj,instead of src/foo/instrument.clj, when both exist. I can't find anything in the lein or clojure docs that specifies this behavior. Is this a safe assumption? On Friday, March 17, 2017 at 5:24:30 PM UTC-7, Matching Socks wrote: > > In a nutshell: leverage distinct classpaths. Adjust the :dev profile in > project.clj to prepend a directory other than src to :source-paths, and > likewise a directory other than resources for :resource-paths. In > development, use code or resources from the dev classpath to override > default behaviors on the uberjarred classpath. See the sample Lein > project.clj: > > https://github.com/technomancy/leiningen/blob/master/sample.project.clj > > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
instrument code only in dev, with lein
How do you instrument code, e.g. apply ring wrap-reload, wrap-stacktrace-web, etc., only in dev, not in uberjar? -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: structuring parallel code
I think java locks may be the only good answer. I can't usefully divide the vector, because the distribution of updates is uniform along the length of it. Perhaps there's a solution with queues, with multiple threads generating potential placements, and a single thread updating the vector and re-queuing any conflicting placements. On Monday, January 30, 2017 at 7:11:03 PM UTC-8, Alex Miller wrote: > > One technique is to batch locks at a coarser granularity. You've explored > both ends of the spectrum - 1 lock and N locks. You can also divide the > overall vector into any group of refs between 1 and N. > > If refs are too heavy, there are several other locking mechanisms on the > JVM. You could try Clojure atoms or Java locks. Atoms can only be used to > protect a single value so you would need a protocol for locking acquisition > to deal with that. For something like that, you'd probably end up using a > mutable data structure like a Java array. > > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: structuring parallel code
I don't think parallel reducers can handle the conflict issue: two threads making incompatible changes to the vector. On Monday, January 30, 2017 at 7:01:22 PM UTC-8, Josh Tilles wrote: > > If your goal is “to operate on large data structures in parallel” then > you’ll probably find Clojure’s reducers library > <https://clojure.org/reference/reducers> to be helpful. > > On Monday, January 30, 2017 at 9:38:01 PM UTC-5, Brian Craft wrote: >> >> ans: this scales badly. >> >> There must be a way in clojure to operate on large data structures in >> parallel, no? >> >> On Monday, January 30, 2017 at 6:03:39 PM UTC-8, Brian Craft wrote: >>> >>> Would this not scale badly? When the vector is hundreds of thousands, or >>> millions? >>> >>> On Monday, January 30, 2017 at 5:54:32 PM UTC-8, tbc++ wrote: >>>> >>>> Instead of looking at the state as a ref with a vector in it, think of >>>> it as a vector of refs. That then allows multiple refs to be modified at >>>> once without stepping on other unrelated refs. >>>> >>>> On Mon, Jan 30, 2017 at 5:26 PM, Brian Craft <craft...@gmail.com> >>>> wrote: >>>> >>>>> I'm experimenting with ref, dosync, and alter to run some code in >>>>> parallel, but so far haven't been able to figure out how to structure it >>>>> such that it runs faster, instead of slower. >>>>> >>>>> The problem looks something like this. >>>>> >>>>> The current state is a long vector. Input is a long sequence of >>>>> values. At each step some positions in the vector are populated, if >>>>> empty, >>>>> based on computations over the next value in the sequence. >>>>> >>>>> It's like placing puzzle pieces: search for positions that satisfy a >>>>> constraint, and fill them. >>>>> >>>>> Using threads, I'd like to place the next several values in parallel. >>>>> But running in parallel it's possible that two threads would find >>>>> solutions >>>>> that conflict. In this case, the latter one must continue searching. >>>>> >>>>> Is there a way to express this with clojure? The problem is that I >>>>> can't tell 'dosync' when a previous transaction invalidates the current >>>>> transaction: it just always retries. >>>>> >>>>> e.g. if each thread is doing >>>>> >>>>> >>>>> (dosync >>>>> (let [positions (find-positions next-value @state)] >>>>>(alter state (fn [s] (update-state s positions)) >>>>> >>>>> they interfere with each other constantly, because any write to >>>>> 'state' causes the other threads to retry, even if their positions are >>>>> still free. >>>>> >>>>> One really wants to reverse the order of find-positions and dosync >>>>> here, so it computes positions, then tries to commit them, checking in >>>>> the >>>>> transaction that the positions are still free, and continuing the search >>>>> if >>>>> they're not. >>>>> >>>>> I suppose there's some solution involving exceptions. Is there a >>>>> better way to think about this? >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Clojure" group. >>>>> To post to this group, send email to clo...@googlegroups.com >>>>> Note that posts from new members are moderated - please be patient >>>>> with your first post. >>>>> To unsubscribe from this group, send email to >>>>> clojure+u...@googlegroups.com >>>>> For more options, visit this group at >>>>> http://groups.google.com/group/clojure?hl=en >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Clojure" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to clojure+u...@googlegroups.com. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> >>>> >>>> -- >>>> “One of the main causes of the fall of the Roman Empire was >>>> that–lacking zero–they had no way to indicate successful termination of >>>> their C programs.” >>>> (Robert Firth) >>>> >>> -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: structuring parallel code
ans: this scales badly. There must be a way in clojure to operate on large data structures in parallel, no? On Monday, January 30, 2017 at 6:03:39 PM UTC-8, Brian Craft wrote: > > Would this not scale badly? When the vector is hundreds of thousands, or > millions? > > On Monday, January 30, 2017 at 5:54:32 PM UTC-8, tbc++ wrote: >> >> Instead of looking at the state as a ref with a vector in it, think of it >> as a vector of refs. That then allows multiple refs to be modified at once >> without stepping on other unrelated refs. >> >> On Mon, Jan 30, 2017 at 5:26 PM, Brian Craft <craft...@gmail.com> wrote: >> >>> I'm experimenting with ref, dosync, and alter to run some code in >>> parallel, but so far haven't been able to figure out how to structure it >>> such that it runs faster, instead of slower. >>> >>> The problem looks something like this. >>> >>> The current state is a long vector. Input is a long sequence of values. >>> At each step some positions in the vector are populated, if empty, based on >>> computations over the next value in the sequence. >>> >>> It's like placing puzzle pieces: search for positions that satisfy a >>> constraint, and fill them. >>> >>> Using threads, I'd like to place the next several values in parallel. >>> But running in parallel it's possible that two threads would find solutions >>> that conflict. In this case, the latter one must continue searching. >>> >>> Is there a way to express this with clojure? The problem is that I can't >>> tell 'dosync' when a previous transaction invalidates the current >>> transaction: it just always retries. >>> >>> e.g. if each thread is doing >>> >>> >>> (dosync >>> (let [positions (find-positions next-value @state)] >>>(alter state (fn [s] (update-state s positions)) >>> >>> they interfere with each other constantly, because any write to 'state' >>> causes the other threads to retry, even if their positions are still free. >>> >>> One really wants to reverse the order of find-positions and dosync here, >>> so it computes positions, then tries to commit them, checking in the >>> transaction that the positions are still free, and continuing the search if >>> they're not. >>> >>> I suppose there's some solution involving exceptions. Is there a better >>> way to think about this? >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Clojure" group. >>> To post to this group, send email to clo...@googlegroups.com >>> Note that posts from new members are moderated - please be patient with >>> your first post. >>> To unsubscribe from this group, send email to >>> clojure+u...@googlegroups.com >>> For more options, visit this group at >>> http://groups.google.com/group/clojure?hl=en >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Clojure" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to clojure+u...@googlegroups.com. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> >> -- >> “One of the main causes of the fall of the Roman Empire was that–lacking >> zero–they had no way to indicate successful termination of their C >> programs.” >> (Robert Firth) >> > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: structuring parallel code
Would this not scale badly? When the vector is hundreds of thousands, or millions? On Monday, January 30, 2017 at 5:54:32 PM UTC-8, tbc++ wrote: > > Instead of looking at the state as a ref with a vector in it, think of it > as a vector of refs. That then allows multiple refs to be modified at once > without stepping on other unrelated refs. > > On Mon, Jan 30, 2017 at 5:26 PM, Brian Craft <craft...@gmail.com > > wrote: > >> I'm experimenting with ref, dosync, and alter to run some code in >> parallel, but so far haven't been able to figure out how to structure it >> such that it runs faster, instead of slower. >> >> The problem looks something like this. >> >> The current state is a long vector. Input is a long sequence of values. >> At each step some positions in the vector are populated, if empty, based on >> computations over the next value in the sequence. >> >> It's like placing puzzle pieces: search for positions that satisfy a >> constraint, and fill them. >> >> Using threads, I'd like to place the next several values in parallel. But >> running in parallel it's possible that two threads would find solutions >> that conflict. In this case, the latter one must continue searching. >> >> Is there a way to express this with clojure? The problem is that I can't >> tell 'dosync' when a previous transaction invalidates the current >> transaction: it just always retries. >> >> e.g. if each thread is doing >> >> >> (dosync >> (let [positions (find-positions next-value @state)] >>(alter state (fn [s] (update-state s positions)) >> >> they interfere with each other constantly, because any write to 'state' >> causes the other threads to retry, even if their positions are still free. >> >> One really wants to reverse the order of find-positions and dosync here, >> so it computes positions, then tries to commit them, checking in the >> transaction that the positions are still free, and continuing the search if >> they're not. >> >> I suppose there's some solution involving exceptions. Is there a better >> way to think about this? >> >> -- >> You received this message because you are subscribed to the Google >> Groups "Clojure" group. >> To post to this group, send email to clo...@googlegroups.com >> >> Note that posts from new members are moderated - please be patient with >> your first post. >> To unsubscribe from this group, send email to >> clojure+u...@googlegroups.com >> For more options, visit this group at >> http://groups.google.com/group/clojure?hl=en >> --- >> You received this message because you are subscribed to the Google Groups >> "Clojure" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to clojure+u...@googlegroups.com . >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > “One of the main causes of the fall of the Roman Empire was that–lacking > zero–they had no way to indicate successful termination of their C > programs.” > (Robert Firth) > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
structuring parallel code
I'm experimenting with ref, dosync, and alter to run some code in parallel, but so far haven't been able to figure out how to structure it such that it runs faster, instead of slower. The problem looks something like this. The current state is a long vector. Input is a long sequence of values. At each step some positions in the vector are populated, if empty, based on computations over the next value in the sequence. It's like placing puzzle pieces: search for positions that satisfy a constraint, and fill them. Using threads, I'd like to place the next several values in parallel. But running in parallel it's possible that two threads would find solutions that conflict. In this case, the latter one must continue searching. Is there a way to express this with clojure? The problem is that I can't tell 'dosync' when a previous transaction invalidates the current transaction: it just always retries. e.g. if each thread is doing (dosync (let [positions (find-positions next-value @state)] (alter state (fn [s] (update-state s positions)) they interfere with each other constantly, because any write to 'state' causes the other threads to retry, even if their positions are still free. One really wants to reverse the order of find-positions and dosync here, so it computes positions, then tries to commit them, checking in the transaction that the positions are still free, and continuing the search if they're not. I suppose there's some solution involving exceptions. Is there a better way to think about this? -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: finding clojure functions from the mangled names
It doesn't, really, because you can't tell how nested functions are in clojure due to all the macros. In my code it wasn't nested at all, but I see now that defmethod and core.match added nested anonymous functions. On Friday, December 9, 2016 at 4:32:51 PM UTC-8, Sean Corfield wrote: > > How big a piece of code is clojure.something100/foo ? > > > > That sort of name indicates nested anonymous functions inside foo, nested > three deep in this case. That should narrow it down quite a bit. > > > > (if you give us real names, that might help, especially if it’s in a > well-known third party library – or you can post the containing ‘foo’ > function for us to see?) > > > > Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN > An Architect's View -- http://corfield.org/ > > "If you're not annoying somebody, you're not really alive." > -- Margaret Atwood > > > > On 12/9/16, 3:00 PM, "Brian Craft" <clo...@googlegroups.com > on behalf of craft...@gmail.com > wrote: > > > > Trying to profile some code, and the stack traces look like > > > > clojure.something0 > > clojure.something1 > >clojure.something2 > > > > clojure.something100 > > foo$fn_1000$fn_1002$fn_10003.invoke() > > > > > > How can I figure out what that last function is? I can access the symbol > from the repl, but no idea how to connect it to my code. > > > > > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: finding clojure functions from the mangled names
This is in a profiler, not a repl: visualvm. If there's a way to make visualvm aware of clojure fns, I'll be very happy. On Friday, December 9, 2016 at 3:52:38 PM UTC-8, Ghadi Shayban wrote: > > The stacktrace should be pointing to the correct file & line number -- no > need to reverse engineer the mangling. If it's not for some reason, file a > bug. (It's helpful to eliminate nREPL / lein middleware in case something > is transforming the printing of traces) > If you are missing a trace when the JVM JITs your code, run with -XX:- > *OmitStackTraceInFastThrow* > > On Friday, December 9, 2016 at 6:25:24 PM UTC-5, Brian Craft wrote: >> >> Yes, but not very practical: since I don't know which one to change, this >> would be a huge rewrite of code to eliminate #() and (fn []). >> >> >> On Friday, December 9, 2016 at 3:16:06 PM UTC-8, Alex Engelberg wrote: >>> >>> If you're seeing "fn_123", it's probably coming from an anonymous >>> function. Giving those functions a name with the (fn my-name [] ...) syntax >>> will make the stack trace a little easier to decipher: >>> >>> user=> ((fn [] (/ 1 0))) >>> >>> ArithmeticException Divide by zero clojure.lang.Numbers.divide >>> (Numbers.java:158) >>> >>> user=> (pst) >>> >>> ArithmeticException Divide by zero >>> clojure.lang.Numbers.divide (Numbers.java:158) >>> clojure.lang.Numbers.divide (Numbers.java:3808) >>> user/eval20851/fn--20852 (form-init8836862334241327650.clj:1) >>> ... >>> >>> nil >>> user=> ((fn my-fn-with-a-name [] (/ 1 0)) >>> >>> ArithmeticException Divide by zero clojure.lang.Numbers.divide >>> (Numbers.java:158) >>> user=> (pst) >>> ArithmeticException Divide by zero >>> clojure.lang.Numbers.divide (Numbers.java:158) >>> clojure.lang.Numbers.divide (Numbers.java:3808) >>> user/eval20857/my-fn-with-a-name--20858 >>> (form-init8836862334241327650.clj:1) >>> ... >>> >>> nil >>> >>> On Fri, Dec 9, 2016 at 3:00 PM, Brian Craft <craft...@gmail.com> wrote: >>> >>>> Trying to profile some code, and the stack traces look like >>>> >>>> clojure.something0 >>>> clojure.something1 >>>>clojure.something2 >>>> >>>> clojure.something100 >>>> foo$fn_1000$fn_1002$fn_10003.invoke() >>>> >>>> >>>> How can I figure out what that last function is? I can access the >>>> symbol from the repl, but no idea how to connect it to my code. >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Clojure" group. >>>> To post to this group, send email to clo...@googlegroups.com >>>> Note that posts from new members are moderated - please be patient with >>>> your first post. >>>> To unsubscribe from this group, send email to >>>> clojure+u...@googlegroups.com >>>> For more options, visit this group at >>>> http://groups.google.com/group/clojure?hl=en >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "Clojure" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to clojure+u...@googlegroups.com. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: finding clojure functions from the mangled names
Well, a truly painful way is to dump the const__ fields on the class, which appear to be values in the closure, from which the function might be inferred. Like (. foo$fn__7840$fn__7846$fn__7847 const__0) (. foo$fn__7840$fn__7846$fn__7847 const__1) ... On Friday, December 9, 2016 at 3:25:24 PM UTC-8, Brian Craft wrote: > > Yes, but not very practical: since I don't know which one to change, this > would be a huge rewrite of code to eliminate #() and (fn []). > > > On Friday, December 9, 2016 at 3:16:06 PM UTC-8, Alex Engelberg wrote: >> >> If you're seeing "fn_123", it's probably coming from an anonymous >> function. Giving those functions a name with the (fn my-name [] ...) syntax >> will make the stack trace a little easier to decipher: >> >> user=> ((fn [] (/ 1 0))) >> >> ArithmeticException Divide by zero clojure.lang.Numbers.divide >> (Numbers.java:158) >> >> user=> (pst) >> >> ArithmeticException Divide by zero >> clojure.lang.Numbers.divide (Numbers.java:158) >> clojure.lang.Numbers.divide (Numbers.java:3808) >> user/eval20851/fn--20852 (form-init8836862334241327650.clj:1) >> ... >> >> nil >> user=> ((fn my-fn-with-a-name [] (/ 1 0)) >> >> ArithmeticException Divide by zero clojure.lang.Numbers.divide >> (Numbers.java:158) >> user=> (pst) >> ArithmeticException Divide by zero >> clojure.lang.Numbers.divide (Numbers.java:158) >> clojure.lang.Numbers.divide (Numbers.java:3808) >> user/eval20857/my-fn-with-a-name--20858 >> (form-init8836862334241327650.clj:1) >> ... >> >> nil >> >> On Fri, Dec 9, 2016 at 3:00 PM, Brian Craft <craft...@gmail.com> wrote: >> >>> Trying to profile some code, and the stack traces look like >>> >>> clojure.something0 >>> clojure.something1 >>>clojure.something2 >>> >>> clojure.something100 >>> foo$fn_1000$fn_1002$fn_10003.invoke() >>> >>> >>> How can I figure out what that last function is? I can access the symbol >>> from the repl, but no idea how to connect it to my code. >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Clojure" group. >>> To post to this group, send email to clo...@googlegroups.com >>> Note that posts from new members are moderated - please be patient with >>> your first post. >>> To unsubscribe from this group, send email to >>> clojure+u...@googlegroups.com >>> For more options, visit this group at >>> http://groups.google.com/group/clojure?hl=en >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Clojure" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to clojure+u...@googlegroups.com. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: finding clojure functions from the mangled names
Yes, but not very practical: since I don't know which one to change, this would be a huge rewrite of code to eliminate #() and (fn []). On Friday, December 9, 2016 at 3:16:06 PM UTC-8, Alex Engelberg wrote: > > If you're seeing "fn_123", it's probably coming from an anonymous > function. Giving those functions a name with the (fn my-name [] ...) syntax > will make the stack trace a little easier to decipher: > > user=> ((fn [] (/ 1 0))) > > ArithmeticException Divide by zero clojure.lang.Numbers.divide > (Numbers.java:158) > > user=> (pst) > > ArithmeticException Divide by zero > clojure.lang.Numbers.divide (Numbers.java:158) > clojure.lang.Numbers.divide (Numbers.java:3808) > user/eval20851/fn--20852 (form-init8836862334241327650.clj:1) > ... > > nil > user=> ((fn my-fn-with-a-name [] (/ 1 0)) > > ArithmeticException Divide by zero clojure.lang.Numbers.divide > (Numbers.java:158) > user=> (pst) > ArithmeticException Divide by zero > clojure.lang.Numbers.divide (Numbers.java:158) > clojure.lang.Numbers.divide (Numbers.java:3808) > user/eval20857/my-fn-with-a-name--20858 > (form-init8836862334241327650.clj:1) > ... > > nil > > On Fri, Dec 9, 2016 at 3:00 PM, Brian Craft <craft...@gmail.com > > wrote: > >> Trying to profile some code, and the stack traces look like >> >> clojure.something0 >> clojure.something1 >>clojure.something2 >> >> clojure.something100 >> foo$fn_1000$fn_1002$fn_10003.invoke() >> >> >> How can I figure out what that last function is? I can access the symbol >> from the repl, but no idea how to connect it to my code. >> >> -- >> You received this message because you are subscribed to the Google >> Groups "Clojure" group. >> To post to this group, send email to clo...@googlegroups.com >> >> Note that posts from new members are moderated - please be patient with >> your first post. >> To unsubscribe from this group, send email to >> clojure+u...@googlegroups.com >> For more options, visit this group at >> http://groups.google.com/group/clojure?hl=en >> --- >> You received this message because you are subscribed to the Google Groups >> "Clojure" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to clojure+u...@googlegroups.com . >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
finding clojure functions from the mangled names
Trying to profile some code, and the stack traces look like clojure.something0 clojure.something1 clojure.something2 clojure.something100 foo$fn_1000$fn_1002$fn_10003.invoke() How can I figure out what that last function is? I can access the symbol from the repl, but no idea how to connect it to my code. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
'lein trampoline test' fails on first run with 'No implementation of method'
With an empty target directory 'lein trampoline test' fails with a protocol exception, 'No implementation of method'. Running 'lein test' works fine. After running 'lein test', 'lein trampoline test' also works. Anyone know what's going on? Seems like something isn't being compiled, or loaded, but I don't know why. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: separation of concerns w/o encapsulation
http://en.wikipedia.org/wiki/Encapsulation_(object-oriented_programming)#As_information_hiding_mechanism On Wednesday, May 13, 2015 at 1:09:48 AM UTC-7, Juan A. Ruz @tangrammer wrote: Hi guys, when you talk about encapsulation do you mean using defrecords + protocols ? In this case, we are talking of choosing defrecords instead of plain functions. Maybe it would be better if we talk too about the drawbacks of this choice. For example, can we compose or extend protocol functions (I mean to extend existent component protocol functions). In some occasion, I wanted to use juxt/bolt [(formerly juxt/cylon) security component based system] in GoogleAppEngine but juxt/bolt uses http-kit and GAE dont' let you use this lib dependency. So at last I had to fork juxt/bolt to switch the http client library for plain java.net classes. So, I wonder myself which are the limits of reusability in component pattern. El viernes, 8 de mayo de 2015, 18:29:50 (UTC+2), Brian Craft escribió: Talk on the list about encapsulation usually comes back to some variation of you don't need it when you have immutable data structures. But in the long term I'm finding the problem of working w/o encapsulation is not the danger of data being mutated under you. Rather, it's the danger of all the module boundaries blurring over time, leading to the big ball of mud: a very fragile code base where everything depends on everything else. E.g. if you model your application with a plain data structure which you pass around to different modules, each concerned with a small part of that data structure, the tendency over time is for every module to become concerned with every part of that data structure. Then you have no separation, nothing is reusable, and the code is very fragile. How do you avoid this, practically? In OO it would be enforced with encapsulation, so the next developer (which might be me, six months later) trying to fix a bug, or add a feature, knows Oh, I shouldn't peek in here: this module isn't supposed to depend on that piece of data. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: separation of concerns w/o encapsulation
On Tuesday, May 12, 2015 at 4:29:22 AM UTC-7, Georgi Danov wrote: coding and designing defensively because you are concerned about your teammates is big waste of time. if this is the reality (in the enterprise it's the norm) Yes, it is the norm in the enterprise. In a decade of enterprise coding, encapsulation became a conditioned response. Anything exposed would be abused, guaranteed. It was impossible to enforce through documentation, due to the size of the organization. Requiring another team to read the docs, and respect API boundaries when committing, involved trying to persuade three layers of management that it was important enough to institute such a policy. From the VP perspective, shipping next week was always more important. Encapsulation was the only tool that worked. To some degree, it's the norm in other contexts as well. We see in popular languages w/o encapsulation that users invariably code to the internals, making it difficult or impossible to refactor libraries. http://ziade.org/2010/03/03/the-fate-of-distutils-pycon-summit-packaging-sprint-detailed-report/ When you have enough users, the cost of such breakage is quite high. Then they should have known better doesn't fly as an argument. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: separation of concerns w/o encapsulation
Thanks, everyone, this has been very helpful. On Sunday, May 10, 2015 at 8:28:37 AM UTC-7, Stuart Sierra wrote: I find it's really the same as in any other language. Certainly if you don't have any clearly-defined boundaries at all, you'll get a big ball of mud. Encapsulation is about code organization and self-discipline. Define module responsibilities and boundaries in your developer documentation. Make it clear that X is not supposed to depend on Y. Enforce those boundaries through code review and refactoring. Regularly review module definitions to make sure they match the real requirements of the system. I developed Component[1] to help with one aspect of this problem. One shared map defines the structure of the entire system, but each “module” is only exposed to the subset of the system it needs. Other approaches: With a shared map, namespaced keywords can be a hint that something is “private” to a particular module. Alternately, you could establish the convention that elements of a shared data structure should *only* be accessed via helper functions, and use public/private Vars to enforce which aspects of a data structure are meant to be “public” to other modules. –S [1]: https://github.com/stuartsierra/component On Friday, May 8, 2015 at 5:29:50 PM UTC+1, Brian Craft wrote: Talk on the list about encapsulation usually comes back to some variation of you don't need it when you have immutable data structures. But in the long term I'm finding the problem of working w/o encapsulation is not the danger of data being mutated under you. Rather, it's the danger of all the module boundaries blurring over time, leading to the big ball of mud: a very fragile code base where everything depends on everything else. E.g. if you model your application with a plain data structure which you pass around to different modules, each concerned with a small part of that data structure, the tendency over time is for every module to become concerned with every part of that data structure. Then you have no separation, nothing is reusable, and the code is very fragile. How do you avoid this, practically? In OO it would be enforced with encapsulation, so the next developer (which might be me, six months later) trying to fix a bug, or add a feature, knows Oh, I shouldn't peek in here: this module isn't supposed to depend on that piece of data. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
separation of concerns w/o encapsulation
Talk on the list about encapsulation usually comes back to some variation of you don't need it when you have immutable data structures. But in the long term I'm finding the problem of working w/o encapsulation is not the danger of data being mutated under you. Rather, it's the danger of all the module boundaries blurring over time, leading to the big ball of mud: a very fragile code base where everything depends on everything else. E.g. if you model your application with a plain data structure which you pass around to different modules, each concerned with a small part of that data structure, the tendency over time is for every module to become concerned with every part of that data structure. Then you have no separation, nothing is reusable, and the code is very fragile. How do you avoid this, practically? In OO it would be enforced with encapsulation, so the next developer (which might be me, six months later) trying to fix a bug, or add a feature, knows Oh, I shouldn't peek in here: this module isn't supposed to depend on that piece of data. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
guides for porting clojure to clojurescript
Has anyone written a guide, or best-practices doc for converting clojure libs to clojurescript? I'm thinking of things like what to do with java methods calls? Write shim objects? Convert all the method calls to protocols? I expect this has been done many times, especially for common needs, like string methods. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
case-insensitive data structures, contrib fnmap
I found an earlier thread about case-insensitive maps, which pointed to a contrib lib, fnmap. It doesn't appear to have survived. Is there another lib that does this, or a better way of doing case-insensitive data structs? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
vectorz, Unable to find implementation, uberjar, excluding source
I'm excluding source when building uberjar, due to jar size and class loader problems at run time. However I now get Unable to find implementation :vectorz errors at run time. Is there some way to work around this? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: vectorz, Unable to find implementation, uberjar, excluding source
Fixed it by adding :aot [mikera.vectorz.core] to the uberjar profile. On Wednesday, January 21, 2015 at 11:26:47 AM UTC-8, Brian Craft wrote: I'm excluding source when building uberjar, due to jar size and class loader problems at run time. However I now get Unable to find implementation :vectorz errors at run time. Is there some way to work around this? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: debugging MyClass cannot be cast to MyClass exceptions
Oh! Yes, I think that's it. I didn't realize the uberjar would include source, which clojure would attempt to re-compile. I'm not sure why that would be useful in an uberjar. I tried the lein :omit-source flag, which does indeed omit the source for my project, but still includes the source for all the dependencies, like clojure core libs. So I then get java.lang.ClassCastException: clojure.core.VecNode cannot be cast to clojure.core.VecNode and what-not. If I manually prune all the .clj from the zip file, then this exception goes away. Is there some easier way to achieve this? Either tell clojure not to recompile, or tell lein not to include source from dependencies? On Tuesday, December 9, 2014 10:08:08 PM UTC-8, Laurent PETIT wrote: Hello, I think it does for deciding whether to use the class file found on disk, or to recompile the namespace in memory from the .clj file. Does the above make sense ? Laurent Le mercredi 10 décembre 2014, Brian Craft craft...@gmail.com javascript: a écrit : This exception is related to the pack200 -m option, which alters class file timestamps. Do clojure class loaders depend on class file timestamps? On Monday, December 8, 2014 9:15:57 AM UTC-8, Brian Craft wrote: Assuming this is something to do with class loaders going wrong, how would I approach finding the code paths involved? Could I identify where the class is being loaded; set breakpoints at those places to get the stack traces? Something else? In my case it seems to be triggered by a type hint on a function argument that is a record. On Sunday, December 7, 2014 11:46:01 AM UTC-8, Brian Craft wrote: Not sure if I followed the non-interactive case. Is it just 1) deftype or defrecord in one file 2) import the class in a different file 3) AOT compile (e.g. uberjar)? On Saturday, December 6, 2014 11:07:36 PM UTC-8, Ambrose Bonnaire-Sergeant wrote: Perhaps this issue is biting you http://dev.clojure.org/ jira/browse/CLJ-979 Thanks, Ambrose On Sat, Dec 6, 2014 at 5:22 PM, Brian Craft craft...@gmail.com wrote: Yes, I know. ;) In this case it's happening with an uberjar, not with the repl. I do java -jar myapp.jar, and later, while it is processing data, get this exception. No repl involved. On Saturday, December 6, 2014 2:02:01 PM UTC-8, juan.facorro wrote: Hi Brian, This problem usually happens when working on the REPL and you redefine a record or type (derecord and deftype), but there are still some existing instances lying around, that belong to the previous definition of that same type. See this thread for more information: https://groups.go ogle.com/forum/#!msg/clojure/N2ivUM8bvB8/xgiFVtsXKnkJ Cheers, Juan On Saturday, December 6, 2014 5:55:23 PM UTC-3, Brian Craft wrote: I'm experimenting with jwrapper, and am getting runtime exceptions like this, due to some jar manipulation that it's doing. I know one of the steps is pack200, however running pack200 manually doesn't create these issues. Anyone have suggestions for debugging this? I've seen this type of error countless times in clojure, but only when reloading interactively. This is the first time I've see it when running an uberjar. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clo...@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+u...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- Laurent Petit -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr
Re: debugging MyClass cannot be cast to MyClass exceptions
This exception is related to the pack200 -m option, which alters class file timestamps. Do clojure class loaders depend on class file timestamps? On Monday, December 8, 2014 9:15:57 AM UTC-8, Brian Craft wrote: Assuming this is something to do with class loaders going wrong, how would I approach finding the code paths involved? Could I identify where the class is being loaded; set breakpoints at those places to get the stack traces? Something else? In my case it seems to be triggered by a type hint on a function argument that is a record. On Sunday, December 7, 2014 11:46:01 AM UTC-8, Brian Craft wrote: Not sure if I followed the non-interactive case. Is it just 1) deftype or defrecord in one file 2) import the class in a different file 3) AOT compile (e.g. uberjar)? On Saturday, December 6, 2014 11:07:36 PM UTC-8, Ambrose Bonnaire-Sergeant wrote: Perhaps this issue is biting you http://dev.clojure.org/jira/browse/CLJ-979 Thanks, Ambrose On Sat, Dec 6, 2014 at 5:22 PM, Brian Craft craft...@gmail.com wrote: Yes, I know. ;) In this case it's happening with an uberjar, not with the repl. I do java -jar myapp.jar, and later, while it is processing data, get this exception. No repl involved. On Saturday, December 6, 2014 2:02:01 PM UTC-8, juan.facorro wrote: Hi Brian, This problem usually happens when working on the REPL and you redefine a record or type (derecord and deftype), but there are still some existing instances lying around, that belong to the previous definition of that same type. See this thread for more information: https://groups. google.com/forum/#!msg/clojure/N2ivUM8bvB8/xgiFVtsXKnkJ Cheers, Juan On Saturday, December 6, 2014 5:55:23 PM UTC-3, Brian Craft wrote: I'm experimenting with jwrapper, and am getting runtime exceptions like this, due to some jar manipulation that it's doing. I know one of the steps is pack200, however running pack200 manually doesn't create these issues. Anyone have suggestions for debugging this? I've seen this type of error countless times in clojure, but only when reloading interactively. This is the first time I've see it when running an uberjar. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clo...@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+u...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: debugging MyClass cannot be cast to MyClass exceptions
Assuming this is something to do with class loaders going wrong, how would I approach finding the code paths involved? Could I identify where the class is being loaded; set breakpoints at those places to get the stack traces? Something else? In my case it seems to be triggered by a type hint on a function argument that is a record. On Sunday, December 7, 2014 11:46:01 AM UTC-8, Brian Craft wrote: Not sure if I followed the non-interactive case. Is it just 1) deftype or defrecord in one file 2) import the class in a different file 3) AOT compile (e.g. uberjar)? On Saturday, December 6, 2014 11:07:36 PM UTC-8, Ambrose Bonnaire-Sergeant wrote: Perhaps this issue is biting you http://dev.clojure.org/jira/browse/CLJ-979 Thanks, Ambrose On Sat, Dec 6, 2014 at 5:22 PM, Brian Craft craft...@gmail.com wrote: Yes, I know. ;) In this case it's happening with an uberjar, not with the repl. I do java -jar myapp.jar, and later, while it is processing data, get this exception. No repl involved. On Saturday, December 6, 2014 2:02:01 PM UTC-8, juan.facorro wrote: Hi Brian, This problem usually happens when working on the REPL and you redefine a record or type (derecord and deftype), but there are still some existing instances lying around, that belong to the previous definition of that same type. See this thread for more information: https://groups. google.com/forum/#!msg/clojure/N2ivUM8bvB8/xgiFVtsXKnkJ Cheers, Juan On Saturday, December 6, 2014 5:55:23 PM UTC-3, Brian Craft wrote: I'm experimenting with jwrapper, and am getting runtime exceptions like this, due to some jar manipulation that it's doing. I know one of the steps is pack200, however running pack200 manually doesn't create these issues. Anyone have suggestions for debugging this? I've seen this type of error countless times in clojure, but only when reloading interactively. This is the first time I've see it when running an uberjar. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clo...@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+u...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: debugging MyClass cannot be cast to MyClass exceptions
Not sure if I followed the non-interactive case. Is it just 1) deftype or defrecord in one file 2) import the class in a different file 3) AOT compile (e.g. uberjar)? On Saturday, December 6, 2014 11:07:36 PM UTC-8, Ambrose Bonnaire-Sergeant wrote: Perhaps this issue is biting you http://dev.clojure.org/jira/browse/CLJ-979 Thanks, Ambrose On Sat, Dec 6, 2014 at 5:22 PM, Brian Craft craft...@gmail.com javascript: wrote: Yes, I know. ;) In this case it's happening with an uberjar, not with the repl. I do java -jar myapp.jar, and later, while it is processing data, get this exception. No repl involved. On Saturday, December 6, 2014 2:02:01 PM UTC-8, juan.facorro wrote: Hi Brian, This problem usually happens when working on the REPL and you redefine a record or type (derecord and deftype), but there are still some existing instances lying around, that belong to the previous definition of that same type. See this thread for more information: https://groups. google.com/forum/#!msg/clojure/N2ivUM8bvB8/xgiFVtsXKnkJ Cheers, Juan On Saturday, December 6, 2014 5:55:23 PM UTC-3, Brian Craft wrote: I'm experimenting with jwrapper, and am getting runtime exceptions like this, due to some jar manipulation that it's doing. I know one of the steps is pack200, however running pack200 manually doesn't create these issues. Anyone have suggestions for debugging this? I've seen this type of error countless times in clojure, but only when reloading interactively. This is the first time I've see it when running an uberjar. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clo...@googlegroups.com javascript: Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+u...@googlegroups.com javascript: For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com javascript:. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
debugging MyClass cannot be cast to MyClass exceptions
I'm experimenting with jwrapper, and am getting runtime exceptions like this, due to some jar manipulation that it's doing. I know one of the steps is pack200, however running pack200 manually doesn't create these issues. Anyone have suggestions for debugging this? I've seen this type of error countless times in clojure, but only when reloading interactively. This is the first time I've see it when running an uberjar. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: perform action after events stop for some period
I'm confused on the thread issue. Don't futures run from the agent thread pool? So they don't really create a thread? Also found climatecorp's claypoole library, which has a 'future' call that works with a thread pool, further confusing me. Not sure how that's different from the agent thread pool. On Saturday, December 6, 2014 1:16:56 PM UTC-8, juan.facorro wrote: Hi Brian, I had the same requirement while building an application, it included a GUI and I wanted to perform some actions only after the user was done editing some text. I also first implemented a solution using an atom, Thread/sleep and a future. It didn't seem right though, since I was creating a bunch of threads all the time, so I tried an approach with the tools core.async provides and the result was the following: (defn timeout-channel Creates a channel and a go block that takes from it. The go block keeps an internal status with two possible values, `:wait` and `:receive`. In ':wait' status, execution is blocked until there's a value available in the channel, it then enters the ':receive' status, until the timeout wins. Returns the channel where events need to be pushed. [timeout-ms f] (let [c (async/chan)] (async/go-loop [status :wait args nil] (condp = status :wait (recur :receive (async/! c)) :receive (let [[_ ch] (async/alts! [c (async/timeout timeout-ms)])] (if (= ch c) (recur :receive args) (do (async/thread (if (sequential? args) (apply f args) (f args))) (recur :wait nil)) c)) Signalling the go block to terminate when the event source is no longer available is missing, but that simple enough to implement. Maybe this is something you can use as a starter point. Hope it helps, Juan On Monday, December 1, 2014 6:37:56 PM UTC-3, Brian Craft wrote: I have need to perform an action when a series of events is quiet for some period. That is, if one event arrives an action is queued to execute after some timeout. If a second event arrives the timeout is reset, and so-forth. The following code seems to work, however I'm wondering if calling 'future' from 'swap!' is a bad idea (side effecting), and if there's a better way. (defn queue-with-delay [period func] (let [f (atom nil)] (fn [] (when @f (future-cancel @f)) (swap! f (fn [_] (future (Thread/sleep period) (func))) Use like (def event (queue-with-delay 2000 #(println running))) (event) (event) (event) ; pause 2 sec running -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: debugging MyClass cannot be cast to MyClass exceptions
Yes, I know. ;) In this case it's happening with an uberjar, not with the repl. I do java -jar myapp.jar, and later, while it is processing data, get this exception. No repl involved. On Saturday, December 6, 2014 2:02:01 PM UTC-8, juan.facorro wrote: Hi Brian, This problem usually happens when working on the REPL and you redefine a record or type (derecord and deftype), but there are still some existing instances lying around, that belong to the previous definition of that same type. See this thread for more information: https://groups.google.com/forum/#!msg/clojure/N2ivUM8bvB8/xgiFVtsXKnkJ Cheers, Juan On Saturday, December 6, 2014 5:55:23 PM UTC-3, Brian Craft wrote: I'm experimenting with jwrapper, and am getting runtime exceptions like this, due to some jar manipulation that it's doing. I know one of the steps is pack200, however running pack200 manually doesn't create these issues. Anyone have suggestions for debugging this? I've seen this type of error countless times in clojure, but only when reloading interactively. This is the first time I've see it when running an uberjar. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
pack200 unknown attributes
Experimenting with pack200, I get very many warnings like so: Dec 04, 2014 5:11:31 AM com.sun.java.util.jar.pack.Utils$Pack200Logger warning WARNING: Passing class file uncompressed due to unrecognized attribute: clojure/core$long.class Without having any real understanding of what it's doing, it looks like something about the clojure code is defeating compression? Anyone have a clue? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: perform action after events stop for some period
The nested send-off call doesn't happen on the same thread (it's in a future). Seems like that would be the same as if an unrelated thread called send-off while the outer send-off was running. It does seem like a single-thread solution would be better, not creating so many futures. Polling seems pretty crude, but I don't see another way of doing it with clojure abstractions. Maybe a pure java solution. On Tuesday, December 2, 2014 3:47:58 AM UTC-8, Gary Verhaegen wrote: In the general case, side effects within the swap! function are a bad idea because of the optimistic locking. In your first code snippet, if there is any contention on the atom (and maybe in your app you know there is none because it's only ever accesses by the same single thread), you run the risk of having orphaned futures. As far as I know there should be no such problem with the agent version. I'm not really sure about the nesting of send-off calls though; that might be the source of your stack overflow. I seem to remember that this was not supported up until 1.4 or 1.5; not sure what the current semantics is. Depending on how many different event types you're watching for (and how many differet actions you need to take), it might be worth having a single thread managing the queue. Somethin along the line of having a single atom containing a priority queue (or a sorted map?) with, for each event type, the last time the event was observed. At some interval, that event thread could check the queue and run the required handlers based on the current time. When an event arrives, it resets the time associated to its type in the queue. Whether this is better will depend on your usage pattern. I would just like to point out that creating a future has some non trivial overhead as it also creates a thread (at least, the last time I checked, futures where not created out of a limited thread pool). On Tuesday, 2 December 2014, Erik Price er...@zensight.co javascript: wrote: Coincidentally, we recently wrote code to do something very similar. The following function will invoke f after period milliseconds, unless a value is sent on events-ch, in which case the timeout is reset (and starts counting down again): (defn invoke-after-uninterrupted-delay ([period events-ch f] (invoke-after-uninterrupted-delay period events-ch f [])) ([period events-ch f args] (async/go-loop [] (let [[_ p] (async/alts! [(async/timeout period) events-ch])] (if (= p events-ch) (recur) (apply f args)) e On Mon, Dec 1, 2014 at 6:50 PM, Brian Craft craft.br...@gmail.com wrote: That version has the unfortunate behavior that (func) can be interrupted if (event) is called while it is running. Here's another version using an agent: (defn queue-with-delay2 [period func] (let [q (agent nil)] (fn [] (send-off q (fn [t] (when t (future-cancel t)) (future (Thread/sleep period) (send-off q (fn [_] (func) nil Running with a sleep to see that (func) is not canceled by subsequence (event) calls: (def event (queue-with-delay2 2000 #(do (println running) (Thread/sleep 2000) (println ending Oddly, if calling (event) between running and ending messages, the repl will stack-overflow on the return value. No idea what that's about. But, running like this is fine: (do (event) nil) On Monday, December 1, 2014 1:37:56 PM UTC-8, Brian Craft wrote: I have need to perform an action when a series of events is quiet for some period. That is, if one event arrives an action is queued to execute after some timeout. If a second event arrives the timeout is reset, and so-forth. The following code seems to work, however I'm wondering if calling 'future' from 'swap!' is a bad idea (side effecting), and if there's a better way. (defn queue-with-delay [period func] (let [f (atom nil)] (fn [] (when @f (future-cancel @f)) (swap! f (fn [_] (future (Thread/sleep period) (func))) Use like (def event (queue-with-delay 2000 #(println running))) (event) (event) (event) ; pause 2 sec running -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received
Re: perform action after events stop for some period
Cool. I haven't used core.async before, and am a bit reluctant to pull in another dependency just for this. But maybe it's the right solution. On Tuesday, December 2, 2014 10:03:54 AM UTC-8, Erik Price wrote: On Tue, Dec 2, 2014 at 12:22 PM, Brian Craft craft...@gmail.com javascript: wrote: It does seem like a single-thread solution would be better, not creating so many futures. Polling seems pretty crude, but I don't see another way of doing it with clojure abstractions. Maybe a pure java solution. FWIW, the core.async-based solution satisfies both the criteria of being single-threaded and non-polling. e -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
perform action after events stop for some period
I have need to perform an action when a series of events is quiet for some period. That is, if one event arrives an action is queued to execute after some timeout. If a second event arrives the timeout is reset, and so-forth. The following code seems to work, however I'm wondering if calling 'future' from 'swap!' is a bad idea (side effecting), and if there's a better way. (defn queue-with-delay [period func] (let [f (atom nil)] (fn [] (when @f (future-cancel @f)) (swap! f (fn [_] (future (Thread/sleep period) (func))) Use like (def event (queue-with-delay 2000 #(println running))) (event) (event) (event) ; pause 2 sec running -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: perform action after events stop for some period
That version has the unfortunate behavior that (func) can be interrupted if (event) is called while it is running. Here's another version using an agent: (defn queue-with-delay2 [period func] (let [q (agent nil)] (fn [] (send-off q (fn [t] (when t (future-cancel t)) (future (Thread/sleep period) (send-off q (fn [_] (func) nil Running with a sleep to see that (func) is not canceled by subsequence (event) calls: (def event (queue-with-delay2 2000 #(do (println running) (Thread/sleep 2000) (println ending Oddly, if calling (event) between running and ending messages, the repl will stack-overflow on the return value. No idea what that's about. But, running like this is fine: (do (event) nil) On Monday, December 1, 2014 1:37:56 PM UTC-8, Brian Craft wrote: I have need to perform an action when a series of events is quiet for some period. That is, if one event arrives an action is queued to execute after some timeout. If a second event arrives the timeout is reset, and so-forth. The following code seems to work, however I'm wondering if calling 'future' from 'swap!' is a bad idea (side effecting), and if there's a better way. (defn queue-with-delay [period func] (let [f (atom nil)] (fn [] (when @f (future-cancel @f)) (swap! f (fn [_] (future (Thread/sleep period) (func))) Use like (def event (queue-with-delay 2000 #(println running))) (event) (event) (event) ; pause 2 sec running -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
ring jetty adaptor: any way to abort on unrecoverable error?
If jetty fails to bind a port because it's in use an exception is thrown, but the app keeps running. A bunch of jetty threads are created in spite of having no port to listen on, and they burn cpu. How can I have it exit on unrecoverable errors? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
seque examples
Anyone have examples of when how to use seque? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: map function generator
juxt?
user= (map (juxt zero? even?) (range 5))
([true true] [false false] [false true] [false false] [false true])
user= (map (apply juxt [zero? even?]) (range 5))
([true true] [false false] [false true] [false false] [false true])
On Monday, November 17, 2014 8:25:07 PM UTC-8, Andy L wrote:
Hi,
This is another puzzle/exercise based on a very practical need. I could
not find a built in function, hoping something like colfn already exists.
Otherwise I wonder about an idiomatic solution. This is self-explanatory
code:
user= (require '[clojure.algo.generic.functor :as fu])
user= (require '[me.raynes.fs :as fs])
user= (defn colfn[col] (fn [a] (fu/fmap #(% a) col)))
user= (map (colfn [fs/directory?,identity]) (filter fs/directory?(set
(fs/list-dir .
([true src] [true target] [true .git])
user= (map (colfn {:is-dir fs/directory?, :dir identity}) (filter
fs/directory?(set (fs/list-dir .
({:is-dir true, :dir src} {:is-dir true, :dir target} {:is-dir true,
:dir .git})
My question is, if something like colfn already exists? The idea is to
generate a function of a sequence (vector, list, map) of functions which
would used in e.g. map.
Best,
Andy
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: test.check slow shrinking
Interesting that you don't see a performance problem. What version did you
try? I'm using 0.5.9.
I just re-ran this example to make sure I wasn't imagining it. On the 11th
run, it wedged, not returning, and burning a lot of cpu, presumably trying
to shrink.
It's a larger problem with the real test cases. One test ran for about
eight hours yesterday before I killed it. Hard to tell what it's doing, and
not sure what sort of times I should expect for shrinking larger test cases.
On Wednesday, November 12, 2014 9:05:40 AM UTC-8, Lucas Bradstreet wrote:
I've also had some tricky shrinking type issues with recursive generators
using bind. I had a play with your generators, using such-that to reduce
the row/column name length and also preventing some generator shrinking by
using no-shrink, but I didn't have much luck improving the resulting
shrinks (though the shrinks did finish in a timely fashion).
You might have more luck if you generated an m x n matrix size
independently of the matrix itself and feed the size into the matrix
generator and the samples and probes generators. This may allow the matrix
size to be shrunk down more easily, while also shrinking the samples and
probes vectors without them having to be regenerated for ever new matrix
shrink (i.e. where the matrix size is maintained but the gen/int values
within it are shrunk).
However, this is all guess work and conjecture, as I don't understand the
shrinking algorithm well enough, and I haven't tried my suggestion. Also,
modifying your generators in this way may make your model less general if
you wish to use it in other ways later.
Cheers
Lucas
On 12 Nov 2014, at 02:43, Brian Craft craft...@gmail.com javascript:
wrote:
Using test.check, I'm finding the shrinking to be very, very slow. Running
a hundred cases takes a few seconds, unless it hits an error, in which case
it takes 40-60 minutes to shrink, and the shrinking is not very effective
(resulting test case is much larger than necessary). Sometimes the
shrinking is much faster. It behaves a bit like it's occasionally getting
into a pathological state, or a difficult shrinking scenario.
Are there any docs on generators or the shrinking algorithm that would
help build tests that shrink more effectively?
The problematic generator builds a randomly-sized matrix of integers, with
randomly assigned names for the rows and columns. The failure case is when
either a column or row name is repeated. I expect the slow shrinking has
something to do with it being rare for the generator to emit the same name
twice.
; Generator of randomly sized matrices of random numbers.
(def gen-matrix
(gen/bind
gen/s-pos-int
(fn [x] (gen/bind
gen/s-pos-int
(fn [y] (gen/vector (gen/vector gen/int x) y))
; Generator of matrix with probe and sample ids.
(def gen-tsv
(gen/bind
gen-matrix
(fn [m]
(gen/hash-map
:probes (gen/vector
(gen/such-that not-empty gen/string-alpha-numeric)
(count m))
:samples (gen/vector
(gen/such-that not-empty gen/string-alpha-numeric)
(count (first m)))
:matrix (gen/return m)
Shrinking will result in a case like
{:matrix [[1 4 -3] [-4 -3 -5] [-5 2 3] [4 -5 -5] [1 -2 3] [1 4 1]],
:samples [0 0 0], :probes [0 0 0 0 0 0]}
where :samples [0 0] :probes[0] would do.
The following test will exhibit the behavior, sometimes succeeding,
sometimes failing quickly, sometimes shrinking for a very long time:
(tc/quick-check 100 (prop/for-all [tsv gen-tsv] (= (count (set (:probes
tsv))) (count (:probes tsv)
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clo...@googlegroups.com javascript:
Note that posts from new members are moderated - please be patient with
your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com javascript:
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an
email to clojure+u...@googlegroups.com javascript:.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe
Re: test.check slow shrinking
I tried your idea of generating the size first, then passing it to the
matrix vector generators. This does seem to work better. The shrunk cases
that return are actually worse, but so far it hasn't wedged itself, which
is a great improvement. They all return within a few seconds. I don't yet
understand the shrinking well enough to know why it's better.
Example shrunk state:
:samples [0 0 0 0 0 0 0 0 0], :probes [0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0]
(def gen-matrix-size
(gen/tuple gen/s-pos-int gen/s-pos-int))
(def gen-mostly-ints
(gen/frequency [[9 gen/int] [1 (gen/return Double/NaN)]]))
(defn gen-matrix2 [x y]
(gen/vector (gen/vector gen-mostly-ints x) y))
(defn gen-names [n]
(gen/vector
(gen/such-that not-empty gen/string-alpha-numeric)
n))
(def gen-tsv2
(gen/bind
gen-matrix-size
(fn [[x y]]
(gen/hash-map
:probes (gen-names x)
:samples (gen-names y)
:matrix (gen-matrix2 x y)
On Wednesday, November 12, 2014 12:47:37 PM UTC-8, Lucas Bradstreet wrote:
I'm pretty sure I did encounter the performance problem you're talking
about, but I killed it and re-ran until I hit cases that shrink quickly.
I'm afraid I'm not much help with those, although I agree that the bad
shrinking is probably related to the performance issues.
On 13 Nov 2014, at 04:22, Brian Craft craft...@gmail.com javascript:
wrote:
Interesting that you don't see a performance problem. What version did you
try? I'm using 0.5.9.
I just re-ran this example to make sure I wasn't imagining it. On the 11th
run, it wedged, not returning, and burning a lot of cpu, presumably trying
to shrink.
It's a larger problem with the real test cases. One test ran for about
eight hours yesterday before I killed it. Hard to tell what it's doing, and
not sure what sort of times I should expect for shrinking larger test cases.
On Wednesday, November 12, 2014 9:05:40 AM UTC-8, Lucas Bradstreet wrote:
I've also had some tricky shrinking type issues with recursive generators
using bind. I had a play with your generators, using such-that to reduce
the row/column name length and also preventing some generator shrinking by
using no-shrink, but I didn't have much luck improving the resulting
shrinks (though the shrinks did finish in a timely fashion).
You might have more luck if you generated an m x n matrix size
independently of the matrix itself and feed the size into the matrix
generator and the samples and probes generators. This may allow the matrix
size to be shrunk down more easily, while also shrinking the samples and
probes vectors without them having to be regenerated for ever new matrix
shrink (i.e. where the matrix size is maintained but the gen/int values
within it are shrunk).
However, this is all guess work and conjecture, as I don't understand the
shrinking algorithm well enough, and I haven't tried my suggestion. Also,
modifying your generators in this way may make your model less general if
you wish to use it in other ways later.
Cheers
Lucas
On 12 Nov 2014, at 02:43, Brian Craft craft...@gmail.com wrote:
Using test.check, I'm finding the shrinking to be very, very slow.
Running a hundred cases takes a few seconds, unless it hits an error, in
which case it takes 40-60 minutes to shrink, and the shrinking is not very
effective (resulting test case is much larger than necessary). Sometimes
the shrinking is much faster. It behaves a bit like it's occasionally
getting into a pathological state, or a difficult shrinking scenario.
Are there any docs on generators or the shrinking algorithm that would
help build tests that shrink more effectively?
The problematic generator builds a randomly-sized matrix of integers,
with randomly assigned names for the rows and columns. The failure case is
when either a column or row name is repeated. I expect the slow shrinking
has something to do with it being rare for the generator to emit the same
name twice.
; Generator of randomly sized matrices of random numbers.
(def gen-matrix
(gen/bind
gen/s-pos-int
(fn [x] (gen/bind
gen/s-pos-int
(fn [y] (gen/vector (gen/vector gen/int x) y))
; Generator of matrix with probe and sample ids.
(def gen-tsv
(gen/bind
gen-matrix
(fn [m]
(gen/hash-map
:probes (gen/vector
(gen/such-that not-empty gen/string-alpha-numeric)
(count m))
:samples (gen/vector
(gen/such-that not-empty gen/string-alpha-numeric)
(count (first m)))
:matrix (gen/return m)
Shrinking will result in a case like
{:matrix [[1 4 -3] [-4 -3 -5] [-5 2 3] [4 -5 -5] [1 -2 3] [1 4 1]],
:samples [0 0 0], :probes [0 0 0 0 0 0]}
where :samples [0 0] :probes[0] would do
test.check slow shrinking
Using test.check, I'm finding the shrinking to be very, very slow. Running
a hundred cases takes a few seconds, unless it hits an error, in which case
it takes 40-60 minutes to shrink, and the shrinking is not very effective
(resulting test case is much larger than necessary). Sometimes the
shrinking is much faster. It behaves a bit like it's occasionally getting
into a pathological state, or a difficult shrinking scenario.
Are there any docs on generators or the shrinking algorithm that would help
build tests that shrink more effectively?
The problematic generator builds a randomly-sized matrix of integers, with
randomly assigned names for the rows and columns. The failure case is when
either a column or row name is repeated. I expect the slow shrinking has
something to do with it being rare for the generator to emit the same name
twice.
; Generator of randomly sized matrices of random numbers.
(def gen-matrix
(gen/bind
gen/s-pos-int
(fn [x] (gen/bind
gen/s-pos-int
(fn [y] (gen/vector (gen/vector gen/int x) y))
; Generator of matrix with probe and sample ids.
(def gen-tsv
(gen/bind
gen-matrix
(fn [m]
(gen/hash-map
:probes (gen/vector
(gen/such-that not-empty gen/string-alpha-numeric)
(count m))
:samples (gen/vector
(gen/such-that not-empty gen/string-alpha-numeric)
(count (first m)))
:matrix (gen/return m)
Shrinking will result in a case like
{:matrix [[1 4 -3] [-4 -3 -5] [-5 2 3] [4 -5 -5] [1 -2 3] [1 4 1]],
:samples [0 0 0], :probes [0 0 0 0 0 0]}
where :samples [0 0] :probes[0] would do.
The following test will exhibit the behavior, sometimes succeeding,
sometimes failing quickly, sometimes shrinking for a very long time:
(tc/quick-check 100 (prop/for-all [tsv gen-tsv] (= (count (set (:probes
tsv))) (count (:probes tsv)
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: extending custom data types to clojure core abstractions
Nice, thanks! On Friday, October 31, 2014 9:59:26 AM UTC-7, Daniel wrote: Brian, this may help: https://github.com/ztellman/collection-check On Tuesday, October 28, 2014 2:24:48 PM UTC-5, Brian Craft wrote: Following up on the thread about the massive overhead of String, I tried writing a string collection type that stores strings as bytes, converting to String on-demand. It seems to work. Memory footprint and performance are good for the application. The hard part was trying to track down the correct interfaces and invocations. I note that Clojure Programming makes the same observation in the section about clojure abstractions: such things are largely undocumented. I guess this situation hasn't improved? I had to proceed mostly by experimentation, and am still unclear on, for example, why I needed to use an interop call in some places (like cons), but should not in others. Would be happy for any feedback on this attempt: (deftype StringVec [pv] clojure.lang.IPersistentVector (seq [self] (map #(String. ^bytes %) pv)) (nth [self i] (String. ^bytes (.nth ^clojure.lang.IPersistentVector pv i))) (nth [self i notfound] (String. ^bytes (.nth ^clojure.lang.IPersistentVector pv i (.getBytes ^String notfound clojure.lang.ILookup (valAt [self i] (when-let [res (.valAt ^clojure.lang.IPersistentVector pv i)] (String. ^bytes res))) (valAt [self i notfound] (String. ^bytes (.valAt ^clojure.lang.IPersistentVector pv i (.getBytes ^String notfound clojure.lang.ISeq (first [self] (String. ^bytes (first pv))) (next [self] (-StringVec (next pv))) (more [self] (-StringVec (rest pv))) (cons [self s] (-StringVec (.cons ^clojure.lang.IPersistentVector pv (.getBytes ^String s (count [self] (count pv)) Object (toString [self] (str (into [] self (defn stringvec [coll] (into (-StringVec []) coll)) (defmethod print-method StringVec [v, ^java.io.Writer w] (.write w (.toString ^StringVec v))) Speak of cons, I gather ISeq cons is unrelated to cons, the function, but rather is required for conj? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: io/writer + map/recur
Your first implementation probably failed because map is lazy, so none of the '.write' calls happened during the with-open block. Try doseq. On Friday, October 31, 2014 1:08:01 PM UTC-7, Sam Raker wrote: I'm writing some stuff to interact with the Twitter API. I want to be able to write tweets (as JSON) to a file, so I can, e.g., test things without connecting to the API. I've proxied the LinkedBlockingQueue that Twitter's HBC library uses to use an agent, so ideally I want to be able to write the contents of the agent AND the LBQ. Here's what I have right now: (defmulti write-tweets (fn [q f] (class q))) (defmethod write-tweets clojure.lang.Agent [a f] (with-open [w (clojure.java.io/writer f :append true)] (.write w (apply str (interpose \n @a) (defmethod write-tweets java.util.concurrent.LinkedBlockingQueue [lbq f] (loop [res (.take lbq)] (if res (recur (with-open [w (clojure.java.io/writer f :append true)] (.write w (str (generate-string res) \n))) My first implementation of this used `(map #(.write w %) @a)` and had the `recur` within the `with-open` block. Unfortunately, at least with the agent part, I ran into an error about the file being closed when I tried to write to it. I assumed `with-open` kept the file open within the block, but maybe I'm missing something? I'm worried about the performance of either creating a potentially super-huge string in memory for the agent method (twitter returns pretty sizable JSON blobs) or repeatedly opening/closing a file for the LBQ method (I realize I could collapse this into one problem by taking everything out of the LBQ and putting it into an agent, but that's not really a solution...) Does `writer` auto-close the file after it's done? Is there some better way of handling this kind of situation? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
memory leak, vector vs list vs chunking
I have a vector of values which compute large result sets, similar to a vector of lazy seqs. It's a vector because it has to be sorted before the values are evaluated, in order. When evaluating each value in order of the vector, it's important that they are garbage collected, because two won't fit in memory. Here's an equivalent example: (let [bigval (fn [] (map #(.getBytes %) (repeat (* 1000 1000) blah)))] (map #(do (println (count %)) (Thread/sleep (* 10 1000))) [(bigval) (bigval)])) If you watch this with jmap -histo:live, or whatever, both elements end up in memory at the same time (2 million byte arrays). But this is some property of the vector, or chunking, or something. If I change the collection to this: (into '() [(bigval) (bigval)]) the first bigval is garbage collected before the second is computed. Also, if I use Stuart's unchunk from this post http://stackoverflow.com/questions/3407876/how-do-i-avoid-clojures-chunking-behavior-for-lazy-seqs-that-i-want-to-short-ci (unchunk [(bigval) (bigval)]) the first bigval is garbage collected before the second is computed. Wondering if someone can explain what's happening in each example. With the first, form [a b], the 'map' call returns a chunked sequence. The 'count' call realizes the first sequence, but it can't be garbage collected because of a reference in the chunk? With the second form, (into '() [a b]), the 'into' eagerly builds a persistent list. The 'map' call does not chunk the list, because (seq some-list) returns a list, not a chunked sequence. Not sure why lists are different this way. With the third form, (unchunk [a b]), unchunk returns a lazy seq. The 'map' call does not chunk the seq. Not sure what's happening here. (seq (unchunk [1 2])) is a Cons, but (seq (range 5)) is a ChunkedCons. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
extending custom data types to clojure core abstractions
Following up on the thread about the massive overhead of String, I tried writing a string collection type that stores strings as bytes, converting to String on-demand. It seems to work. Memory footprint and performance are good for the application. The hard part was trying to track down the correct interfaces and invocations. I note that Clojure Programming makes the same observation in the section about clojure abstractions: such things are largely undocumented. I guess this situation hasn't improved? I had to proceed mostly by experimentation, and am still unclear on, for example, why I needed to use an interop call in some places (like cons), but should not in others. Would be happy for any feedback on this attempt: (deftype StringVec [pv] clojure.lang.IPersistentVector (seq [self] (map #(String. ^bytes %) pv)) (nth [self i] (String. ^bytes (.nth ^clojure.lang.IPersistentVector pv i))) (nth [self i notfound] (String. ^bytes (.nth ^clojure.lang.IPersistentVector pv i (.getBytes ^String notfound clojure.lang.ILookup (valAt [self i] (when-let [res (.valAt ^clojure.lang.IPersistentVector pv i)] (String. ^bytes res))) (valAt [self i notfound] (String. ^bytes (.valAt ^clojure.lang.IPersistentVector pv i (.getBytes ^String notfound clojure.lang.ISeq (first [self] (String. ^bytes (first pv))) (next [self] (-StringVec (next pv))) (more [self] (-StringVec (rest pv))) (cons [self s] (-StringVec (.cons ^clojure.lang.IPersistentVector pv (.getBytes ^String s (count [self] (count pv)) Object (toString [self] (str (into [] self (defn stringvec [coll] (into (-StringVec []) coll)) (defmethod print-method StringVec [v, ^java.io.Writer w] (.write w (.toString ^StringVec v))) Speak of cons, I gather ISeq cons is unrelated to cons, the function, but rather is required for conj? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: extending custom data types to clojure core abstractions
Hm, I just tested it with a 160M file, split into 2 million strings. The memory footprint was substantially worse than String. With String I get about a 3.2X increase in memory footprint for this file, but with pjstadig.utf8 I get about 5X. Also, it's slower by orders of magnitude. On Tuesday, October 28, 2014 12:49:53 PM UTC-7, Andy Fingerhut wrote: Sorry, no feedback on your attempt, but a note that you may want to check out Paul Stadig's utf8 library to see if it serves your purpose. I believe it should store text that fits within the ASCII subset into 1 byte of memory per character, only using 2, 3, or 4 bytes for other Unicode characters, depending on the code point. https://github.com/pjstadig/utf8 Andy On Tue, Oct 28, 2014 at 12:24 PM, Brian Craft craft...@gmail.com javascript: wrote: Following up on the thread about the massive overhead of String, I tried writing a string collection type that stores strings as bytes, converting to String on-demand. It seems to work. Memory footprint and performance are good for the application. The hard part was trying to track down the correct interfaces and invocations. I note that Clojure Programming makes the same observation in the section about clojure abstractions: such things are largely undocumented. I guess this situation hasn't improved? I had to proceed mostly by experimentation, and am still unclear on, for example, why I needed to use an interop call in some places (like cons), but should not in others. Would be happy for any feedback on this attempt: (deftype StringVec [pv] clojure.lang.IPersistentVector (seq [self] (map #(String. ^bytes %) pv)) (nth [self i] (String. ^bytes (.nth ^clojure.lang.IPersistentVector pv i))) (nth [self i notfound] (String. ^bytes (.nth ^clojure.lang.IPersistentVector pv i (.getBytes ^String notfound clojure.lang.ILookup (valAt [self i] (when-let [res (.valAt ^clojure.lang.IPersistentVector pv i)] (String. ^bytes res))) (valAt [self i notfound] (String. ^bytes (.valAt ^clojure.lang.IPersistentVector pv i (.getBytes ^String notfound clojure.lang.ISeq (first [self] (String. ^bytes (first pv))) (next [self] (-StringVec (next pv))) (more [self] (-StringVec (rest pv))) (cons [self s] (-StringVec (.cons ^clojure.lang.IPersistentVector pv (.getBytes ^String s (count [self] (count pv)) Object (toString [self] (str (into [] self (defn stringvec [coll] (into (-StringVec []) coll)) (defmethod print-method StringVec [v, ^java.io.Writer w] (.write w (.toString ^StringVec v))) Speak of cons, I gather ISeq cons is unrelated to cons, the function, but rather is required for conj? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clo...@googlegroups.com javascript: Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+u...@googlegroups.com javascript: For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com javascript:. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: extending custom data types to clojure core abstractions
Looks like it's based on the vector-of feature. I've seen this described elsewhere as a compact data structure, but it doesn't seem very compact. Maybe I'm using it wrong? (apply vector-of :byte (into [] (.getBytes foo))) [102 111 111] Looks like this works. Applying to the same test file: (def g (mapv #(apply vector-of :byte (into [] (.getBytes ^String %))) (line-seq f))) OutOfMemoryError Java heap space java.util.Arrays.copyOf (Arrays.java:2894) :-p This is a 160M file. Trying again, with more memory, jhat tells me this: num #instances #bytes class name -- 1: 2000961 285940064 [Ljava.lang.Object; 2: 5726264 257130296 [B 3: 5721690 137320560 clojure.core.VecNode 4: 1917764 76710560 clojure.core.Vec ... Total 15717203 785217096 Started with about 28M, so loading a 160M file with 2 million lines consumed 757M, or nearly 5X the size of the data. Same as what I see with pjstadig.utf8. On Tuesday, October 28, 2014 4:50:04 PM UTC-7, Andy Fingerhut wrote: I am not certain whether Paul's intention was a lower memory footprint than Java's strings, but I can't think of a strong reason to use his library other than that. Would you be willing to file an issue on Github with your findings to see if he thinks there is a problem there? Also, his code itself may have some answers to your questions about how to make new data structures satisfy some Clojure abstractions. Andy On Tue, Oct 28, 2014 at 3:57 PM, Brian Craft craft...@gmail.com javascript: wrote: Hm, I just tested it with a 160M file, split into 2 million strings. The memory footprint was substantially worse than String. With String I get about a 3.2X increase in memory footprint for this file, but with pjstadig.utf8 I get about 5X. Also, it's slower by orders of magnitude. On Tuesday, October 28, 2014 12:49:53 PM UTC-7, Andy Fingerhut wrote: Sorry, no feedback on your attempt, but a note that you may want to check out Paul Stadig's utf8 library to see if it serves your purpose. I believe it should store text that fits within the ASCII subset into 1 byte of memory per character, only using 2, 3, or 4 bytes for other Unicode characters, depending on the code point. https://github.com/pjstadig/utf8 Andy On Tue, Oct 28, 2014 at 12:24 PM, Brian Craft craft...@gmail.com wrote: Following up on the thread about the massive overhead of String, I tried writing a string collection type that stores strings as bytes, converting to String on-demand. It seems to work. Memory footprint and performance are good for the application. The hard part was trying to track down the correct interfaces and invocations. I note that Clojure Programming makes the same observation in the section about clojure abstractions: such things are largely undocumented. I guess this situation hasn't improved? I had to proceed mostly by experimentation, and am still unclear on, for example, why I needed to use an interop call in some places (like cons), but should not in others. Would be happy for any feedback on this attempt: (deftype StringVec [pv] clojure.lang.IPersistentVector (seq [self] (map #(String. ^bytes %) pv)) (nth [self i] (String. ^bytes (.nth ^clojure.lang.IPersistentVector pv i))) (nth [self i notfound] (String. ^bytes (.nth ^clojure.lang.IPersistentVector pv i (.getBytes ^String notfound clojure.lang.ILookup (valAt [self i] (when-let [res (.valAt ^clojure.lang.IPersistentVector pv i)] (String. ^bytes res))) (valAt [self i notfound] (String. ^bytes (.valAt ^clojure.lang.IPersistentVector pv i (.getBytes ^String notfound clojure.lang.ISeq (first [self] (String. ^bytes (first pv))) (next [self] (-StringVec (next pv))) (more [self] (-StringVec (rest pv))) (cons [self s] (-StringVec (.cons ^clojure.lang.IPersistentVector pv (.getBytes ^String s (count [self] (count pv)) Object (toString [self] (str (into [] self (defn stringvec [coll] (into (-StringVec []) coll)) (defmethod print-method StringVec [v, ^java.io.Writer w] (.write w (.toString ^StringVec v))) Speak of cons, I gather ISeq cons is unrelated to cons, the function, but rather is required for conj? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clo...@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+u...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u
processing large text files
The java overhead for Strings is incredible. Even moderate-sized input files will consume all memory. Are there good existing solutions? I found iota, which looks like a good solution for the read portion of the problem. However I also need to process the data in the file. If I start with an iota/vec and need to sort it, something like (sort (iota/vec foo)) will end up building Strings, again. The memory usage is absurd. It's also possible to use iota's random-access support instead of converting the whole collection to Strings, like (def f (iota/vec foo)) (sort (fn [i j] (compare (nth f i) (nth f j))) (into [] (range (count f) to return a list of row indexes in sorted order. But this is painfully slow. iota doesn't appear to provide a solution for efficient storage of transformed collections of strings. Are there other libraries, or standard java techniques for dealing with this? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: processing large text files
On Sunday, October 26, 2014 6:51:18 PM UTC-7, TheBusby wrote: On Mon, Oct 27, 2014 at 7:10 AM, Brian Craft craft...@gmail.com javascript: wrote: I found iota, which looks like a good solution for the read portion of the problem. However I also need to process the data in the file. If I start with an iota/vec and need to sort it, something like (sort (iota/vec foo)) Short disclaimer: I'm the one to blame for Iota. In this situation I've found it easier just to use GNU sort, or an external tool, and then use Iota with the pre-sorted file. Or generate an index with a smaller subset of the data. Makes sense, but not an option for this application. What about something similar to iota, backed with byte arrays, or something? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Interop nightmare
On Monday, September 8, 2014 11:09:50 AM UTC-7, Sean Corfield wrote: I find that it is quite difficult to use clojure unless one knows Java, which I believe to be a barrier to new comers. I'm surprised every time I hear this. You can write a lot of Clojure without having to do any interop so you can mostly ignore Java altogether unless you specifically want to work with a Java library. Yes, the stacktraces bleed Java but after the initial OMG! shock, they're generally easy to read - they're just LONG and you have to ignore a lot of the irrelevant parts. Leiningen mostly hides the ugly Java ecosystem as regards library management so, again, you can mostly ignore Java there too. Coming to clojure with no java experience is pretty brutal, in my experience. You absolutely can't ignore the Java, because of the embrace-the-platform architecture. I suspect this difficulty is surprising to java developers because it's so easy to forget how much one has learned. An error message like no matching ctor is very confusing without Java experience. It takes only a few minutes on google to sort this out, but there are at least hundreds of details like this that bleed through from Java. It adds up quickly. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
om tutorial, java.lang.UnsupportedClassVersionError
The basic tutorial fails on cljsbuild, with Exception in thread main java.lang.UnsupportedClassVersionError: com/google/javascript/jscomp/CompilerOptions : Unsupported major.minor version 51.0, compiling:(cljs/closure.clj:1:1) I gather this is some java version issue. Is there a workaround? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: running a jar-based cli tool
On Wednesday, August 13, 2014 11:30:59 PM UTC-7, Shantanu Kumar wrote: On Thursday, 14 August 2014 03:25:48 UTC+5:30, Brian Craft wrote: Thanks! This works perfectly. It took a few tries to find the right incantation, but this seems to do: (defn -main [ args] (Main/main (into-array String args))) I note that the .clj file for this namespace ends up in the uberjar, but looks like it doesn't pull in the jar for the tool. That could be because the JAR dependency is a dev dependency, whereas the uberjar didn't include dev dependencies. If you need the JAR to be included in the uberjar you may want to make that a regular dependency. Ah, no, I don't want the jar included, but don't really want the shim clj file included, either. I just added it to the uberjar-exclusions. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
vim java
fireplace is working pretty well for me when working with clojure files, but I often find I need to delve into the java to see what's going on. Is there any way to set this up so it's more seamless? I'd like to be able to jump to java source the same way I can jump to clojure source. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
running a jar-based cli tool
I need to run a tool while building docs which is distributed as a jar file, and is run with java -jar. Not sure the best way to do this. lein can fetch the jar if I add it to dev dependencies, but then it's in some directory in ~/.m2. Is there some simple way to get the path so I can pass it to java? Does lein expose the m2 repository directory somehow? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: running a jar-based cli tool
Thanks! This works perfectly. It took a few tries to find the right incantation, but this seems to do: (defn -main [ args] (Main/main (into-array String args))) I note that the .clj file for this namespace ends up in the uberjar, but looks like it doesn't pull in the jar for the tool. On Wednesday, August 13, 2014 2:04:04 PM UTC-7, Shantanu Kumar wrote: On Thursday, 14 August 2014 02:23:50 UTC+5:30, Brian Craft wrote: I need to run a tool while building docs which is distributed as a jar file, and is run with java -jar. Not sure the best way to do this. lein can fetch the jar if I add it to dev dependencies, but then it's in some directory in ~/.m2. Is there some simple way to get the path so I can pass it to java? Does lein expose the m2 repository directory somehow? If lein can fetch it, it'd be on the project classpath too -- just write a Clojure ns with a `-main` fn that invokes the main class in the JAR (find that in MANIFEST.MF file in the JAR), and trigger this ns using `lein run`. HTH. Shantanu -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: handling agent errors
On Monday, August 11, 2014 10:08:09 PM UTC-7, Jeremy Heiler wrote: On Mon, Aug 11, 2014 at 3:07 PM, Brian Craft craft...@gmail.com javascript: wrote: Two questions, really. I'm running large batch jobs with an agent. The jobs may fail in any number of unanticipated ways (due to user input), so I do a try/catch in the agent, log any errors, and continue with the next job. First question: how to deal with OOM (the Xmx limit, not the OS out of mem)? The try/catch doesn't appear to help, here. The agent dies, and all subsequent jobs never run. Is there some way to prevent or recover from this? Once it has happened, I don't think there's much you can do to save the process. I'd suggest that you figure out where you're improperly managing memory by throwing a profiler at it. Profiling is not going to change user behavior: users can give us inputs too large for their systems. If we can't recover from it, then we need to predict the event and abort. Perhaps we could monitor heap usage in another thread, and shut down the agent if it goes past some fraction. But there doesn't appear to be any mechanism for shutting down an agent. Maybe the agent action could stash the thread id in an atom, and a watcher thread could shut it down somehow. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: handling agent errors
On Tuesday, August 12, 2014 9:59:54 AM UTC-7, Brian Craft wrote: On Monday, August 11, 2014 10:08:09 PM UTC-7, Jeremy Heiler wrote: On Mon, Aug 11, 2014 at 3:07 PM, Brian Craft craft...@gmail.com wrote: Two questions, really. I'm running large batch jobs with an agent. The jobs may fail in any number of unanticipated ways (due to user input), so I do a try/catch in the agent, log any errors, and continue with the next job. First question: how to deal with OOM (the Xmx limit, not the OS out of mem)? The try/catch doesn't appear to help, here. The agent dies, and all subsequent jobs never run. Is there some way to prevent or recover from this? Once it has happened, I don't think there's much you can do to save the process. I'd suggest that you figure out where you're improperly managing memory by throwing a profiler at it. Profiling is not going to change user behavior: users can give us inputs too large for their systems. If we can't recover from it, then we need to predict the event and abort. Perhaps we could monitor heap usage in another thread, and shut down the agent if it goes past some fraction. But there doesn't appear to be any mechanism for shutting down an agent. Maybe the agent action could stash the thread id in an atom, and a watcher thread could shut it down somehow. http://docs.oracle.com/javase/1.5.0/docs/guide/misc/threadPrimitiveDeprecation.html Hm... from this doc it appears that maybe the thing to do is implement a please stop atom that the agent action would check periodically. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
handling agent errors
Two questions, really. I'm running large batch jobs with an agent. The jobs may fail in any number of unanticipated ways (due to user input), so I do a try/catch in the agent, log any errors, and continue with the next job. First question: how to deal with OOM (the Xmx limit, not the OS out of mem)? The try/catch doesn't appear to help, here. The agent dies, and all subsequent jobs never run. Is there some way to prevent or recover from this? I posed this to a long-time java developer, who said I shouldn't be doing a blanket try/catch in the first place, because it's likely to result in leaked memory (presumably due to badly-coded libs that don't release resources on error). His suggestion: let the thread die on error, and restart it from a monitoring thread. I'm not sure how this translates to clojure agents. If I don't catch the errors, then log the agent error state and restart it, is that equivalent? Is this best practice? Seems like badly-coded libs could still leak memory this way, if they've stashed a reference in any shared state. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
workflow with aot compiling
A number of clojure tools, like tools.namespaces, and codox, don't work with aot compiling, and their docs say to turn it off. However aot compiling isn't really optional: we're embracing the platform, and java libs like their named classes. How do you work around this? I can move aot compiling to the uberjar target, but then I can't work on the system from the repl, because it can't be launched. The java libs won't find the classes they need. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: setting c3p0 logging
In case anyone hits the same problem: strace showed the file being read, but it was having no effect, and c3p0 reported no errors in the config. Checking the c3p0 changelog, I found I needed to update to the c3p0 prerelease to get slf4j support. Now it's working. On Friday, August 8, 2014 10:11:17 PM UTC-7, Brian Craft wrote: I'm trying to get c3p0 to log to slf4j. Anyone have this working? I'm dropping a c3p0.properties file in the resources directory, but it's not having any effect. I can't tell if the file is being found. Is there a good way to debug this? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
setting c3p0 logging
I'm trying to get c3p0 to log to slf4j. Anyone have this working? I'm dropping a c3p0.properties file in the resources directory, but it's not having any effect. I can't tell if the file is being found. Is there a good way to debug this? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
docs with sphinx rst
I need to integrate with a project which uses sphinx to generate documentation. Are there any solutions for documenting clojure apps with sphinx? I found lein-sphinx, which allows running a sphinx build from lein, but doesn't include a clojure domain for sphinx. Any sphinx users here? Maybe adapting the common lisp domain, or something? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
honeysql tagged literal problem
Trying to do a #sql/call with honeysql, like [#sql/call [:unpackValue probe-field :row]] where probe-field is a local variable, it builds a SqlCall object with symbol probe-field, instead of using the value in the probe-field variable. I guess, then, that tagged literal handlers are passed symbols before they've been resolved? Is there a workaround for this? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: timbre logging, java libs
Very useful, thanks! I was considering just dropping timbre, or extracting the profiling macros. The main things I use from timbre are the profiling macros. I found Stuart's post about the original contrib profiling macros being a half-baked idea, but I'm unaware of a usable alternative. GUIs like visualvm are only good for spot-checking, not for logging performance vs. a few thousand use cases, for example, or for keeping a slow-query log. Are there other scriptable profiling solutions? On Friday, July 25, 2014 1:55:49 PM UTC-7, Hugo Duncan wrote: craft...@gmail.com javascript: writes: Is there any good way to use timbre in a project with java libs, e.g. c3p0, that use java logging APIs? You might want to look at: http://ptaoussanis.github.io/timbre/taoensso.timbre.tools.logging.html https://github.com/palletops/log-config#user-content-timbre-and-java-logging If you set up tools.logging to use your preferred java logging implementation, then timbre can log to it. Hugo -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
timbre logging, java libs
Is there any good way to use timbre in a project with java libs, e.g. c3p0, that use java logging APIs? Java logging is such a hopeless muddle I don't pretend to understand it, but my impression is that libs like c3p0 use assorted backends (log4j, etc.), which in turn use assorted appenders to write to files, dbs, etc. clojure.tools.logging uses the same backends (log4j, etc.), to the same purpose. Timbre, however, just implements appenders, so it can't forward to log4j, and log4j can't forward to timbre. Timbre can operate as a backend for clojure.tools.logging, but that only helps with clojure libs. Is any of this correct? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
unexpected behavior of clojure.core/empty
= (empty [:foo 5])
[]
= (first (mapv identity {:foo 5}))
[:foo 5]
= (empty (first (mapv identity {:foo 5})))
nil
What just happened there? Is this expected? In the second and third cases
the type of the vector is clojure.lang.MapEntry, which I expect is the root
of the behavior, but this seems very inconsistent. Is there a way to get
the behavior one would expect? Should clojure.core/empty be avoided?
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: unexpected behavior of clojure.core/empty
hm, looks even more broken in the context of these examples.
On Friday, July 18, 2014 5:04:34 AM UTC-7, Mike Fikes wrote:
My guess: Perhaps this is a bug, or alternatively, a known issue that
won't be addressed because to do so would be a breaking change.
There is an old demo of Clojure given by Rich where MapEntry's were
printed using some sort of un-readable notation #:foo 5. But clearly
MapEntry's have been revised to act a lot like 2-element vectors.
user= (rseq (first (mapv identity {:foo 5})))
(5 :foo)
user= (conj (first (mapv identity {:foo 5})) :x)
[:foo 5 :x]
user= (assoc (first (mapv identity {:foo 5})) 1 7)
[:foo 7]
user= (subvec (first (mapv identity {:foo 5})) 0 1)
[:foo]
FWIW, ClojureScript behaves in your expected way:
(empty (first (mapv identity {:foo 5})))
= []
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
casting error
Trying to eliminate a reflection warning, I put a type hint on a function, like (defn foo ^PStatement [] ...) which compiles without the reflection warnings, but at run time I get ClassCastException java.lang.ClassCastException: cavm.h2.PStatement cannot be cast to cavm.h2.PStatement What does this mean? PStatement is a defrecord type. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: casting error
Apparently this is due to a class loader problem. I moved it into a different file to try to avoid the problem. However I'm now finding that nothing referencing foo will compile, throwing this error: CompilerException java.lang.IllegalArgumentException: Unable to resolve classname: PStatement No idea what to do with this. Is it not possible to type hint methods returning records? On Wednesday, July 16, 2014 4:21:22 PM UTC-7, Brian Craft wrote: Trying to eliminate a reflection warning, I put a type hint on a function, like (defn foo ^PStatement [] ...) which compiles without the reflection warnings, but at run time I get ClassCastException java.lang.ClassCastException: cavm.h2.PStatement cannot be cast to cavm.h2.PStatement What does this mean? PStatement is a defrecord type. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: casting error
That does, in fact, fix the compile issue. Thanks! The cast error, however, persists. It's something to do with reloading the namespace with the defrecord. Once it happens, it takes a lein clean to clear. After that it works w/o error until the namespace is reloaded. It's something to do with referencing the defrecord from another namespace, or aot compiling. So far I haven't found a solution. It appears that an aot compiled file can't reference a function with a type hint for a defrecord if the namespace with that defrecord is then dynamically reloaded. Or something. It all looks exceedingly complex at this point. On Wednesday, July 16, 2014 8:36:35 PM UTC-7, Andy Fingerhut wrote: Try fully qualifying the class name in the return type hint. Might be the same symptom as in this ticket: http://dev.clojure.org/jira/browse/CLJ-1232 Andy On Wed, Jul 16, 2014 at 7:18 PM, Brian Craft craft...@gmail.com javascript: wrote: Apparently this is due to a class loader problem. I moved it into a different file to try to avoid the problem. However I'm now finding that nothing referencing foo will compile, throwing this error: CompilerException java.lang.IllegalArgumentException: Unable to resolve classname: PStatement No idea what to do with this. Is it not possible to type hint methods returning records? On Wednesday, July 16, 2014 4:21:22 PM UTC-7, Brian Craft wrote: Trying to eliminate a reflection warning, I put a type hint on a function, like (defn foo ^PStatement [] ...) which compiles without the reflection warnings, but at run time I get ClassCastException java.lang.ClassCastException: cavm.h2.PStatement cannot be cast to cavm.h2.PStatement What does this mean? PStatement is a defrecord type. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clo...@googlegroups.com javascript: Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+u...@googlegroups.com javascript: For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com javascript:. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: casting error
I have a wonky work-around. The file that is aot compiled depends on the file with the defrecord reference. I eliminated the direct dependency and exported a function that does an alter-root-var. So the provider injects the required value into the dependent. On Wednesday, July 16, 2014 9:33:52 PM UTC-7, Brian Craft wrote: That does, in fact, fix the compile issue. Thanks! The cast error, however, persists. It's something to do with reloading the namespace with the defrecord. Once it happens, it takes a lein clean to clear. After that it works w/o error until the namespace is reloaded. It's something to do with referencing the defrecord from another namespace, or aot compiling. So far I haven't found a solution. It appears that an aot compiled file can't reference a function with a type hint for a defrecord if the namespace with that defrecord is then dynamically reloaded. Or something. It all looks exceedingly complex at this point. On Wednesday, July 16, 2014 8:36:35 PM UTC-7, Andy Fingerhut wrote: Try fully qualifying the class name in the return type hint. Might be the same symptom as in this ticket: http://dev.clojure.org/jira/browse/CLJ-1232 Andy On Wed, Jul 16, 2014 at 7:18 PM, Brian Craft craft...@gmail.com wrote: Apparently this is due to a class loader problem. I moved it into a different file to try to avoid the problem. However I'm now finding that nothing referencing foo will compile, throwing this error: CompilerException java.lang.IllegalArgumentException: Unable to resolve classname: PStatement No idea what to do with this. Is it not possible to type hint methods returning records? On Wednesday, July 16, 2014 4:21:22 PM UTC-7, Brian Craft wrote: Trying to eliminate a reflection warning, I put a type hint on a function, like (defn foo ^PStatement [] ...) which compiles without the reflection warnings, but at run time I get ClassCastException java.lang.ClassCastException: cavm.h2.PStatement cannot be cast to cavm.h2.PStatement What does this mean? PStatement is a defrecord type. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clo...@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+u...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: working intellij plugin?
Cool, got it working. Is there a way to edit and evaluate a buffer loaded from a jar? Currently it has a lock icon, which I haven't found a way to turn off. On Sunday, July 13, 2014 10:41:04 PM UTC-7, Colin Fleming wrote: HI Brian, Both La Clojure and the IntelliJ Leiningen plugin are effectively discontinued in favour of Cursive. There's full instructions on how to get started here: https://cursiveclojure.com/userguide. Let me know if you have any more questions. Cheers, Colin On 14 July 2014 03:35, Brian Craft craft...@gmail.com javascript: wrote: In need of a way to breakpoint in java code, I decided to look at intellij. I tried the directions here: http://wiki.jetbrains.net/intellij/Getting_started_with_La_Clojure which don't work. When trying to open a project, it throws: java.lang.NoSuchMethodError: com.intellij.openapi.module.ModifiableModuleModel.newModule(Ljava/lang/String;Lcom/intellij/openapi/module/ModuleType;)Lcom/intellij/openapi/module/Module; at de.janthomae.leiningenplugin.project.LeiningenProject.reimport(LeiningenProject.java:133) at de.janthomae.leiningenplugin.project.LeiningenProjectsManager.importLeiningenProject(LeiningenProjectsManager.java:71) at de.janthomae.leiningenplugin.project.LeiningenProjectBuilder.commit(LeiningenProjectBuilder.java:57) Tracking down the leinningen plugin project I see that it's dead, and points to the cursive project, which has no downloads. Is there a way to get this working? Or, alternatively, is there a less painful way to set a breakpoint in java code from a lein project? My usual environment is fireplace. Command line would be great, but I'm not up for switching to emacs this week. ;) -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clo...@googlegroups.com javascript: Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+u...@googlegroups.com javascript: For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com javascript:. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: working intellij plugin?
That's unfortunate. I do this constantly in vim for debugging/investigation. I worked around it with in-ns from a writable file. On Monday, July 14, 2014 9:44:43 AM UTC-7, Colin Fleming wrote: No, there isn't, unfortunately - there are a couple of outstanding issues about this. IntelliJ has a fairly strict notion of files' origin, and it doesn't permit the concept of editing files that it won't be able to write back later. So that one will be tricky, I'll have to copy the file content to a scratch buffer or something. Not being able to load them is just a bug, I'll try to fix that for the next build. On 14 July 2014 17:15, Brian Craft craft...@gmail.com javascript: wrote: Cool, got it working. Is there a way to edit and evaluate a buffer loaded from a jar? Currently it has a lock icon, which I haven't found a way to turn off. On Sunday, July 13, 2014 10:41:04 PM UTC-7, Colin Fleming wrote: HI Brian, Both La Clojure and the IntelliJ Leiningen plugin are effectively discontinued in favour of Cursive. There's full instructions on how to get started here: https://cursiveclojure.com/userguide. Let me know if you have any more questions. Cheers, Colin On 14 July 2014 03:35, Brian Craft craft...@gmail.com wrote: In need of a way to breakpoint in java code, I decided to look at intellij. I tried the directions here: http://wiki.jetbrains.net/intellij/Getting_started_with_La_Clojure which don't work. When trying to open a project, it throws: java.lang.NoSuchMethodError: com.intellij.openapi.module. ModifiableModuleModel.newModule(Ljava/lang/String; Lcom/intellij/openapi/module/ModuleType;)Lcom/intellij/ openapi/module/Module; at de.janthomae.leiningenplugin.project.LeiningenProject. reimport(LeiningenProject.java:133) at de.janthomae.leiningenplugin.project. LeiningenProjectsManager.importLeiningenProject( LeiningenProjectsManager.java:71) at de.janthomae.leiningenplugin.project. LeiningenProjectBuilder.commit(LeiningenProjectBuilder.java:57) Tracking down the leinningen plugin project I see that it's dead, and points to the cursive project, which has no downloads. Is there a way to get this working? Or, alternatively, is there a less painful way to set a breakpoint in java code from a lein project? My usual environment is fireplace. Command line would be great, but I'm not up for switching to emacs this week. ;) -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clo...@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+u...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clo...@googlegroups.com javascript: Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+u...@googlegroups.com javascript: For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com javascript:. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
working intellij plugin?
In need of a way to breakpoint in java code, I decided to look at intellij. I tried the directions here: http://wiki.jetbrains.net/intellij/Getting_started_with_La_Clojure which don't work. When trying to open a project, it throws: java.lang.NoSuchMethodError: com.intellij.openapi.module.ModifiableModuleModel.newModule(Ljava/lang/String;Lcom/intellij/openapi/module/ModuleType;)Lcom/intellij/openapi/module/Module; at de.janthomae.leiningenplugin.project.LeiningenProject.reimport(LeiningenProject.java:133) at de.janthomae.leiningenplugin.project.LeiningenProjectsManager.importLeiningenProject(LeiningenProjectsManager.java:71) at de.janthomae.leiningenplugin.project.LeiningenProjectBuilder.commit(LeiningenProjectBuilder.java:57) Tracking down the leinningen plugin project I see that it's dead, and points to the cursive project, which has no downloads. Is there a way to get this working? Or, alternatively, is there a less painful way to set a breakpoint in java code from a lein project? My usual environment is fireplace. Command line would be great, but I'm not up for switching to emacs this week. ;) -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
syntax tree manipulation
What clojure tools should I be considering for doing syntax tree manipulations? In general, I'm recursively matching patterns in subtrees and rewriting them. The patterns are usually more complex than, say, core.match patterns (e.g. match subtree having vector that contains term, and split the term out of the vector and put in its own subtree). -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
stderr with repl development
Still not able to track down stderr. Anyone know how this stuff works? Dumping *err* from a ring handler, running from jetty, I get this: java.io.PrintWriter@1d9e436a. This will write to the repl terminal, so long as I flush (wtf... stderr is buffered?). If I dump *err* from a future, I get this: java.io.PrintWriter@fa4b83c. This is the same value I get if I eval (str *err*) in fireplace. Writing to this PrintWriter doesn't output anywhere that I can find. I'm guessing this is the channel used by fireplace to capture output and return it to vim. So, something is different in how *err* is set up, between the jetty adaptor and a future? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: stderr with repl development
A small clue, gleaned from a cider issue: This outputs to the repl terminal. (future (.start (Thread. #(println blah2 This output is captured by vim. (future (println blah3)) Still no idea what's going on. On Wednesday, May 21, 2014 1:09:31 PM UTC-7, Brian Craft wrote: Still not able to track down stderr. Anyone know how this stuff works? Dumping *err* from a ring handler, running from jetty, I get this: java.io.PrintWriter@1d9e436a. This will write to the repl terminal, so long as I flush (wtf... stderr is buffered?). If I dump *err* from a future, I get this: java.io.PrintWriter@fa4b83c. This is the same value I get if I eval (str *err*) in fireplace. Writing to this PrintWriter doesn't output anywhere that I can find. I'm guessing this is the channel used by fireplace to capture output and return it to vim. So, something is different in how *err* is set up, between the jetty adaptor and a future? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: stderr with repl development
On Wednesday, May 21, 2014 2:10:06 PM UTC-7, Gary Trakhman wrote: I use this trick pretty much all the time. (alter-var-root #'*out* (constantly *out*)), same for *err*, though I'll wire *err* to *out* Wow, I have no clue what that does. When do you run this? When would *out* not be the value in #'*out*? You've closed over *out* in some context, and the alter-var-root is running in some different context, where *out* has been reassigned? (future (println blah3)) works because *out* is conveyed to the future via binding-conveyor-function. That's not the case in the first. To be clear, it's the future case that is problematic. It writes (I think) to some vim channel and is lost forever. The Thread case works. It writes to the repl terminal, where I can see it. So, future invokes the binding conveyor, which sets *err* to the vim channel. Where does Thread get its *err*? On Wed, May 21, 2014 at 4:41 PM, Brian Craft craft...@gmail.comjavascript: wrote: A small clue, gleaned from a cider issue: This outputs to the repl terminal. (future (.start (Thread. #(println blah2 This output is captured by vim. (future (println blah3)) Still no idea what's going on. On Wednesday, May 21, 2014 1:09:31 PM UTC-7, Brian Craft wrote: Still not able to track down stderr. Anyone know how this stuff works? Dumping *err* from a ring handler, running from jetty, I get this: java.io.PrintWriter@1d9e436a. This will write to the repl terminal, so long as I flush (wtf... stderr is buffered?). If I dump *err* from a future, I get this: java.io.PrintWriter@fa4b83c. This is the same value I get if I eval (str *err*) in fireplace. Writing to this PrintWriter doesn't output anywhere that I can find. I'm guessing this is the channel used by fireplace to capture output and return it to vim. So, something is different in how *err* is set up, between the jetty adaptor and a future? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clo...@googlegroups.comjavascript: Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+u...@googlegroups.com javascript: For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com javascript:. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
debug output with ring and fireplace
Running a ring server from vim-fireplace, any errors that occur during request processing will dump to the repl terminal, which is fine. I'd like to add debug output and have it also dump to the repl. However printing to stdout or stderr from a ring handler doesn't work. How can I output to the repl? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
passing commands to a running server
I'm sure I've seen a thread on this, but can't find it now. Is there a common pattern for sending management commands to a running server via cli? E.g. if foo starts my ring server, I need something like foo -x twiddle to send twiddle to the running process. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
doall example
I came across this today in a library: results (seq (doall (filter modified? files)))] I believe the seq is to coerce the empty list to nil. What is the doall for? Context here: https://github.com/ibdknox/watchtower/blob/master/src/watchtower/core.clj -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: doall example
Ah, thanks. Seems like there's still a race, though. In a long list of files, a sequence like 1) doall evaluates modification date of the first file, which is less than *last-pass*, so it is filtered out 2) something touches the first file 3) the doall finishes evaluating the rest of the list, after which *last-pass* is set to the current time Now the first file has been updated, but still has modification date less than *last-pass*, and will be filtered out of the next pass as well. On Friday, May 16, 2014 4:50:19 PM UTC-7, James Reeves wrote: The doall evaluates all items in the seq, effectively removing any lazy evaluation. In this case it's necessary because the filter is checking the modification date of the file, which is obviously mutable. - James On 17 May 2014 00:39, Brian Craft craft...@gmail.com javascript:wrote: I came across this today in a library: results (seq (doall (filter modified? files)))] I believe the seq is to coerce the empty list to nil. What is the doall for? Context here: https://github.com/ibdknox/watchtower/blob/master/src/watchtower/core.clj -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clo...@googlegroups.comjavascript: Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+u...@googlegroups.com javascript: For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com javascript:. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
