Re: Issue when moving to Clojure 1.10
On 16 Jan 2019, at 18:17, Alex Miller wrote: > Yes, it's one of the downsides of AOT. I'd say it's not so much a downside of AOT - but of having a single output path for classes. I've long wanted Chas's patch to be applied, or something like it. Having everyone reinvent the mechanism whenever they happen to need AOT is kind of annoying - rare, but still annoying. --- "The ease with which a change can be implemented has no relevance at all to whether it is the right change for the (Java) Platform for all time." Mark Reinhold. Mark Derricutt http://www.theoryinpractice.net http://www.chaliceofblood.net http://plus.google.com/+MarkDerricutt http://twitter.com/talios http://facebook.com/mderricutt -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. signature.asc Description: OpenPGP digital signature
Re: Custom vectors/maps and sequence functions
Am Do., 17. Jan. 2019 um 00:52 Uhr schrieb Herwig Hochleitner < hhochleit...@gmail.com>: > 5. I could attach table like descriptions to each Record object (be it in >> its metadata or else), but then enforcing that all Records share the same >> Table data could get penalizing at runtime. >> > > That's the same trade-off as whether to implement substring by copying or > pointing to the original string. I don't think there is a universal answer > to that. Depends on your use case. > I misread your point, sorry. Please disregard this comment. > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Custom vectors/maps and sequence functions
Am Di., 15. Jan. 2019 um 14:58 Uhr schrieb :
>
> Imagine I try on the one side to represent something like a database table
> in memory, while on the other to make it pluggable into all meaningful
> sequence and vector/map functions in Clojure. In the most naive
> implementation a table is a vector of maps. If i would like to add more
> functionality, while trying to remain transparent to Clojure functions I
> could implement a custom Record type (IPersistentMap etc.) for the rows
> (for a custom storage, for column ordering, type checking etc.) and this
> works. I could also implement a custom Table (IPersistentVector etc.) for
> the actual collection of records while maintaining internally some indexes,
> the schema of the table etc.
>
IPersistentCollection and its derivatives are great to have on a custom
data type, as applicable.
To be a good citicen in modern clojure, you should also implement
IReduceInit / IReduce. This allows your collection to drive a transducer
stack, in order to evaluate whole-collection transformations at top-speed.
All transducer machinery is based on IReduceInit, so as soon as you
implement that, you enable efficient usage of reduce, transduce, into,
eduction, ... on your collection.
Also consider parameterizing your collection with a transducer stack, as a
basis for composable, postponed-eager transformations:
(table-transform #db{:trafo xf-db :db-data ...} xf-a)
=> #db{:trafo (comp xf-db xf-a) :db-data ...}
The point is that if possible to apply let say (filter.. or (take 2 ...,
> (sort .. incl. define/apply transducers. or whatever other Clojure function
> compatible with these interfaces on the
> Table/IPersistentVector/IPersistentCollection and get back a Table.
>
Implementing IEmptiableCollection allows the standard idiom:
(into (empty db) xform db)
If you have a more efficient way to transform your collection, e.g. by
composing transducers, there is no direct Clojure interface for that, sorry.
Though, you can implement something congruent to #(into (empty %1) %2 %1)
and call it table-transform (traverse?).
What I know:
> 1. Clojure's doc actually says that sequence functions return sequences
> and say nothing about preserving the original type, the implementations of
> the sequence functions do actually as advertised and return usually Cons-es.
>
This is on purpose. As soon as you've called seq on something, you're not
supposed to recover its original type and can only use more sequence
functions on it.
This allows for any number of intermediate sequence representations, most
notably lazy-seq, but also chunked-seq, ...
> 2. monads, yes, but the point is not to force the user to use custom
> functions instead of the built in ones.
>
In short, Clojure being a dynamic language and hence not having type-guided
expression generation (the return/pure problem), I think the best approach
for implementing monads is to hardcode the continuation monad and embed
other monads into that. See
http://blog.sigfpe.com/2008/12/mother-of-all-monads.html#c6884748521935561127
That's what transducers essentially do, IMO.
5. I could attach table like descriptions to each Record object (be it in
> its metadata or else), but then enforcing that all Records share the same
> Table data could get penalizing at runtime.
>
That's the same trade-off as whether to implement substring by copying or
pointing to the original string. I don't think there is a universal answer
to that. Depends on your use case.
So - do I miss something either in my knowledge or in the Clojure's
> documentation/implementation and is there a meaningful way to apply
> Clojure's and not mine filter/map/take/drop/sort etc. functions on a Table
> and to get a Table back, without going the monads/own functions for
> everything route or should I take it for granted that I can implement some
> custom types, but no way to get them be fully transparent to Clojure?
>
As detailed above, clojure's seq abstraction is intentionally opaque, so
own functions + implementing seq, empty, conj, being reducible and
accepting transducers is as transparent as it gets.
For optimizing single-thread performance, implement IEditableCollection
For utilizing multiple cores, implement clojure.core.reducers/CollFold
best regards
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: NoSuchMethodError when AOT'ing with Clojure 1.10
Oh, well disregard my last message then! > On Jan 16, 2019, at 5:47 PM, Alex Miller wrote: > > Off the top of my head, I think the only stuff in this area is the big bump > in the asm bytecode version, which could theoretically come into play here. > There were changes in reflection but I don’t think you’re doing reflection. I > am going to poke at this some more so it would probably be good to file a CLJ > jira to track it, if that’s possible. > >> On Jan 16, 2019, at 5:10 PM, Matthew Phillips wrote: >> >> I am pretty sure I'm using Java 8. I do have both Java 8 and Java 11 >> installed, but the environment it's built with: >> >> $ java -version >> java version "1.8.0_162" >> Java(TM) SE Runtime Environment (build 1.8.0_162-b12) >> Java HotSpot(TM) 64-Bit Server VM (build 25.162-b12, mixed mode) >> >> $ echo $JAVA_HOME >> /Library/Java/JavaVirtualMachines/jdk1.8.0_162.jdk/Contents/Home >> >> Same environment, just changing Clojure versions. I assume there's a change >> between Clojure 1.9 and 1.10 Java interop that explains the difference in >> behaviour? >> >> But AOT'ing using the wrong version of Java and deploying backwards >> certainly seems like it fits the problem perfectly, given the change in >> ByteBuffer between 8 and 9. Will further investigate. >> >> Cheers. And thanks, Alex. >> >>> On Wednesday, January 16, 2019 at 11:37:00 PM UTC+10:30, Alex Miller wrote: >>> Are you absolutely sure you’re not compiling on Java 9? I think Java 9 >>> added a position on ByteBuffer with covariant return (other versions have >>> it on the Buffer super class). >>> >>> From the error it looks like AOT compilation found the method and compiled >>> it into the byte code, but then that method is not found at runtime. I >>> think from eyeballing things (didn’t try it) that compiling with Java 9 and >>> running on Java 8 could result in this. >> >> -- >> You received this message because you are subscribed to the Google >> Groups "Clojure" group. >> To post to this group, send email to clojure@googlegroups.com >> Note that posts from new members are moderated - please be patient with your >> first post. >> To unsubscribe from this group, send email to >> clojure+unsubscr...@googlegroups.com >> For more options, visit this group at >> http://groups.google.com/group/clojure?hl=en >> --- >> You received this message because you are subscribed to a topic in the >> Google Groups "Clojure" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/clojure/n6wpzSGj-Z0/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> clojure+unsubscr...@googlegroups.com. >> For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: NoSuchMethodError when AOT'ing with Clojure 1.10
Off the top of my head, I think the only stuff in this area is the big bump in the asm bytecode version, which could theoretically come into play here. There were changes in reflection but I don’t think you’re doing reflection. I am going to poke at this some more so it would probably be good to file a CLJ jira to track it, if that’s possible. > On Jan 16, 2019, at 5:10 PM, Matthew Phillips wrote: > > I am pretty sure I'm using Java 8. I do have both Java 8 and Java 11 > installed, but the environment it's built with: > > $ java -version > java version "1.8.0_162" > Java(TM) SE Runtime Environment (build 1.8.0_162-b12) > Java HotSpot(TM) 64-Bit Server VM (build 25.162-b12, mixed mode) > > $ echo $JAVA_HOME > /Library/Java/JavaVirtualMachines/jdk1.8.0_162.jdk/Contents/Home > > Same environment, just changing Clojure versions. I assume there's a change > between Clojure 1.9 and 1.10 Java interop that explains the difference in > behaviour? > > But AOT'ing using the wrong version of Java and deploying backwards certainly > seems like it fits the problem perfectly, given the change in ByteBuffer > between 8 and 9. Will further investigate. > > Cheers. And thanks, Alex. > >> On Wednesday, January 16, 2019 at 11:37:00 PM UTC+10:30, Alex Miller wrote: >> Are you absolutely sure you’re not compiling on Java 9? I think Java 9 added >> a position on ByteBuffer with covariant return (other versions have it on >> the Buffer super class). >> >> From the error it looks like AOT compilation found the method and compiled >> it into the byte code, but then that method is not found at runtime. I think >> from eyeballing things (didn’t try it) that compiling with Java 9 and >> running on Java 8 could result in this. > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with your > first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to a topic in the Google > Groups "Clojure" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/clojure/n6wpzSGj-Z0/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > clojure+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: NoSuchMethodError when AOT'ing with Clojure 1.10
I've been able to reproduce this by deliberately building with Java 11/Clojure 1.10, then running on Java 8. Doing the same thing but built with Java 8 is fine. So, somehow some classes built with Java 11 must have gotten into the build. Time to check my build cleanliness... Thanks again, and sorry for taking up your time on a silly build error! Matt. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: NoSuchMethodError when AOT'ing with Clojure 1.10
I am pretty sure I'm using Java 8. I do have both Java 8 and Java 11 installed, but the environment it's built with: $ java -version java version "1.8.0_162" Java(TM) SE Runtime Environment (build 1.8.0_162-b12) Java HotSpot(TM) 64-Bit Server VM (build 25.162-b12, mixed mode) $ echo $JAVA_HOME /Library/Java/JavaVirtualMachines/jdk1.8.0_162.jdk/Contents/Home Same environment, just changing Clojure versions. I assume there's a change between Clojure 1.9 and 1.10 Java interop that explains the difference in behaviour? But AOT'ing using the wrong version of Java and deploying backwards certainly seems like it fits the problem perfectly, given the change in ByteBuffer between 8 and 9. Will further investigate. Cheers. And thanks, Alex. On Wednesday, January 16, 2019 at 11:37:00 PM UTC+10:30, Alex Miller wrote: > > Are you absolutely sure you’re not compiling on Java 9? I think Java 9 > added a position on ByteBuffer with covariant return (other versions have > it on the Buffer super class). > > From the error it looks like AOT compilation found the method and compiled > it into the byte code, but then that method is not found at runtime. I > think from eyeballing things (didn’t try it) that compiling with Java 9 and > running on Java 8 could result in this. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Online meeting: clojure data science
Hi! We are discussing the possibility of an online meeting around data science in clojure: https://clojureverse.org/t/online-meeting-clojure-data-science/3503 - would love to hear your comments. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: using durable-queue, works locally, get :time-out when moving to an EC2
Are you using aot?
We have used that durable queue with 1.8. In fact, we have a compatibility
later that allows you to move from the durable queue to an Aws queue:
(mostly undocumented)
https://github.com/techascent/tech.queue
Is the problem possibly a difference between your compilation environment
and your deploy env?
On Wed, Jan 16, 2019, 1:29 PM So, I upgraded to Java 11, and now everything works. So I guess this was a
> version conflict.
>
> Just curious, but is there a way for Factual to make durable-queue to tell
> Leiningen that Java 11 is necessary?
>
>
>
> On Wednesday, January 16, 2019 at 3:17:49 PM UTC-5, lawrence...@gmail.com
> wrote:
>>
>> On the new EC2 instance, running Ubuntu:
>>
>> java -version
>> openjdk version "1.8.0_191"
>> OpenJDK Runtime Environment (build
>> 1.8.0_191-8u191-b12-0ubuntu0.18.04.1-b12)
>> OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)
>>
>> Is it possible this version does not have the checksum signature that
>> durable-queue is looking for?
>>
>> I'm thinking this is some kind of version problem.
>>
>>
>>
>>
>>
>>
>> On Wednesday, January 16, 2019 at 2:50:14 PM UTC-5, lawrence...@gmail.com
>> wrote:
>>>
>>> Sorry, I'm an idiot. The real error was when I called put!
>>>
>>> I don't understand this error:
>>>
>>> INFO: java.lang.NoSuchMethodError: java.util.zip.Checksum.update([B)V
>>> java.lang.NoSuchMethodError: java.util.zip.Checksum.update([B)V
>>> at durable_queue$checksum.invokeStatic (durable_queue.clj:64)
>>> durable_queue$checksum.invokePrim (durable_queue.clj:-1)
>>> durable_queue.TaskSlab.append_to_slab_BANG_ (durable_queue.clj:314)
>>> durable_queue$queues$reify__6549$slab_BANG___6570.invoke
>>> (durable_queue.clj:702)
>>> durable_queue$queues$reify__6549$fn__6575.invoke
>>> (durable_queue.clj:719)
>>> durable_queue$queues$reify__6549.put_BANG_ (durable_queue.clj:717)
>>> durable_queue$queues$reify__6549.put_BANG_ (durable_queue.clj:732)
>>> humongorous_nlp.core$cycle_to_database$fn__13673$fn__13693.invoke
>>> (core.clj:665)
>>> humongorous_nlp.core$cycle_to_database$fn__13673.invoke
>>> (core.clj:663)
>>> clojure.lang.AFn.run (AFn.java:22)
>>> java.util.concurrent.Executors$RunnableAdapter.call
>>> (Executors.java:511)
>>> java.util.concurrent.FutureTask.runAndReset (FutureTask.java:308)
>>>
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301
>>> (ScheduledThreadPoolExecutor.java:180)
>>>
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run
>>> (ScheduledThreadPoolExecutor.java:294)
>>> java.util.concurrent.ThreadPoolExecutor.runWorker
>>> (ThreadPoolExecutor.java:1149)
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run
>>> (ThreadPoolExecutor.java:624)
>>> java.lang.Thread.run (Thread.java:748)
>>>
>>>
>>>
>>>
>>> On Wednesday, January 16, 2019 at 1:47:21 PM UTC-5,
>>> lawrence...@gmail.com wrote:
I was away from Clojure for a year and I missed it. I am pleased to be
back. But I've forgotten certain common errors. I feel like this is
something I used to know but now I've lost the knowledge.
I'm using Factual's durable-queue to put a step inbetween the import of
large JSON files, and their writes to the database. This works fine on my
local MacBook Pro, but when I move to an EC2 instance, I'm instead getting
time-outs when durable-queue tries to read from the queue.
At start up the app creates a few of these workers, which run for as
long as the app is running:
(defn worker
[from-topics-to-persistence-queue current-database-connection]
(slingshot/try+
(loop [message (durable/take! from-topics-to-persistence-queue
:message 6 :timed-out!)]
(slingshot/try+
(log-seq " the message in the work function " message)
(when (= (type message) durable_queue.Task)
(advance message current-database-connection))
(catch Object o
(log-seq "error in worker function")
(durable/retry! message)
(log-seq o)))
(recur (durable/take! from-topics-to-persistence-queue :message
6 :timed-out!)))
(catch Object o
(error o)
(slingshot/throw+ {
:type worker
:error o
:from-topics-to-persistence-queue
from-topics-to-persistence-queue
:current-database-connection
current-database-connection
}
On the EC2 instance, I see in the output:
Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke
INFO: the message in the work function
Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke
INFO:
Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke
INFO: :timed-out!
I'm using an
Re: using durable-queue, works locally, get :time-out when moving to an EC2
So, I upgraded to Java 11, and now everything works. So I guess this was a
version conflict.
Just curious, but is there a way for Factual to make durable-queue to tell
Leiningen that Java 11 is necessary?
On Wednesday, January 16, 2019 at 3:17:49 PM UTC-5, lawrence...@gmail.com
wrote:
>
> On the new EC2 instance, running Ubuntu:
>
> java -version
> openjdk version "1.8.0_191"
> OpenJDK Runtime Environment (build
> 1.8.0_191-8u191-b12-0ubuntu0.18.04.1-b12)
> OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)
>
> Is it possible this version does not have the checksum signature that
> durable-queue is looking for?
>
> I'm thinking this is some kind of version problem.
>
>
>
>
>
>
> On Wednesday, January 16, 2019 at 2:50:14 PM UTC-5, lawrence...@gmail.com
> wrote:
>>
>> Sorry, I'm an idiot. The real error was when I called put!
>>
>> I don't understand this error:
>>
>> INFO: java.lang.NoSuchMethodError: java.util.zip.Checksum.update([B)V
>> java.lang.NoSuchMethodError: java.util.zip.Checksum.update([B)V
>> at durable_queue$checksum.invokeStatic (durable_queue.clj:64)
>> durable_queue$checksum.invokePrim (durable_queue.clj:-1)
>> durable_queue.TaskSlab.append_to_slab_BANG_ (durable_queue.clj:314)
>> durable_queue$queues$reify__6549$slab_BANG___6570.invoke
>> (durable_queue.clj:702)
>> durable_queue$queues$reify__6549$fn__6575.invoke
>> (durable_queue.clj:719)
>> durable_queue$queues$reify__6549.put_BANG_ (durable_queue.clj:717)
>> durable_queue$queues$reify__6549.put_BANG_ (durable_queue.clj:732)
>> humongorous_nlp.core$cycle_to_database$fn__13673$fn__13693.invoke
>> (core.clj:665)
>> humongorous_nlp.core$cycle_to_database$fn__13673.invoke (core.clj:663)
>> clojure.lang.AFn.run (AFn.java:22)
>> java.util.concurrent.Executors$RunnableAdapter.call
>> (Executors.java:511)
>> java.util.concurrent.FutureTask.runAndReset (FutureTask.java:308)
>>
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301
>>
>> (ScheduledThreadPoolExecutor.java:180)
>>
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run
>> (ScheduledThreadPoolExecutor.java:294)
>> java.util.concurrent.ThreadPoolExecutor.runWorker
>> (ThreadPoolExecutor.java:1149)
>> java.util.concurrent.ThreadPoolExecutor$Worker.run
>> (ThreadPoolExecutor.java:624)
>> java.lang.Thread.run (Thread.java:748)
>>
>>
>>
>>
>> On Wednesday, January 16, 2019 at 1:47:21 PM UTC-5, lawrence...@gmail.com
>> wrote:
>>>
>>> I was away from Clojure for a year and I missed it. I am pleased to be
>>> back. But I've forgotten certain common errors. I feel like this is
>>> something I used to know but now I've lost the knowledge.
>>>
>>> I'm using Factual's durable-queue to put a step inbetween the import of
>>> large JSON files, and their writes to the database. This works fine on my
>>> local MacBook Pro, but when I move to an EC2 instance, I'm instead getting
>>> time-outs when durable-queue tries to read from the queue.
>>>
>>> At start up the app creates a few of these workers, which run for as
>>> long as the app is running:
>>>
>>> (defn worker
>>> [from-topics-to-persistence-queue current-database-connection]
>>> (slingshot/try+
>>>(loop [message (durable/take! from-topics-to-persistence-queue
>>> :message 6 :timed-out!)]
>>> (slingshot/try+
>>> (log-seq " the message in the work function " message)
>>> (when (= (type message) durable_queue.Task)
>>> (advance message current-database-connection))
>>> (catch Object o
>>> (log-seq "error in worker function")
>>> (durable/retry! message)
>>> (log-seq o)))
>>> (recur (durable/take! from-topics-to-persistence-queue :message
>>> 6 :timed-out!)))
>>>(catch Object o
>>> (error o)
>>> (slingshot/throw+ {
>>> :type worker
>>> :error o
>>> :from-topics-to-persistence-queue
>>> from-topics-to-persistence-queue
>>> :current-database-connection
>>> current-database-connection
>>> }
>>>
>>>
>>> On the EC2 instance, I see in the output:
>>>
>>> Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke
>>> INFO: the message in the work function
>>>
>>> Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke
>>> INFO:
>>>
>>> Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke
>>> INFO: :timed-out!
>>>
>>> I'm using an environment var to set the path to the queue:
>>>
>>> PATH_TO_DURABLE_QUEUE_S3_TO_DATABASE=/tmp/s3_to_database_queue
>>>
>>> I feel like I ran into this problem 2 years ago, but I can't recall the
>>> solution. Is the problem specific to AWS?
>>>
>>> An important clue, I think, is that I'm not getting an error on the
>>> put!, only on the take! Why would that be?
>>>
>>>
>>> --- lawrence
>>>
>>>
>>>
>>>
--
You received this
Re: using durable-queue, works locally, get :time-out when moving to an EC2
On the new EC2 instance, running Ubuntu:
java -version
openjdk version "1.8.0_191"
OpenJDK Runtime Environment (build 1.8.0_191-8u191-b12-0ubuntu0.18.04.1-b12)
OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)
Is it possible this version does not have the checksum signature that
durable-queue is looking for?
I'm thinking this is some kind of version problem.
On Wednesday, January 16, 2019 at 2:50:14 PM UTC-5, lawrence...@gmail.com
wrote:
>
> Sorry, I'm an idiot. The real error was when I called put!
>
> I don't understand this error:
>
> INFO: java.lang.NoSuchMethodError: java.util.zip.Checksum.update([B)V
> java.lang.NoSuchMethodError: java.util.zip.Checksum.update([B)V
> at durable_queue$checksum.invokeStatic (durable_queue.clj:64)
> durable_queue$checksum.invokePrim (durable_queue.clj:-1)
> durable_queue.TaskSlab.append_to_slab_BANG_ (durable_queue.clj:314)
> durable_queue$queues$reify__6549$slab_BANG___6570.invoke
> (durable_queue.clj:702)
> durable_queue$queues$reify__6549$fn__6575.invoke
> (durable_queue.clj:719)
> durable_queue$queues$reify__6549.put_BANG_ (durable_queue.clj:717)
> durable_queue$queues$reify__6549.put_BANG_ (durable_queue.clj:732)
> humongorous_nlp.core$cycle_to_database$fn__13673$fn__13693.invoke
> (core.clj:665)
> humongorous_nlp.core$cycle_to_database$fn__13673.invoke (core.clj:663)
> clojure.lang.AFn.run (AFn.java:22)
> java.util.concurrent.Executors$RunnableAdapter.call
> (Executors.java:511)
> java.util.concurrent.FutureTask.runAndReset (FutureTask.java:308)
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301
>
> (ScheduledThreadPoolExecutor.java:180)
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run
> (ScheduledThreadPoolExecutor.java:294)
> java.util.concurrent.ThreadPoolExecutor.runWorker
> (ThreadPoolExecutor.java:1149)
> java.util.concurrent.ThreadPoolExecutor$Worker.run
> (ThreadPoolExecutor.java:624)
> java.lang.Thread.run (Thread.java:748)
>
>
>
>
> On Wednesday, January 16, 2019 at 1:47:21 PM UTC-5, lawrence...@gmail.com
> wrote:
>>
>> I was away from Clojure for a year and I missed it. I am pleased to be
>> back. But I've forgotten certain common errors. I feel like this is
>> something I used to know but now I've lost the knowledge.
>>
>> I'm using Factual's durable-queue to put a step inbetween the import of
>> large JSON files, and their writes to the database. This works fine on my
>> local MacBook Pro, but when I move to an EC2 instance, I'm instead getting
>> time-outs when durable-queue tries to read from the queue.
>>
>> At start up the app creates a few of these workers, which run for as long
>> as the app is running:
>>
>> (defn worker
>> [from-topics-to-persistence-queue current-database-connection]
>> (slingshot/try+
>>(loop [message (durable/take! from-topics-to-persistence-queue
>> :message 6 :timed-out!)]
>> (slingshot/try+
>> (log-seq " the message in the work function " message)
>> (when (= (type message) durable_queue.Task)
>> (advance message current-database-connection))
>> (catch Object o
>> (log-seq "error in worker function")
>> (durable/retry! message)
>> (log-seq o)))
>> (recur (durable/take! from-topics-to-persistence-queue :message
>> 6 :timed-out!)))
>>(catch Object o
>> (error o)
>> (slingshot/throw+ {
>> :type worker
>> :error o
>> :from-topics-to-persistence-queue
>> from-topics-to-persistence-queue
>> :current-database-connection
>> current-database-connection
>> }
>>
>>
>> On the EC2 instance, I see in the output:
>>
>> Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke
>> INFO: the message in the work function
>>
>> Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke
>> INFO:
>>
>> Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke
>> INFO: :timed-out!
>>
>> I'm using an environment var to set the path to the queue:
>>
>> PATH_TO_DURABLE_QUEUE_S3_TO_DATABASE=/tmp/s3_to_database_queue
>>
>> I feel like I ran into this problem 2 years ago, but I can't recall the
>> solution. Is the problem specific to AWS?
>>
>> An important clue, I think, is that I'm not getting an error on the put!,
>> only on the take! Why would that be?
>>
>>
>> --- lawrence
>>
>>
>>
>>
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are
Re: using durable-queue, works locally, get :time-out when moving to an EC2
Sorry, I'm an idiot. The real error was when I called put!
I don't understand this error:
INFO: java.lang.NoSuchMethodError: java.util.zip.Checksum.update([B)V
java.lang.NoSuchMethodError: java.util.zip.Checksum.update([B)V
at durable_queue$checksum.invokeStatic (durable_queue.clj:64)
durable_queue$checksum.invokePrim (durable_queue.clj:-1)
durable_queue.TaskSlab.append_to_slab_BANG_ (durable_queue.clj:314)
durable_queue$queues$reify__6549$slab_BANG___6570.invoke
(durable_queue.clj:702)
durable_queue$queues$reify__6549$fn__6575.invoke (durable_queue.clj:719)
durable_queue$queues$reify__6549.put_BANG_ (durable_queue.clj:717)
durable_queue$queues$reify__6549.put_BANG_ (durable_queue.clj:732)
humongorous_nlp.core$cycle_to_database$fn__13673$fn__13693.invoke
(core.clj:665)
humongorous_nlp.core$cycle_to_database$fn__13673.invoke (core.clj:663)
clojure.lang.AFn.run (AFn.java:22)
java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:511)
java.util.concurrent.FutureTask.runAndReset (FutureTask.java:308)
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301
(ScheduledThreadPoolExecutor.java:180)
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run
(ScheduledThreadPoolExecutor.java:294)
java.util.concurrent.ThreadPoolExecutor.runWorker
(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run
(ThreadPoolExecutor.java:624)
java.lang.Thread.run (Thread.java:748)
On Wednesday, January 16, 2019 at 1:47:21 PM UTC-5, lawrence...@gmail.com
wrote:
>
> I was away from Clojure for a year and I missed it. I am pleased to be
> back. But I've forgotten certain common errors. I feel like this is
> something I used to know but now I've lost the knowledge.
>
> I'm using Factual's durable-queue to put a step inbetween the import of
> large JSON files, and their writes to the database. This works fine on my
> local MacBook Pro, but when I move to an EC2 instance, I'm instead getting
> time-outs when durable-queue tries to read from the queue.
>
> At start up the app creates a few of these workers, which run for as long
> as the app is running:
>
> (defn worker
> [from-topics-to-persistence-queue current-database-connection]
> (slingshot/try+
>(loop [message (durable/take! from-topics-to-persistence-queue :message
> 6 :timed-out!)]
> (slingshot/try+
> (log-seq " the message in the work function " message)
> (when (= (type message) durable_queue.Task)
> (advance message current-database-connection))
> (catch Object o
> (log-seq "error in worker function")
> (durable/retry! message)
> (log-seq o)))
> (recur (durable/take! from-topics-to-persistence-queue :message 6
> :timed-out!)))
>(catch Object o
> (error o)
> (slingshot/throw+ {
> :type worker
> :error o
> :from-topics-to-persistence-queue
> from-topics-to-persistence-queue
> :current-database-connection
> current-database-connection
> }
>
>
> On the EC2 instance, I see in the output:
>
> Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke
> INFO: the message in the work function
>
> Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke
> INFO:
>
> Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke
> INFO: :timed-out!
>
> I'm using an environment var to set the path to the queue:
>
> PATH_TO_DURABLE_QUEUE_S3_TO_DATABASE=/tmp/s3_to_database_queue
>
> I feel like I ran into this problem 2 years ago, but I can't recall the
> solution. Is the problem specific to AWS?
>
> An important clue, I think, is that I'm not getting an error on the put!,
> only on the take! Why would that be?
>
>
> --- lawrence
>
>
>
>
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
using durable-queue, works locally, get :time-out when moving to an EC2
I was away from Clojure for a year and I missed it. I am pleased to be
back. But I've forgotten certain common errors. I feel like this is
something I used to know but now I've lost the knowledge.
I'm using Factual's durable-queue to put a step inbetween the import of
large JSON files, and their writes to the database. This works fine on my
local MacBook Pro, but when I move to an EC2 instance, I'm instead getting
time-outs when durable-queue tries to read from the queue.
At start up the app creates a few of these workers, which run for as long
as the app is running:
(defn worker
[from-topics-to-persistence-queue current-database-connection]
(slingshot/try+
(loop [message (durable/take! from-topics-to-persistence-queue :message
6 :timed-out!)]
(slingshot/try+
(log-seq " the message in the work function " message)
(when (= (type message) durable_queue.Task)
(advance message current-database-connection))
(catch Object o
(log-seq "error in worker function")
(durable/retry! message)
(log-seq o)))
(recur (durable/take! from-topics-to-persistence-queue :message 6
:timed-out!)))
(catch Object o
(error o)
(slingshot/throw+ {
:type worker
:error o
:from-topics-to-persistence-queue
from-topics-to-persistence-queue
:current-database-connection
current-database-connection
}
On the EC2 instance, I see in the output:
Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke
INFO: the message in the work function
Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke
INFO:
Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke
INFO: :timed-out!
I'm using an environment var to set the path to the queue:
PATH_TO_DURABLE_QUEUE_S3_TO_DATABASE=/tmp/s3_to_database_queue
I feel like I ran into this problem 2 years ago, but I can't recall the
solution. Is the problem specific to AWS?
An important clue, I think, is that I'm not getting an error on the put!,
only on the take! Why would that be?
--- lawrence
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Issue when moving to Clojure 1.10
Okay, I think I have a good understanding now. Internally, I was using the Clojure build which bundles the dependency with it. That's why that namespace shows up inside my Clojure Jar. It makes sense also not to guarantee binary compatibility. We will make sure to always build and run using the same version of Clojure from now on. Is source compatibility guaranteed to some extent? Lile Clojure 1.10 compiling Clojure 1.8 source? I'd assume if so, its only guaranteed backward, like old Clojure is not guarantee to support newer Clojure code? -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
NoSuchMethodError when AOT'ing with Clojure 1.10
Are you absolutely sure you’re not compiling on Java 9? I think Java 9 added a position on ByteBuffer with covariant return (other versions have it on the Buffer super class). >From the error it looks like AOT compilation found the method and compiled it >into the byte code, but then that method is not found at runtime. I think from >eyeballing things (didn’t try it) that compiling with Java 9 and running on >Java 8 could result in this. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
