Concatenating InputStreams (was: Re: Parsing SGML)
On Wed, Jul 11, 2012 at 14:46 +0200, Tassilo Horn wrote: Hm, probably yes. But you can implement the other arities quiet easily. So that version also accepts the version that reads into a byte array. The third version is left as an exercise for the reader. ;-) Thanks, I've done that and had to change your code a little in order to provide the same behaviour as the Java equivalent. (return -1 when no data can be read). The following is the implementation I use without problems now: --- snip --- (defn concat-input-stream Gets one or many input streams and returns a new input stream that concatenates the given streams. ^java.io.InputStream [^java.io.InputStream is more] (proxy [java.io.InputStream] [] (read ([] (let [input (.read is)] (if (and (== -1 input) (seq more)) (.read (apply concat-input-stream (first more) (rest more))) input))) ([arr] (loop [i 0] (if (== i (alength arr)) i (let [input (.read this)] (if (== -1 input) (if (== 0 i) input i) (do (clojure.core/aset arr i (byte input)) (recur (inc i ([arr off len] (loop [i 0] (if (or (== i len) (== (+ off i) (alength arr))) i (let [input (.read this)] (if (== -1 input) (if (== 0 i) input i) (do (clojure.core/aset arr (+ off i) (byte input)) (recur (inc i))) --- snip --- Thanks again for your help. Unfortunately I am still not sure what to use if I face even more horrible SGML in the future, but I think the best solution depends on the exact data ... Be that as it may: I am happy now. :) -- Wolodja babi...@gmail.com 4096R/CAF14EFC 081C B7CD FF04 2BA9 94EA 36B2 8B7F 7D30 CAF1 4EFC signature.asc Description: Digital signature
Re: Concatenating InputStreams (was: Re: Parsing SGML)
Hi, I think you don't handle the switching of streams correctly. You have to keep track with atoms. But then you run into thread-safety issues. (defn concat-input-stream Gets one or many input streams and returns a new input stream that concatenates the given streams. [is streams] (let [is (atom is) streams (atom streams) switch! (fn [] (locking is (.close @is) (reset! is (first streams)) (swap! streams rest))) do-read (fn ([is] (if is (.read is) -1)) ([is arr] (if is (.read is arr) -1)) ([is arr off len] (if is (.read is arr off len) -1)))] (proxy [java.io.InputStream] [] (close [] (when @is (.close @is) (doseq [s @streams] (.close s (read ([] (let [ch (do-read @is)] (if (neg? ch) (do (switch!) (if @is (.read this) -1)) ch))) ([arr] (let [n (do-read @is arr)] (if (neg? n) (do (switch!) (if @is (.read this arr) -1)) n))) ([arr off len] (let [n (do-read @is arr off len)] (if (neg? n) (do (switch!) (if @is (.read this arr off len) -1)) n))) You could also use java.io.SequenceInputStream with a small helper. (defn coll-enumeration [coll] (let [s (atom coll)] (reify [java.util.Enumeration] [] (hasMoreElements [this] (boolean (swap! s seq))) (nextElement [this] (locking this (if-let [sq (seq @s)] (let [e (first sq)] (reset! s (rest sq)) e) (throw (java.util.NoSuchElementException. (defn concat-input-stream Gets one or many input streams and returns a new input stream that concatenates the given streams. [is streams] (java.io.SequenceInputStream. (coll-enumeration (cons is streams All code untested. Kind regards Meikel -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Concatenating InputStreams (was: Re: Parsing SGML)
Hi again, talking about thread safety. Am Freitag, 13. Juli 2012 16:13:54 UTC+2 schrieb Meikel Brandmeyer (kotarak): (close [] (when @is (.close @is) (doseq [s @streams] (.close s (close [] (locking this (when @is (.close @is) (doseq [s @streams] (.close s) Kind regards Meikel -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Concatenating InputStreams (was: Re: Parsing SGML)
http://docs.oracle.com/javase/1.5.0/docs/api/java/io/SequenceInputStream.html On Fri, Jul 13, 2012 at 7:29 AM, Meikel Brandmeyer (kotarak) m...@kotka.de wrote: Hi again, talking about thread safety. Am Freitag, 13. Juli 2012 16:13:54 UTC+2 schrieb Meikel Brandmeyer (kotarak): (close [] (when @is (.close @is) (doseq [s @streams] (.close s (close [] (locking this (when @is (.close @is) (doseq [s @streams] (.close s) Kind regards Meikel -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -- And what is good, Phaedrus, And what is not good— Need we ask anyone to tell us these things? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Concatenating InputStreams (was: Re: Parsing SGML)
(defn coll-enumeration [coll] (clojure.lang.SeqEnumeration. coll)) On Friday, July 13, 2012 7:13:54 AM UTC-7, Meikel Brandmeyer (kotarak) wrote: Hi, I think you don't handle the switching of streams correctly. You have to keep track with atoms. But then you run into thread-safety issues. (defn concat-input-stream Gets one or many input streams and returns a new input stream that concatenates the given streams. [is streams] (let [is (atom is) streams (atom streams) switch! (fn [] (locking is (.close @is) (reset! is (first streams)) (swap! streams rest))) do-read (fn ([is] (if is (.read is) -1)) ([is arr] (if is (.read is arr) -1)) ([is arr off len] (if is (.read is arr off len) -1)))] (proxy [java.io.InputStream] [] (close [] (when @is (.close @is) (doseq [s @streams] (.close s (read ([] (let [ch (do-read @is)] (if (neg? ch) (do (switch!) (if @is (.read this) -1)) ch))) ([arr] (let [n (do-read @is arr)] (if (neg? n) (do (switch!) (if @is (.read this arr) -1)) n))) ([arr off len] (let [n (do-read @is arr off len)] (if (neg? n) (do (switch!) (if @is (.read this arr off len) -1)) n))) You could also use java.io.SequenceInputStream with a small helper. (defn coll-enumeration [coll] (let [s (atom coll)] (reify [java.util.Enumeration] [] (hasMoreElements [this] (boolean (swap! s seq))) (nextElement [this] (locking this (if-let [sq (seq @s)] (let [e (first sq)] (reset! s (rest sq)) e) (throw (java.util.NoSuchElementException. (defn concat-input-stream Gets one or many input streams and returns a new input stream that concatenates the given streams. [is streams] (java.io.SequenceInputStream. (coll-enumeration (cons is streams All code untested. Kind regards Meikel -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Concatenating InputStreams (was: Re: Parsing SGML)
Hi, Am 13.07.2012 um 19:25 schrieb Alan Malloy: (defn coll-enumeration [coll] (clojure.lang.SeqEnumeration. coll)) Ah. I knew there must be something. But it doesn't seem to be official, public API. Meikel signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Concatenating InputStreams (was: Re: Parsing SGML)
thank you for responding personally to me, but I sent my message to clojure google group. given the thread has the subject Concatenating InputStreams and I linked javadocs for a standard part of java.io that does exactly that, I am surprised that you decided to respond with tl;dr, but I have noted your response and will in the future let you prattle on endless reinventing functionality that already exists. On Fri, Jul 13, 2012 at 12:42 PM, Meikel Brandmeyer m...@kotka.de wrote: tl;dr Am 13.07.2012 um 18:35 schrieb Kevin Downey: http://docs.oracle.com/javase/1.5.0/docs/api/java/io/SequenceInputStream.html -- And what is good, Phaedrus, And what is not good— Need we ask anyone to tell us these things? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Concatenating InputStreams (was: Re: Parsing SGML)
It's a public class in clojure.lang that is not used by any part of the Clojure runtime or standard libraries. I can't think what other reason it would be included for, if not to be used by you. Just like clojure.lang.PersistentQueue - there's no nice Clojure wrapper for it (why on Earth not?), but it's still intended to be used by Clojure code. On Friday, July 13, 2012 12:40:07 PM UTC-7, Meikel Brandmeyer (kotarak) wrote: Hi, Am 13.07.2012 um 19:25 schrieb Alan Malloy: (defn coll-enumeration [coll] (clojure.lang.SeqEnumeration. coll)) Ah. I knew there must be something. But it doesn't seem to be official, public API. Meikel -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Concatenating InputStreams (was: Re: Parsing SGML)
Hi, Am 13.07.2012 um 23:48 schrieb Kevin Downey: thank you for responding personally to me, but I sent my message to clojure google group. given the thread has the subject Concatenating InputStreams and I linked javadocs for a standard part of java.io that does exactly that, I am surprised that you decided to respond with tl;dr, but I have noted your response and will in the future let you prattle on endless reinventing functionality that already exists. Sometimes it is necessary to write an endless prattle of reinvention to show that things are not as trivial as they might seem. The devil often lies in the details and one must pay close attention. (Even my endless prattle is still broken.) At the end of my message I referred to SequenceInputStream together with a reinvention of SeqEnumeration, which I didn't know, but was pointed to by Alan Malloy, who obviously – in contrast to you – read my message before responding. Kind regards Meikel signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Concatenating InputStreams (was: Re: Parsing SGML)
Hi, and more errata: reading is not thread-safe when the result is -1. Then an unintended switch! might happen. There things have to be packed into a locking together with a second read to verify it's still −1. Details! Details! Kind regards Meikel signature.asc Description: Message signed with OpenPGP using GPGMail
