Hello! After more than 3 years in the making, I am proud to announce the
release of Skyscraper 0.3.0, a scraping framework that helps you build
structured dumps of whole websites.
Home: https://github.com/nathell/skyscraper/
Major improvements in 0.3.0:
- Skyscraper has been rewritten from
Dear Clojurians,
Skyscraper 0.2.3, an Enlive-based library for scraping information from
whole sites in a structural way, has been released.
Homepage / GitHub: https://github.com/nathell/skyscraper
Clojars: https://clojars.org/skyscraper
New in this release:
- New feature: Custom parse
Skyscraper 0.2.2, an Enlive-based library for scraping information from
whole sites in a structural way, has been released.
Homepage / GitHub: https://github.com/nathell/skyscraper
Clojars: https://clojars.org/skyscraper
New in this release:
-
- Skyscraper now uses Timbre for logging.
Skyscraper 0.2.1, an Enlive-based library for scraping information from
whole sites in a structural way, has been released.
Homepage / GitHub: https://github.com/nathell/skyscraper
Clojars: https://clojars.org/skyscraper
New in this release:
- New function: get-cache-keys.
- scrape and
Skyscraper 0.2.0 has been released. New in this release:
- Skyscraper now supports pluggable cache backends.
- The caching mechanism has been completely overhauled and Skyscraper no
longer creates temporary files when the HTML cache is disabled.
- Support for capturing scraping
Skyscraper 0.1.2 has been released. New in this release:
- A processor can now return one context only. (Thanks to Bryan Maass.)
- The processed-cache option to scrape now works as advertised.
- New scrape option: :html-cache. (Thanks to ayato-p.)
- New official defprocessor clauses:
-chunking).
- Fixed a bug where relative URLs were incorrectly resolved in certain
circumstances.
Happy using,
-dj
W dniu wtorek, 11 sierpnia 2015 19:29:03 UTC+2 użytkownik Sergey Didenko
napisał:
Looks interesting, thank you.
On Tue, Aug 11, 2015 at 5:00 PM, Daniel Janus nat...@gmail.com
she’s interested in. Sites have tree-like structure, and you want to keep
track of this structure as you traverse the site, and reflect it in your
output. I call it “structural scraping”.
This is where Skyscraper comes in.
Happy using,
--Daniel Janus
--
You received this message because you
I'd like to join the thank-you chorus -- Instaparse rocks, period.
I'm using it to integrate my concordancing utility, Smyrna [1] with a
POS-tagging engine for Polish originally written in C++; specifically, to
parse the rules generated by the latter. I've given a talk on it this year
at
What *I* would very much like to see is something along the lines of
http://www.informatimago.com/linux/emacs-on-user-mode-linux.html but with
the JVM running Clojure directly on top of the Linux kernel. Plus some
editor/IDE to hack comfortably (Clooj?).
This cannot be done easily since the
Dear Clojurians,
I have released version 0.3.0 of clj-tagsoup [1], the HTML parser for
Clojure. clj-tagsoup is a wrapper around TagSoup [2] and can parse
arbitrary (potentially malformed) HTML (or XML) into Clojure data
structures.
New in this release is the ability to lazily parse XML using
I'm seeing it on Arch Linux as well, using both pip2 and easy_install-2.7.
Thanks,
Daniel
W dniu czwartek, 8 marca 2012, 12:00:24 UTC użytkownik tbc++ napisał:
Congrat's on the release! I am getting the following error on my
Macbook (running 64-bit Lion, Python 2.7.1) when trying to run sudo
Hi,
I've just released version 0.1 of clj-json-rpc to Clojars. clj-json-rpc is
a Clojure library that makes it easy to create web services using the
JSON-RPC protocol and Ring. Check it out
at https://github.com/nathell/clj-json-rpc
Enjoy,
Daniel
--
You received this message because you are
Hello,
In Clojure 1.3 this works:
(eval `(let [f# ~(fn [x] (+ x 1))] f#))
This does not, failing with an ExceptionInInitializerError:
(eval `(let [f# ~(with-meta #(+ % 1) {:foo :bar})] f#))
Both work in Clojure 1.2.1. Have I hit a regression bug? If not, what is
the rationale of this not
On Thursday, January 19, 2012 9:52:44 PM UTC, bsmith.occs wrote:
Incidentally, what's with this strange form of let?
(let (font-lock-mode) ;; - shouldn't this bind variables?
(clojure-mode-font-lock-setup))
In Common Lisp (and presumably in Emacs Lisp as well), 'let' takes an
(sort-by second (map-indexed vector [25 5 70]))
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To
On Thursday, December 22, 2011 3:35:35 PM UTC, Tassilo Horn wrote:
Brian Hurt bhu...@gmail.com writes:
Hi Brian,
Vectors are actually a great trade-off, giving you almost the same
access and memory costs arrays do, but with all the advantages of
being immutable (multi-threaded
On Wednesday, November 23, 2011 10:42:13 PM UTC, Nils Bertschinger wrote:
It solves a
common problem, namely to drop the last element of a sequence and
reads better in this case than the equivalent idiom using drop-last.
I don't quite get it. How does (butlast x) read better than
Hi,
I've encountered this behaviour of *print-dup*:
user (defstruct foo :field)
#'user/foo
user (binding [*print-dup* true] (pr-str (struct foo 10)))
#=(clojure.lang.PersistentStructMap/create {:field 10})
user (read-string (binding [*print-dup* true] (pr-str (struct foo 10
Hi,
Why keep both butlast and drop-last in clojure.core? The latter has the
advantage that it's lazy and can drop off more than one element from the
end of a seq. In contrast, I can't think of any advantage of butlast,
except that it seems to be slightly (ca 20%) faster than (doall (drop-last
As far as I know, the only NoSQL DB supporting transactions right now
is Redis. It also satisfies the rest of your points (well, I'm not
sure if Redis run on windows...).
+1 for Redis -- we are using it in production, in fact it's a central piece
of
our architecture, and so far it's
subset? is in the clojure.set namespace, so you must (use 'clojure.set)
before you can
use subset? unqualified.
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new
Hey, thanks for recommending my library! :-)
Best,
Daniel
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first
2) Network independence. I often work without Internet access, and I
don't want to be blocked at some point because some build tool wants
to access some repository to see if my version is still current.
For the record, this is easily doable with both Leiningen and Cake
(which both use
Hi Andreas,
(some #{:fred} [:fred :barney])
= :fred
This expected.
Would one be write to expect
(some #{:fred} {:fred flinstone :barney rubble})
to return
= :fred
as well?
(some #{:fred} (keys {:fred flinstone :barney rubble}))
will get you what you need.
Best,
Daniel
--
You received
On 3 Kwi, 07:37, Andreas Kostler andreas.koestler.le...@gmail.com
wrote:
Hi Daniel,
Thanks for your reply. I know how to get to :fred.
I'm just wondering why some wouldn't work on maps.
Well, `some' finds an element in a seq that satisfies the given
predicate, and seq'ing a map will get you a
Hi,
I've recently heard about the locals clearing feature of Clojure 1.2 (from a
recent post by Ken Wesson), and decided to test-drive it. Here is a
contrived example:
(defn garbage []
(make-array Byte/TYPE 10485760))
(defn -main [ args]
(let [a (garbage)
b (garbage)
c
On 23 Gru, 05:51, Robert McIntyre r...@mit.edu wrote:
I think it would be really cool to have a function that gives the
total number of bytes that a data structure consumes.
Here is a tiny utility I wrote some time ago; it's not very accurate,
but
might come in handy:
Hi Daniel,
I'm fairly certain this is not exactly what you're looking for, but
it's somewhat related and it might give you a fuller image -- my
tiny clj-bitfields library:
https://github.com/nathell/clj-bitfields
Best,
Daniel Janus
--
You received this message because you are subscribed
While on the topic, I'd like to raise a naming issue.
The 'mapmap' function seems to be a recurring theme (see, e.g.,
http://tech.puredanger.com/2010/09/24/meet-my-little-friend-mapmap/)
and many Clojure projects include one -- Incanter comes to mind. My
project used to, too. But we found out
Hi,
c.c.json/json-str seems to handle maps with keys containing quotes
incorrectly:
(println (json-str {\ 1}))
{:1}
...while I (and my parsers) would expect {\:1}.
I'd much rather report this on Assembla than here, but I seem to be
needing a CA to post a ticket there, and I'm in way too big
I forgot to add that this happens both with contrib 1.2.0 and 1.3-
alpha1.
Daniel
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be
On 30 Wrz, 20:46, Steve Purcell st...@sanityinc.com wrote:
You can file the bug as a support ticket without a CA here:
http://www.assembla.com/spaces/clojure/support/tickets
Thanks, I've reported it as a contrib support ticket. I wasn't aware
of this functionality.
Daniel
--
You received
On 28 Sie, 07:00, Stuart Sierra the.stuart.sie...@gmail.com wrote:
On Aug 27, 3:42 pm, B Smith-Mannschott bsmith.o...@gmail.com wrote:
This thread got me thinking that when a namespace is partially promoted to
Clojure proper, it might be good to provide a reduced version of the old
Hi,
so I finally got around to port my app to Clojure 1.2 and got confused
about the contrib shuffles.
There's clojure.java.io and clojure.contrib.io. The docs on the latter
says that most of the functions defined in there are deprecated, and
one should use clojure.java.io instead. But
Here's my take using my clj-iter (http://github.com/nathell/clj-iter):
(defn partition-when
[pred coll]
(iter (for x in coll)
(for p = (when x (pred x)))
(for y initially () then (if p (list x) (cons x y)))
(collect (reverse y) if p)
(finally-collect (reverse
agent-errors and await?)
How best to cope with this?
Best regards,
Daniel Janus
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please
://www.rfc-
editor.org/rfc/rfc2606.txt} RFC \n 2606]
, Section 3.]]]
The code is on http://github.com/nathell/clj-tagsoup. I've pushed it
to Clojars, too.
Best regards,
Daniel Janus
--
You received this message because you are subscribed to the Google
Groups Clojure group
On 8 Lis, 08:11, pmf phil.fr...@gmx.de wrote:
Hmm, someone else has made another closure available :).
http://googlecode.blogspot.com/2009/11/introducing-closure-tools.html
There's also Clozure Common Lisp [1], which is conceptually closer to
Clojure.
There's also a web browser written
Dear all,
I am happy to announce the public availability of clj-iter, an Iterate-
like iteration macro. It is free (available under the terms of MIT
license) and can be found on GitHub: http://github.com/nathell/clj-iter
The design goal was to keep it as simple as possible, and make it
blend
On 6 Lis, 02:02, John Harrop jharrop...@gmail.com wrote:
On Thu, Nov 5, 2009 at 7:03 PM, Daniel Janus nath...@gmail.com wrote:
To avoid citing the entire README blurb, I'll just give you some
examples:
(iter (for x in [31 41 59 26])
(for y from 1)
(collect (+ x y
Dear Clojurians,
Consider the following three simple functions:
(defn foo [] 42)
(defn bar [] 44)
(defn baz [s] (map (fn [_] (foo)) s))
Now let's rebind foo to bar:
user= (binding [foo bar] (foo))
44 ;; Just as I expected.
user= (binding [foo bar] (baz [1 2 3]))
(42 42 42) ;; while I would
I'd try to first compile Clojure to 1.5 bytecode, then translate it to
1.4 using Retroweaver (http://retroweaver.sourceforge.net/). I don't
know whether that'll work, though, since I think the Clojure compiler
generates and loads bytecode at runtime. You might need to patch
Clojure to somehow
Hello,
Is there any reason for some of the bitwise functions (bit-and-not,
bit-clear, bit-set, bit-flip, bit-test, bit-shift-left and bit-shift-
right) not having inline variants?
Thanks,
Daniel
--~--~-~--~~~---~--~~
You received this message because you are
and current git. Perhaps you should try
upgrading your setup?
Best,
Daniel Janus
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note
, and thanks to Paul and Meikel for pointing that out
--
I made the obvious mistake of assuming this is always going to work...
I'm going to change the API to replace the maps with vectors of
alternating
symbols and integers, which will make the ordering explicit.
Thanks once again!
Daniel Janus
to read it back and access the individual fields.
Best regards,
Daniel Janus
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from
would be extremely grateful if
someone had a look at this code and comments on how to make it more
Clojurish.
Thanks!
Daniel Janus
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group
certain that some
things might have been done better. I would be extremely grateful if
someone had a look at this code and comments on how to make it more
Clojurish.
Thanks!
Daniel Janus
--~--~-~--~~~---~--~~
You received this message because you are subscribed
Meikel,
I face a little challenge while using vbaxl.
I wanted to achieve the below
Set myRange = Worksheets(Sheet1).Range(A1:C10)
answer = Application.WorksheetFunction.Min(myRange)
I did ;
(def myRange (.. Dispatch (call ws Range A1:C10)(toDispatch)))
Which seems to work, but I was not able to
Hello All,
I can't figure out how to read cells with Jacob. I need a help.
Emeka
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Clojure group.
To post to this group, send email to clojure@googlegroups.com
To
While reading Programming Clojure the other night I found this code
interesting (+), however, when I tried out (-) I got my fingers burnt.
Why this? Or did I do something wrong which has nothing to do with
the code in question?
Emeka
--~--~-~--~~~---~--~~
You
Being new to Clojure , I started off playing with the codes ,and this
is the result, a 2cents calculator. It is not yet a done deal,
however, my laptop got fried, so I would be redundant for few days,
that's why I' ve decided to 'outsource' :).I invite your comments and
any code that might make
It can't figure out why this is not working or was I sleeping while
trying it out.
(apply #(println %) [2 3])
And, why is it that I can't do this?
Instead of (let [{coin :coin :as snake}{...}][coin
snake])
I tried this (let [{:coin coin :as snake}{...}][coin
Mibu,
Thanks for your post because it captures what I am passing through. I
have not done FP before and I am not even a great programmer, and with
FP comes a sea of concepts and abstracts I have not heard before.
These concepts and abstracts led me to conclude that FP is not made
for mere
Mibu
On Dec 18, 1:22 pm, Mibu mibu.cloj...@gmail.com wrote:
On Dec 18, 2:37 pm, janus emekami...@gmail.com wrote:
I think I need a mentor!
Come to the IRC channel (#clojure on irc.freenode.net). The people
there are friendly, helpful, and surprisingly patient.
Thanks for your advice
Timothy,
Your post is a great one indeed , you have developed a template that
anyone could use to introduce Clojure. I would implore to fresh out
thoughts and deepen it for all to enjoy.
Emeka
--~--~-~--~~~---~--~~
You received this message because you are
Thanks All,
It's working now,I followed Stuarts instructions(example 1).
Emeka
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Clojure group.
To post to this group, send email to clojure@googlegroups.com
To
Hi All,
It is working now. I found that I followed the instruction provided by
the author blindly, it was not meant for the windows box(vista) I am
using. The problem is on path separation, the book used forward slash
instead of backward slash.
The solution:
SET
Stuart
(1) You are using an old version of Clojure. You will need to build
Clojure and Clojure-contrib from SVN in order to work through the book
examples.
When I tried (require 'clojure.contrib.str-utils)
I got 'nil' which shows that all is well. I downloaded the most recent
binary
I got Failed to load Main-Class manifest attribute from C;/clojure/
clojure/clojure when I tried to run clojure. Could someone assist me
in getting this solved.
Emeka
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
All,
It is my mistake, this is the error once again. Failed to load Main-
Class manifest attribute from C:\clojure\clojure\clojure-contrib.jar.
I did what Shulz said , however , I still have the same problem.
Emeka
On Dec 6, 7:17 pm, Stuart Halloway [EMAIL PROTECTED] wrote:
If you have a
62 matches
Mail list logo
