[racket-users] Re: appending files

[Scotty C]

ok brandon, that's a thought. build the hash on the hard drive at the time of 
data creation. you mention collision resolution. so let me build my hash on the 
hard drive using my 6 million buckets but increase the size of each bucket from 
5 slots to 20. right? i can't exactly recreate my vector/bignum hash on the 
hard drive because i can't dynamically resize the buckets like i can the 
bignums. this gives me a 4 gb file whereas my original was 1 gb. i have enough 
space for that so that's not a problem. so as my buckets fill up they head 
towards the average of 5 data items per bucket. so on average here's what 
happens with each hd hash record. i go to my hd hash and read 3.5 (think about 
it) items and 90% of the time i don't find my data so i do a write. in my 
process i do an initial write, then a read, a write, a read, a write. compare: 
3.5 vs 2 reads; 1 vs 3 writes. the reads are more costly and if i exceed 20 
items in a bucket the hd hash breaks. what do you think? is it worth it?

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Re: appending files

[Brandon Thomas]

On Tue, 2016-01-26 at 18:40 -0800, Scotty C wrote:
> alright george, i'm open to new ideas. here's what i've got going.
> running 64 bit linux mint OS on a 2 core laptop with 2 gb of ram. my
> key is 128 bits with ~256 bits per record. so my 1 gb file contains
> ~63 million records and ~32 million keys. about 8% will be dupes
> leaving me with ~30 million keys. i run a custom built hash. i use
> separate chaining with a vector of bignums. i am willing to let my
> chains run up to 5 keys per chain so i need a vector of 6 million
> pointers. that's 48 mb for the array. another 480 mb for the bignums.
> let's round that sum to .5 gb. i have another rather large bignum in
> memory that i use to reduce but not eliminate record duplication of
> about .5 gb. i'm attempting to get this thing to run in 2 places so i
> need 2 hashes. add this up .5+.5+.5 is 1.5 gb and that gets me to
> about my memory limit. the generated keys are random but i use one of
> the associated fields for sorting during the initial write to the
> hard drive. what goes in each of those files is totally random but
> dupes do not run across files. also, the number of keys is >1e25.
> 

Sorry, I haven't read through the entire conversation, so I hope I'm
not missing anything. Is there anything stopping you from restructuring
the data on disk and using the hash directly from there (possibly with
the help of a cache if speed is important)? For example, let's say each
entry is 256 bits. Use something like "(file-position dbfile (* 32
(hash-custom key)))" to seek over to the appropriate entry on disk and
read just the entry you need (using whatever colliosion resolution).
Then you'll be using no auxillary memory (unless your caching, which
can just be a smaller ram hash table). Unless of course I'm just
missing something completly.

Regards,
Brandon Thomas

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] appending files

[Benjamin Greenman]

On Tue, Jan 26, 2016 at 1:32 AM, Neil Van Dyke  wrote:

> you want to do "filename globbing"



There's also the glob package [1], which should give the exact same API as
the shell. No need to remember the trailing "$" or specifically exclude
dotfiles.

(require glob)
(glob "foo/*/*.rkt")

[1] http://pkgs.racket-lang.org/#[glob]

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] reaching connection pool limit on connection pool with +inf.0 limit?

['John Clements' via users-redirect]

> On Jan 25, 2016, at 5:51 PM, Greg Williams  wrote:
> 
> On Saturday, April 25, 2015 at 3:12:20 PM UTC-7, 'John Clements' via 
> users-redirect wrote:
>> I have a long-running racket server that’s connecting to a MySQL back end. 
>> It has a connection pool wrapped in a virtual connection, created like this:
>> 
>> ;; create a connection to the database
>> (define conn 
>>  (virtual-connection
>>   (connection-pool
>>(lambda ()
>>  (mysql-connect #:database "sodec2"
>> #:user "clements"
>> #:password “redacted"
>> #:socket mysql-socket)
>> 
>> When this server is running, it periodically produces errors of this form:
>> 
>> connection-pool-lease: connection-pool-limit reached
>> 
>> My reading of the documentation is that the “default” connection pool limit 
>> is +inf.0, which suggests that I shouldn’t be getting this error. 
>> 
>> Is this a bug?
>> 
>> 
>> 
>> Ah, what the heck, lemme ask about my other problem. I’m running this 
>> process with everything at or above ‘warning’ level redirected to syslog, 
>> and I’m a whole lot of these:
>> 
>> System Events
>> =-=-=-=-=-=-=
>> Apr 25 09:02:45 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:02:45 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:02:45 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:02:45 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:02:46 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:02:46 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:03:45 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:03:45 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:03:50 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:03:50 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:03:50 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:03:50 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:03:50 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:04:49 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:04:50 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:04:50 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:04:50 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:04:50 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:04:50 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:05:50 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:05:50 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:05:50 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:05:50 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:05:50 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:05:50 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:06:50 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:06:54 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:06:54 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:06:54 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:06:54 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> Apr 25 09:06:54 linode racket: epoll error at remove: Bad file descriptor; 
>> errno=9
>> ….
>> 
>> This continues for sometimes hours at a time. I just spent five minutes 
>> reading the epoll documentation, and it sounds like … maybe a select() on a 
>> fd that’s already closed? 
>> 
>> John
> 
> Did you ever figure out the "Bad file descriptor" issue? I'm seeing this 
> issue now and haven't been able to figure out what's causing it.


Nope, never did. Just put up with it. I’d love to know, though….

John



-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[racket-users] Re: appending files

[Scotty C]

robby findler, you the man. i like the copy-port idea. i incorporated it and it 
is nice and fast and easily fit into the existing code.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[racket-users] Re: appending files

[Scotty C]

neil van dyke, i have used the system function before but had forgotten what it 
was called and couldn't find it as a result in the documentation. my problem 
with using the system function is that i need 2 versions of it: windoz and 
linux. the copy-port function is a write once use across multiple os solution. 
sweet.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[racket-users] Re: appending files

[Scotty C]

gneuner2 (george), you are over thinking this thing. my test data of 1 gb is 
but a small sample file. i can't even hash that small 1 gb at the time of data 
creation. the hashed data won't fit in ram. at the time i put the redundant 
data on the hard drive, i do some constant time sorting so that the redundant 
data on the hard drive is contained in roughly 200 usefully sorted files. some 
of these files will be small and can be hashed with a single read, hash and 
write. some will be massive (data won't fit in ram) and must be split further. 
this produces another another type of single read, hash and write. these split 
files can now be fully hashed which means a second read, hash and write. 
recombining the second level files is virtually instantaneous (copy-port) 
relative to the effort spent to get to that point. all of these operations are 
constant time. it would be nice to cut into that big fat hard drive induced C 
but i can't do it with a single read and write on the larger files.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] (help ...) form in command-line REPL vs. Dr. Racket

[Robby Findler]

In DrRacket you can type f1 when your insertion point is on the word "help".

Robby

On Tue, Jan 26, 2016 at 4:28 PM, Brian Adkins  wrote:
> I was doing some reading on Scheme, and I came across the following page:
>
> http://bastibe.de/2012-09-20-story-about-schemes.html
>
> The author mentions:
>
> "That said, I found plt-racket to be a joy to work with. (help filter) will 
> open your browser with the appropriate help page for filter. Amazing. "
>
> I thought that was pretty cool, so I tried it in Dr. Racket and got: "help: 
> undefined"
>
> When I try it in the command-line REPL for racket, it worked great. I was 
> just curious why it works in the command-line REPL, but not Dr. Racket - do I 
> need to configure something for Dr. Racket?
>
> It's not a big deal because I usually just keep a browser tab opened to the 
> help and use the integrated search feature, but it could be handy on occasion.
>
> Thanks,
> Brian
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to racket-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] (help ...) form in command-line REPL vs. Dr. Racket

[Brian Adkins]

On Tuesday, January 26, 2016 at 6:03:41 PM UTC-5, Robby Findler wrote:
> In DrRacket you can type f1 when your insertion point is on the word "help".
> 
> Robby

Awesome - thanks. I'm not sure if you meant 'insertion point is on the word 
"filter" ', or not, but for fun, I hit F1 with the insertion point on "help" 
and got meta-help which showed me 18.3 Interactive Help.

I then evaluated (require racket/help) in the repl, then tried evaluating (help 
filter)

A web page was opened, but not the help for filter (it was 4.9 Pairs and 
Lists), and the repl had the following:

> (require racket/help)
> (help filter)
Loading help index...
Sending to web browser...
  file: /Applications/Racket v6.3/doc/reference/pairs.html
  anchor: (def._((lib._racket/private/list..rkt)._filter))
osascript: OpenScripting.framework - scripting addition 
"/Library/ScriptingAdditions/Adobe Unit Types.osax" cannot be used with the 
current OS because it has no OSAXHandlers entry in its Info.plist.
> 

Hitting F1 when the insertion point is on filter works great though :)

I also tried (help "filter")   and that does open up a more relevant page - the 
search results for searching help with filter; where the command-line repl 
opens up the page with the help for filter - the command-line (help) form 
appears to expect a procedure symbol and shows the specific help for it.


> On Tue, Jan 26, 2016 at 4:28 PM, Brian Adkins wrote:
> > I was doing some reading on Scheme, and I came across the following page:
> >
> > http://bastibe.de/2012-09-20-story-about-schemes.html
> >
> > The author mentions:
> >
> > "That said, I found plt-racket to be a joy to work with. (help filter) will 
> > open your browser with the appropriate help page for filter. Amazing. "
> >
> > I thought that was pretty cool, so I tried it in Dr. Racket and got: "help: 
> > undefined"
> >
> > When I try it in the command-line REPL for racket, it worked great. I was 
> > just curious why it works in the command-line REPL, but not Dr. Racket - do 
> > I need to configure something for Dr. Racket?
> >
> > It's not a big deal because I usually just keep a browser tab opened to the 
> > help and use the integrated search feature, but it could be handy on 
> > occasion.
> >
> > Thanks,
> > Brian

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] (help ...) form in command-line REPL vs. Dr. Racket

[Robby Findler]

Right, sorry. :)

On Tue, Jan 26, 2016 at 5:15 PM, Brian Adkins  wrote:
> On Tuesday, January 26, 2016 at 6:03:41 PM UTC-5, Robby Findler wrote:
>> In DrRacket you can type f1 when your insertion point is on the word "help".
>>
>> Robby
>
> Awesome - thanks. I'm not sure if you meant 'insertion point is on the word 
> "filter" ', or not, but for fun, I hit F1 with the insertion point on "help" 
> and got meta-help which showed me 18.3 Interactive Help.
>
> I then evaluated (require racket/help) in the repl, then tried evaluating 
> (help filter)
>
> A web page was opened, but not the help for filter (it was 4.9 Pairs and 
> Lists), and the repl had the following:
>
>> (require racket/help)
>> (help filter)
> Loading help index...
> Sending to web browser...
>   file: /Applications/Racket v6.3/doc/reference/pairs.html
>   anchor: (def._((lib._racket/private/list..rkt)._filter))
> osascript: OpenScripting.framework - scripting addition 
> "/Library/ScriptingAdditions/Adobe Unit Types.osax" cannot be used with the 
> current OS because it has no OSAXHandlers entry in its Info.plist.
>>
>
> Hitting F1 when the insertion point is on filter works great though :)
>
> I also tried (help "filter")   and that does open up a more relevant page - 
> the search results for searching help with filter; where the command-line 
> repl opens up the page with the help for filter - the command-line (help) 
> form appears to expect a procedure symbol and shows the specific help for it.
>
>
>> On Tue, Jan 26, 2016 at 4:28 PM, Brian Adkins wrote:
>> > I was doing some reading on Scheme, and I came across the following page:
>> >
>> > http://bastibe.de/2012-09-20-story-about-schemes.html
>> >
>> > The author mentions:
>> >
>> > "That said, I found plt-racket to be a joy to work with. (help filter) 
>> > will open your browser with the appropriate help page for filter. Amazing. 
>> > "
>> >
>> > I thought that was pretty cool, so I tried it in Dr. Racket and got: 
>> > "help: undefined"
>> >
>> > When I try it in the command-line REPL for racket, it worked great. I was 
>> > just curious why it works in the command-line REPL, but not Dr. Racket - 
>> > do I need to configure something for Dr. Racket?
>> >
>> > It's not a big deal because I usually just keep a browser tab opened to 
>> > the help and use the integrated search feature, but it could be handy on 
>> > occasion.
>> >
>> > Thanks,
>> > Brian
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to racket-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] (help ...) form in command-line REPL vs. Dr. Racket

[Brian Adkins]

On Tuesday, January 26, 2016 at 6:15:41 PM UTC-5, Brian Adkins wrote:
> On Tuesday, January 26, 2016 at 6:03:41 PM UTC-5, Robby Findler wrote:
> > In DrRacket you can type f1 when your insertion point is on the word "help".
> > 
> > Robby
> 
> Awesome - thanks. I'm not sure if you meant 'insertion point is on the word 
> "filter" ', or not, but for fun, I hit F1 with the insertion point on "help" 
> and got meta-help which showed me 18.3 Interactive Help.
> 
> I then evaluated (require racket/help) in the repl, then tried evaluating 
> (help filter)
> 
> A web page was opened, but not the help for filter (it was 4.9 Pairs and 
> Lists), and the repl had the following:


My bad, Dr. Racket *was* pulling up the right page (after requiring 
racket/help) - 4.9 is where filter is described. So both the command-line repl 
and Dr Racket repl give the same behavior after require in the latter.

I'm not sure what the OSAX error is, but it's just noise.


> 
> > (require racket/help)
> > (help filter)
> Loading help index...
> Sending to web browser...
>   file: /Applications/Racket v6.3/doc/reference/pairs.html
>   anchor: (def._((lib._racket/private/list..rkt)._filter))
> osascript: OpenScripting.framework - scripting addition 
> "/Library/ScriptingAdditions/Adobe Unit Types.osax" cannot be used with the 
> current OS because it has no OSAXHandlers entry in its Info.plist.
> > 
> 
> Hitting F1 when the insertion point is on filter works great though :)
> 
> I also tried (help "filter")   and that does open up a more relevant page - 
> the search results for searching help with filter; where the command-line 
> repl opens up the page with the help for filter - the command-line (help) 
> form appears to expect a procedure symbol and shows the specific help for it.
> 
> 
> > On Tue, Jan 26, 2016 at 4:28 PM, Brian Adkins wrote:
> > > I was doing some reading on Scheme, and I came across the following page:
> > >
> > > http://bastibe.de/2012-09-20-story-about-schemes.html
> > >
> > > The author mentions:
> > >
> > > "That said, I found plt-racket to be a joy to work with. (help filter) 
> > > will open your browser with the appropriate help page for filter. 
> > > Amazing. "
> > >
> > > I thought that was pretty cool, so I tried it in Dr. Racket and got: 
> > > "help: undefined"
> > >
> > > When I try it in the command-line REPL for racket, it worked great. I was 
> > > just curious why it works in the command-line REPL, but not Dr. Racket - 
> > > do I need to configure something for Dr. Racket?
> > >
> > > It's not a big deal because I usually just keep a browser tab opened to 
> > > the help and use the integrated search feature, but it could be handy on 
> > > occasion.
> > >
> > > Thanks,
> > > Brian

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] (help ...) form in command-line REPL vs. Dr. Racket

[Brian Adkins]

On Tuesday, January 26, 2016 at 6:21:13 PM UTC-5, Brian Adkins wrote:
> On Tuesday, January 26, 2016 at 6:15:41 PM UTC-5, Brian Adkins wrote:
> > On Tuesday, January 26, 2016 at 6:03:41 PM UTC-5, Robby Findler wrote:
> > > In DrRacket you can type f1 when your insertion point is on the word 
> > > "help".
> > > 
> > > Robby
> > 
> > Awesome - thanks. I'm not sure if you meant 'insertion point is on the word 
> > "filter" ', or not, but for fun, I hit F1 with the insertion point on 
> > "help" and got meta-help which showed me 18.3 Interactive Help.
> > 
> > I then evaluated (require racket/help) in the repl, then tried evaluating 
> > (help filter)
> > 
> > A web page was opened, but not the help for filter (it was 4.9 Pairs and 
> > Lists), and the repl had the following:
> 
> 
> My bad, Dr. Racket *was* pulling up the right page (after requiring 
> racket/help) - 4.9 is where filter is described. So both the command-line 
> repl and Dr Racket repl give the same behavior after require in the latter.
> 
> I'm not sure what the OSAX error is, but it's just noise.

The OSAX error was a red herring - I removed the Adobe Unit Types.osax file and 
no longer get the error, but the anchor tag on the URL isn't working from 
either repl.

It looks like it's *trying* to send the anchor:

> (require racket/help)
> (help filter)
Loading help index...
Sending to web browser...
  file: /Applications/Racket v6.3/doc/reference/pairs.html
  anchor: (def._((lib._racket/private/list..rkt)._filter))

but the URL ends up as:

file:///Applications/Racket%20v6.3/doc/reference/pairs.html

instead of:

file:///Applications/Racket%20v6.3/doc/reference/pairs.html#%28def._%28%28lib._racket%2Fprivate%2Flist..rkt%29._filter%29%29

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] (help ...) form in command-line REPL vs. Dr. Racket

[Brian Adkins]

On Tuesday, January 26, 2016 at 6:30:59 PM UTC-5, Brian Adkins wrote:
> On Tuesday, January 26, 2016 at 6:21:13 PM UTC-5, Brian Adkins wrote:
> > On Tuesday, January 26, 2016 at 6:15:41 PM UTC-5, Brian Adkins wrote:
> > > On Tuesday, January 26, 2016 at 6:03:41 PM UTC-5, Robby Findler wrote:
> > > > In DrRacket you can type f1 when your insertion point is on the word 
> > > > "help".
> > > > 
> > > > Robby
> > > 
> > > Awesome - thanks. I'm not sure if you meant 'insertion point is on the 
> > > word "filter" ', or not, but for fun, I hit F1 with the insertion point 
> > > on "help" and got meta-help which showed me 18.3 Interactive Help.
> > > 
> > > I then evaluated (require racket/help) in the repl, then tried evaluating 
> > > (help filter)
> > > 
> > > A web page was opened, but not the help for filter (it was 4.9 Pairs and 
> > > Lists), and the repl had the following:
> > 
> > 
> > My bad, Dr. Racket *was* pulling up the right page (after requiring 
> > racket/help) - 4.9 is where filter is described. So both the command-line 
> > repl and Dr Racket repl give the same behavior after require in the latter.
> > 
> > I'm not sure what the OSAX error is, but it's just noise.
> 
> The OSAX error was a red herring - I removed the Adobe Unit Types.osax file 
> and no longer get the error, but the anchor tag on the URL isn't working from 
> either repl.
> 
> It looks like it's *trying* to send the anchor:
> 
> > (require racket/help)
> > (help filter)
> Loading help index...
> Sending to web browser...
>   file: /Applications/Racket v6.3/doc/reference/pairs.html
>   anchor: (def._((lib._racket/private/list..rkt)._filter))
> 
> but the URL ends up as:
> 
> file:///Applications/Racket%20v6.3/doc/reference/pairs.html
> 
> instead of:
> 
> file:///Applications/Racket%20v6.3/doc/reference/pairs.html#%28def._%28%28lib._racket%2Fprivate%2Flist..rkt%29._filter%29%29

Manually copying what the repl reports as the anchor to the end of the URL 
(separated by #) works fine, so it knows the file, it knows the anchor, but by 
the time it gets to the browser, only the file is used.

Racket 6.3
OSX 10.10.5

Just out of curiosity, does this work for anyone else? In other words, in the 
Dr. Racket repl:

(require racket/help)
(help filter)

does that scroll down to the filter description in section 4.9, or is your URL 
missing the anchor?

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] (help ...) form in command-line REPL vs. Dr. Racket

[Pierpaolo Bernardi]

On Wed, Jan 27, 2016 at 12:35 AM, Brian Adkins  wrote:

> Just out of curiosity, does this work for anyone else? In other words, in the 
> Dr. Racket repl:
>
> (require racket/help)
> (help filter)
>
> does that scroll down to the filter description in section 4.9, or is your 
> URL missing the anchor?

it works for me (Racket 6.3 / windows)

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] (help ...) form in command-line REPL vs. Dr. Racket

[Brian Adkins]

On Tuesday, January 26, 2016 at 6:39:23 PM UTC-5, olopierpa wrote:
> On Wed, Jan 27, 2016 at 12:35 AM, Brian Adkins wrote:
> 
> > Just out of curiosity, does this work for anyone else? In other words, in 
> > the Dr. Racket repl:
> >
> > (require racket/help)
> > (help filter)
> >
> > does that scroll down to the filter description in section 4.9, or is your 
> > URL missing the anchor?
> 
> it works for me (Racket 6.3 / windows)

Thanks. I just changed my default browser to Safari, and it works fine, but 
when the default browser was Chrome, it failed.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] (help ...) form in command-line REPL vs. Dr. Racket

[Brian Adkins]

On Tuesday, January 26, 2016 at 6:42:43 PM UTC-5, Brian Adkins wrote:
> On Tuesday, January 26, 2016 at 6:39:23 PM UTC-5, olopierpa wrote:
> > On Wed, Jan 27, 2016 at 12:35 AM, Brian Adkins wrote:
> > 
> > > Just out of curiosity, does this work for anyone else? In other words, in 
> > > the Dr. Racket repl:
> > >
> > > (require racket/help)
> > > (help filter)
> > >
> > > does that scroll down to the filter description in section 4.9, or is 
> > > your URL missing the anchor?
> > 
> > it works for me (Racket 6.3 / windows)
> 
> Thanks. I just changed my default browser to Safari, and it works fine, but 
> when the default browser was Chrome, it failed.

Ok, this is odd, it works with both Chrome and Safari *if* they're not already 
open when (help filter) is evaluated. If they are already open, the URL is 
missing the anchor.

I *always* have Chrome open.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] (help ...) form in command-line REPL vs. Dr. Racket

[Pierpaolo Bernardi]

On Wed, Jan 27, 2016 at 12:46 AM, Brian Adkins  wrote:

> Ok, this is odd, it works with both Chrome and Safari *if* they're not 
> already open when (help filter) is evaluated. If they are already open, the 
> URL is missing the anchor.

In my case, the default browser is Chrome, and it was already open.

> I *always* have Chrome open.

Same here.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] (help ...) form in command-line REPL vs. Dr. Racket

[Brian Adkins]

On Tuesday, January 26, 2016 at 6:52:14 PM UTC-5, olopierpa wrote:
> On Wed, Jan 27, 2016 at 12:46 AM, Brian Adkins wrote:
> 
> > Ok, this is odd, it works with both Chrome and Safari *if* they're not 
> > already open when (help filter) is evaluated. If they are already open, the 
> > URL is missing the anchor.
> 
> In my case, the default browser is Chrome, and it was already open.
> 
> > I *always* have Chrome open.
> 
> Same here.

It's likely an OSX problem. The following works from a terminal window whether 
Chrome is already open or not:

osascript -e 'tell application "Chrome" to open location 
"file:///Applications/Racket%20v6.3/doc/reference/pairs.html?q=filter#%28def._%28%28lib._racket%2Fprivate%2Flist..rkt%29._filter%29%29"'

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] (help ...) form in command-line REPL vs. Dr. Racket

[Brian Adkins]

On Tuesday, January 26, 2016 at 6:55:26 PM UTC-5, Brian Adkins wrote:
> On Tuesday, January 26, 2016 at 6:52:14 PM UTC-5, olopierpa wrote:
> > On Wed, Jan 27, 2016 at 12:46 AM, Brian Adkins wrote:
> > 
> > > Ok, this is odd, it works with both Chrome and Safari *if* they're not 
> > > already open when (help filter) is evaluated. If they are already open, 
> > > the URL is missing the anchor.
> > 
> > In my case, the default browser is Chrome, and it was already open.
> > 
> > > I *always* have Chrome open.
> > 
> > Same here.
> 
> It's likely an OSX problem. The following works from a terminal window 
> whether Chrome is already open or not:
> 
> osascript -e 'tell application "Chrome" to open location 
> "file:///Applications/Racket%20v6.3/doc/reference/pairs.html?q=filter#%28def._%28%28lib._racket%2Fprivate%2Flist..rkt%29._filter%29%29"'

Oops, the above does *not* work if Chrome isn't open already - presumable 
because there is no application to "tell".

That seems like a pain - I guess Racket would need to 1) determine the default 
browser, 2) determine whether it's already running, then 3) if it is, use the 
above method; otherwise, use whatever it does now which seems to work fine if 
the browser *isn't* already open.


-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Re: appending files

[George Neuner]

On 1/26/2016 2:51 PM, Scotty C wrote:

gneuner2 (george), you are over thinking this thing. my test data of 1 gb is 
but a small sample file. i can't even hash that small 1 gb at the time of data 
creation. the hashed data won't fit in ram. at the time i put the redundant 
data on the hard drive, i do some constant time sorting so that the redundant 
data on the hard drive is contained in roughly 200 usefully sorted files. some 
of these files will be small and can be hashed with a single read, hash and 
write. some will be massive (data won't fit in ram) and must be split further. 
this produces another another type of single read, hash and write. these split 
files can now be fully hashed which means a second read, hash and write. 
recombining the second level files is virtually instantaneous (copy-port) 
relative to the effort spent to get to that point. all of these operations are 
constant time. it would be nice to cut into that big fat hard drive induced C 
but i can't do it with a single read and write on the larger files.



I reject the term "over thinking".

You may have created something that works, but that doesn't mean it 
can't be improved.  Remember that programs running in just 10's of 
kilobytes routinely processed data files containing 10's of megabytes.  
Modern programmers have forgotten - or never knew - how to work with 
very large data sets that don't necessarily fit into memory.


For example:  it matters greatly what is being hashed and how.  With 
Racket hashes, using a string key retains the string for comparison - so 
hashing a file containing a large percentage of unique strings produces 
a very large hash ... in the worst case the hash may be bigger than the 
data file.  If the strings are suitably long, then, e.g., 
crypto-digesting them down to ~20..32 byte values and hashing digests 
instead of strings might allow the hash to be memory resident or let you 
process much larger files using the same amount of memory.



If the records are sorted on the field(s) that may be duplicated, then 
by splitting the file and using multi-file processing techniques 
duplicates can be filtered from all the sections without hashing and in 
a single pass through the whole of the data - leaving a single sorted 
file as output.   If your input data is unordered, then as you split the 
file you sort each section individually with an in-memory sort.  The 
re-merge step - which is where the filtering is performed - sorts the 
output.


This is very efficient and it works with stupendously huge data file 
because the merge step can (more or less) simultaneously process as many 
files (split sections) as data _records_ will fit into memory.



And there are times when you should give up and embrace a DBMS, if only 
for it's proficiency at disk based data handling.


YMMV,
George

--
You received this message because you are subscribed to the Google Groups "Racket 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Re: appending files

[Neil Van Dyke]

+1 on George Neuner's comments about how one can do smart processing of 
huge files in small space.  (I almost said something about that myself, 
but didn't have time to get into that kind of discussion, so I stuck to 
only the simpler file concatenation question.)


BTW, students who have 8GB RAM and 256GB of SSD in their cheap laptop, 
and who are looking for relevance of learning how to compute with 
resource constraints, should note the current buzzwords "big data", 
"Internet of Things", etc.  (Most any 12 year-old can learn to code a 
phone/tablet app, throw up an AJAX-y Web site, or make 3D game 
assets/mods.  One of the reasons that professional software engineers 
have to keep learning is so that we can do things that 12yos can't.)


Neil V.

--
You received this message because you are subscribed to the Google Groups "Racket 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[racket-users] Re: appending files

[Scotty C]

alright george, i'm open to new ideas. here's what i've got going. running 64 
bit linux mint OS on a 2 core laptop with 2 gb of ram. my key is 128 bits with 
~256 bits per record. so my 1 gb file contains ~63 million records and ~32 
million keys. about 8% will be dupes leaving me with ~30 million keys. i run a 
custom built hash. i use separate chaining with a vector of bignums. i am 
willing to let my chains run up to 5 keys per chain so i need a vector of 6 
million pointers. that's 48 mb for the array. another 480 mb for the bignums. 
let's round that sum to .5 gb. i have another rather large bignum in memory 
that i use to reduce but not eliminate record duplication of about .5 gb. i'm 
attempting to get this thing to run in 2 places so i need 2 hashes. add this up 
.5+.5+.5 is 1.5 gb and that gets me to about my memory limit. the generated 
keys are random but i use one of the associated fields for sorting during the 
initial write to the hard drive. what goes in each of those files is totally 
random but dupes do not run across files. also, the number of keys is >1e25.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.