Re: [Factor-talk] partition-by?

[petern]

Hi Doug,

thank you very much, I'll remember this for future use. I see the
implementation is using locals as well, I guess there's just too much
variables to shuffle. IIUC this is actually more powerful than
partition-by because this has 2 values at each cycle to decide what to
do whereas partition-by has only 1.

FYI I'm still solving each day, learning a bit more of factor along the
way.

Thank you for your help

On 2019-12-04 23:37, Doug Coleman wrote:
> Here's the first example from the clojure docs for partition-by:
> 
> { 1 2 3 4 5 } [ [ 3 = not ] bi@ and ] monotonic-split
> 
> { { 1 2 } { 3 } { 4 5 } }
> 
> Also you could do this if the function is expensive so you only apply
> it once:
> { 1 2 3 4 5 } [ 3 = ] map-zip [ [ second ] bi@ = ] monotonic-split [
> keys ] map .
> 
> { { 1 2 } { 3 } { 4 5 } }
> 
> The second example is 
> { 1 1 1 2 2 3 3 } [ nip odd? ] monotonic-split .
> 
> { { 1 1 1 } { 2 2 } { 3 3 } }
> 
> Cheers,
> Doug
> 
> On Wed, Dec 4, 2019 at 3:07 PM  wrote:
> 
>> Just like last year I'm working through Advent of Code challenges.
>> Jus
>> like last year I'm probably going to fall out in the next day or two
>> :)
>> Nevertheless, I'm still holding up, and have a question.
>>
>> The 4th day part 2 quest has a test, semi-quoting: two adjacent
>> matching
>> digits that are not part of a larger group of matching digits. So
>> 112 is
>> fine (two 1's) but 111 is not. Also 11122 is fine, we're just
>> looking
>> for any occurrence of two adjacent matching digits.
>>
>> I write some clojure in my day job and there's this function
>> partition-by[1]. With a word like that the check could simply be
>>
>> digits [ ] partition-by [ length 2 = ] any?
>>
>> where digits explodes the number into a vector of its digits.
>>
>> Alas, I couldn't find such a word and after a lot of looking around
>> nothing similar. Grouping and clumping and such didn't lead me to
>> any
>> simple solution, so I just rolled something dirty:
>>
>> : -reset ( cnt curr next -- one next ) [ 2drop 1 ] dip ;
>> : -inc ( cnt curr next -- cnt+1 curr ) drop [ 1 + ] dip ;
>> : -init ( seq -- cnt curr seq ) [ 1 ] dip unclip-slice swap ;
>> : has-2? ( seq -- ? ) -init [ 2dup = not [ pick 2 = [ ] [ -reset
>> f ]
>> if ] [ -inc f ] if ] find [ 2drop 2 = ] dip or ;
>>
>> I was thinking about writing partition-by myself but I don't like
>> using
>> locals and I just wasn't able to come up with a simple solution on
>> the
>> stack.
>>
>> Thoughts/pointers welcome!
>>
>> P.S.: if anyone wants to see/compare/review my solutions so far I
>> can
>> post it somewhere, just let me know.
>>
>> [1] https://clojuredocs.org/clojure.core/partition-by
>>
>> --
>> 
>> Peter Nagy
>> 
>>
>> ___
>> Factor-talk mailing list
>> Factor-talk@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/factor-talk
> ___
> Factor-talk mailing list
> Factor-talk@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/factor-talk

-- 

  Peter Nagy



___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk

[Factor-talk] partition-by?

[petern]

Just like last year I'm working through Advent of Code challenges. Jus
like last year I'm probably going to fall out in the next day or two :)
Nevertheless, I'm still holding up, and have a question.

The 4th day part 2 quest has a test, semi-quoting: two adjacent matching
digits that are not part of a larger group of matching digits. So 112 is
fine (two 1's) but 111 is not. Also 11122 is fine, we're just looking
for any occurrence of two adjacent matching digits.

I write some clojure in my day job and there's this function
partition-by[1]. With a word like that the check could simply be

digits [ ] partition-by [ length 2 = ] any?

where digits explodes the number into a vector of its digits.

Alas, I couldn't find such a word and after a lot of looking around
nothing similar. Grouping and clumping and such didn't lead me to any
simple solution, so I just rolled something dirty:

: -reset ( cnt curr next -- one next ) [ 2drop 1 ] dip ;
: -inc ( cnt curr next -- cnt+1 curr ) drop [ 1 + ] dip ;
: -init ( seq -- cnt curr seq ) [ 1 ] dip unclip-slice swap ;
: has-2? ( seq -- ? ) -init [ 2dup = not [ pick 2 = [ ] [ -reset f ]
if ] [ -inc f ] if ] find [ 2drop 2 = ] dip or ;

I was thinking about writing partition-by myself but I don't like using
locals and I just wasn't able to come up with a simple solution on the
stack.

Thoughts/pointers welcome!

P.S.: if anyone wants to see/compare/review my solutions so far I can
post it somewhere, just let me know.


[1] https://clojuredocs.org/clojure.core/partition-by

-- 

  Peter Nagy



___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk

Re: [Factor-talk] maximum of a seq

[petern]

Hi Alex,

thanks, but I'm on Linux. Nice typo by the way, not sure what is the
hotkey on Linus :)

By clojure-like I meant what John linked, the persistent vocab. Clojure
uses what is coined as functional data structures. It all started with
Mr. Okasaki, if you search for "Okasaki functional data structures"
you'll find a lot of content. A search like that revealed an SO link
where many data structures not discussed in Okasaki's paper are linked:

https://cstheory.stackexchange.com/questions/1539/whats-new-in-purely-functional-data-structures-since-okasaki

-- 

  Peter Nagy


On 2018-12-11 21:35, Alexander Ilin wrote:
> Hi, Peter!
> 
>> Any reason why supremum-by and infimum-by are not on the "Searching
>> sequences" doc page? If you agree I can add these together with supremum
>> and infimum.
> 
> A link in the help system would be useful, I think. I remember myself
> discovering those functions by word of mouth, after having read the
> documentation multiple times. It would have been nicer if I read about
> them in the help.
> 
>> I'll take a look at map-reduce, thanks. I switched from using the factor
>> IDE REPL to emacs with Fuel because today I kept creating an infinite
>> loop and couldn't break it for the life of me in the IDE. In emacs it
>> was C-c C-c. Since I couldn't break it I didn't even know it's an
>> infinite loop. That led me to open a file and save the definitions for
>> debugging purposes, so I kind of broke my rules at this point. Oh well,
>> at least I solved another "day" today, although I'm rather behind at
>> this point, not enough time.
> 
>   If you work in Windows, there is experimental support for C-Break.
>   Here's how I enable it in my .factor-rc:
> 
> ! Enable handling of the Ctrl-Break interrupt
> USE: listener
> t handle-ctrl-break set-global
> 
>   Only works on Windows, though. There is no such thing as a global
> hotkey in Linus, AFAIK.
> 
>> Another thing that tricked me:
>>
>> IN: aoc V{ 1 2 3 } 2 head-slice* 3 suffix! >array .
>> { 1 }
>> IN: aoc V{ 1 2 3 } rest-slice 3 suffix! >array .
>> { 2 3 }
>>
>> Slices work differently than I expected :) Are there clojure-like data
>> structures that are immutable and structurally share most of their
>> content?
> 
>   Not sure about clojure-like. Arrays { } are immutable, while vectors
> V{ } are mutable.
>   You can also have read-only TUPLE: members:
> 
> TUPLE: hello-tuple
>   { var1 read-only }
>   { var2 string read-only }
> ...
> ;
> 
> ---=--- 
>  Александр
> 
> 
> 
> ___
> Factor-talk mailing list
> Factor-talk@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/factor-talk


___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk

Re: [Factor-talk] maximum of a seq

[petern]

Hi John,

thanks for taking the time to answer, look through the code, write your
own version and explain it. Your solution to reactions makes more sense,
is simpler and doesn't copy. I also learned about new-resizable, like
and with. And I like the use of ?last which simplifies the bounds check.

Well, I'd expect supremum to go by something that starts with "max", if
max is already for 2 objects than maybe maximum and maximum-by would be
nice for a sequence. Either way having it in the docs for searching
would have been enough for me to find the words.

I'll take a look at the docs and hopefully whip up a pull request.

-- 

  Peter Nagy



On 2018-12-11 22:25, John Benediktsson wrote:
> Hi Peter,
> 
> The performance difficulties in your solution arise from using
> ``head*`` which makes a new sequence (allocations, copying into new
> memory, etc.), rather than a lightweight slice which ``head-slice*``
> does.
> 
> You are right that slices are not what you expect, you might find this
> interesting:
> 
> IN: scratchpad "abcdef" dup 3 head-slice CHAR: D suffix!
> 
> --- Data stack:
> "abcDef"
> T{ slice f 0 3 "abcDef" }
> 
> In-place operations like ``set-nth`` and variations like ``suffix!``
> operate on the underlying sequence.
> 
> The Clojure-like data structures are in ``persistent`` vocabulary. 
> See, for example ``persistent.vectors``.
> 
> Some of your words like ``upper`` are in the standard library
> (``ch>upper``).  Also, I found your version didn't remove the extra
> newline in the input text, which resulted in an answer that was 1 too
> large.
> 
> This looked like a fun puzzle, and I like fun puzzles, so I thought
> I'd implement it and maybe provide another solution which could help
> with learning:
> 
> USING: ascii fry kernel math sequences ;
> 
> You can tell if two characters match for a possible reaction, if they
> are not the same, but are when made to lowercase.  I also really like
> your subtract and absolute difference should equal 32 approach.
> 
> : match? ( ch1 ch2 -- ? )
> 
> 2dup = [ 2drop f ] [ [ ch>lower ] bi@ = ] if ;
> 
> My approach uses a stack (either a ``vector`` when processing an
> array, or an ``sbuf`` when processing a string).  If the stack is
> empty (``?last`` returns ``f``), we don't react, otherwise check for a
> match on the last character in the stack.
> 
> : react? ( accum ch -- ? )
> [ ?last ] dip over [ match? ] [ 2drop f ] if ;
> 
> An internal word that processes the whole string.  If we react, then
> we pop the last character from the stack, otherwise push the new one
> onto the stack
> 
> : (react) ( accum seq -- accum )
> [ 2dup react? [ drop dup pop* ] [ suffix! ] if ] each ;
> 
> The main word that creates a new resizable sequence to use as a stack,
> then applies the reactions:
> 
> : react ( seq -- seq' )
> [ [ length ] [ new-resizable ] [ (react) ] tri ] keep like ;
> 
> The part 2 question tries removing each letter, looking for the
> smallest length of reactions:
> 
> : remove-unit ( seq ch -- seq' )
> [ [ = ] [ ch>upper = ] 2bi or ] curry reject ;
> 
> : shortest-react ( seq -- n )
> "abcdefghijklmnopqrstuvwxyz" [
> remove-unit react length
> ] with map infimum ;
> 
> I find that often I experiment with different approaches, sometimes
> using locals and writing in a more applicative style, other times
> starting with small components like ``match?`` and ``react?`` and then
> building up an algorithm around it.  I do like iteration in the
> listener, but also find saving my code and refactoring easier with a
> text file and doing F2 or ``refresh-all``, or using like you are FUEL.
> 
> Also, as for finding words.  We do want to continue improving
> documentation, so if you have contributions there it would be great. 
> It's sometimes useful to just browse a vocabulary and look at all the
> words in a list.  Maybe we could think about adding the first sentence
> of its documentation, if available, which would also help. You can see
> on the help for ``min`` and ``max`` they have related words
> ``supremum`` and ``infimum``.  Maybe those need better names...
> 
> Best,
> John.
> 
> On Tue, Dec 11, 2018 at 8:22 AM  wrote:
> 
>> On 2018-12-10 16:40, John Benediktsson wrote:
>>> Using fry is convenient.  Due to how we bootstrap factor, we can't
>> use
>>> fry right now in "core" vocabularies, like sequences, so you'll
>> see a
>>> fair amount of curry and compose which are more or less what fry
>> is
>>> doing, just not as clean syntax.
>>>
>>> I love the REPL advent idea, go for it!  And as you learn or hit
>>> roadblocks or have questions, please share.
>>>
>>> You can see how supremum-by is implemented by clicking on it in an
>>> expression, or doing ``see``.  It uses ``after?`` instead of
>> ``>``,
>>> which allows it to compare other kinds of objects with ``<=>``
>> when
>>> numbers are not passed in.
>>>
>>> 

Re: [Factor-talk] maximum of a seq

[petern]

On 2018-12-10 16:40, John Benediktsson wrote:
> Using fry is convenient.  Due to how we bootstrap factor, we can't use
> fry right now in "core" vocabularies, like sequences, so you'll see a
> fair amount of curry and compose which are more or less what fry is
> doing, just not as clean syntax.
> 
> I love the REPL advent idea, go for it!  And as you learn or hit
> roadblocks or have questions, please share.
> 
> You can see how supremum-by is implemented by clicking on it in an
> expression, or doing ``see``.  It uses ``after?`` instead of ``>``,
> which allows it to compare other kinds of objects with ``<=>`` when
> numbers are not passed in.
> 
> Also, you might try using ``map-reduce`` which is a little faster than
> using ``unclip-slice`` and ``reduce``, (and using ``_ bi@`` instead of
> ``[ _ call ] bi@`` to simplify):
> 
> : max-by ( seq quot -- result )
> [ ] swap '[ [ _ bi@ > ] 2keep ? ] map-reduce ; inline
> 
> Best,
> John.
> 
> On Mon, Dec 10, 2018 at 6:46 AM Alexander Ilin 
> wrote:
> 
>> Hey there!
>>
>> Sure, you can ask us here. I'm on sick leave, so got plenty of
>> time.
>> Coincidentally, working on my Factor hobby project.
>>
>> What you're looking for is called supremum-by.
>>
>> Terminology is a bit uncommon. Look up supremum and infimum in the
>> help system.
>>
>> 10.12.2018, 17:06, "pet...@riseup.net" :
>>> Hello fellow concatenative fans.
>>>
>>> I'm working my way through this year's advent of code[0]. I'm
>> doing it
>>> in factor with a small twist - everything happening inside the
>> REPL.
>>> Makes it more fun and challenging (for me), although I don't have
>> any
>>> code to show for it in the end.
>>>
>>> I'm trying to find the right tools for the job since there's a lot
>> of
>>> words already available. Sometimes I reinvent something before I
>> find
>>> it, e.g. I wrote a last-n before finding last*.
>>>
>>> Would it be OK if I sometimes ask you for tips on what would be
>>> idiomatic/simpler/shorter/faster than what I have(n't) found?
>>>
>>> The first problem I really couldn't find is picking a maximum from
>> a
>>> sequence based on a quotation pulling out the key. An example will
>> say
>>> it the best I guess:
>>>
 { { 1 2 } { 1 3 } { -1 4 } } [ second ] max-by .
>>>
>>> { -1 4 }
>>>
>>> Here is my definition:
>>>
>>> : max-by ( seq quot -- result ) [ unclip-slice ] dip '[ [ [ _ call
>> ] bi@
 ] 2keep ? ] reduce ; inline
>>>
>>> Using fry for such a general word would be a bit of an overkill
>> but for
>>> demonstration purposes it's OK I guess.
>>>
>>> Is there no standard word that already does something similar? Or
>> is
>>> there a way to write this with the existing sequence combinators
>> that
>>> it's so short there was no need to create this generic word?
>>>
>>> --
>>> 
>>> Peter Nagy
>>> 
>>>
>>> ___
>>> Factor-talk mailing list
>>> Factor-talk@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/factor-talk
>>
>> ---=---
>> Александр
>>
>> ___
>> Factor-talk mailing list
>> Factor-talk@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/factor-talk
> ___
> Factor-talk mailing list
> Factor-talk@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/factor-talk

Jon, Alex, hi.

Any reason why supremum-by and infimum-by are not on the "Searching
sequences" doc page? If you agree I can add these together with supremum
and infimum.

I'll take a look at map-reduce, thanks. I switched from using the factor
IDE REPL to emacs with Fuel because today I kept creating an infinite
loop and couldn't break it for the life of me in the IDE. In emacs it
was C-c C-c. Since I couldn't break it I didn't even know it's an
infinite loop. That led me to open a file and save the definitions for
debugging purposes, so I kind of broke my rules at this point. Oh well,
at least I solved another "day" today, although I'm rather behind at
this point, not enough time.

As far as roadblocks go, here is my solution to the 5th day. (react) is
ugly with t/f, that's where I had the infinite loop and just quickly
hacked the solution in. It takes quite a while to run, showing profile
output further down:

USING: arrays io.encodings.utf8 io.files kernel math math.ranges
prettyprint
sequences ;

IN: aoc
: reacts? ( a b -- ? ) - abs 32 = ;
: (react) ( seq -- seq ? ) dup [ last ] [ dup length 2 - swap nth ] bi
reacts? [ 2 head* t ] [ f ] if ;
: has-2? ( seq -- ? ) length 2 >= ;
: react ( seq -- seq ) dup has-2? [ (react) [ react ] when ] when ;
: (flow) ( seq seq -- seq seq ) unclip-slice swap [ suffix! ] dip ;
: flow ( seq seq -- seq ) dup empty? [ drop ] [ (flow) [ react ] dip
flow ] if ;
: load5 ( -- str ) "5.txt" utf8 file-contents ;
: solve5 ( seq seq -- len ) flow length ;
: upper ( c -- c ) dup 96 > [ 32 - ] when ;
: nounit ( seq u -- seq ) [ [ upper ] dip = not ] curry 

[Factor-talk] maximum of a seq

[petern]

Hello fellow concatenative fans.

I'm working my way through this year's advent of code[0]. I'm doing it
in factor with a small twist - everything happening inside the REPL.
Makes it more fun and challenging (for me), although I don't have any
code to show for it in the end.

I'm trying to find the right tools for the job since there's a lot of
words already available. Sometimes I reinvent something before I find
it, e.g. I wrote a last-n before finding last*.

Would it be OK if I sometimes ask you for tips on what would be
idiomatic/simpler/shorter/faster than what I have(n't) found?

The first problem I really couldn't find is picking a maximum from a
sequence based on a quotation pulling out the key. An example will say
it the best I guess:

> { { 1 2 } { 1 3 } { -1 4 } } [ second ] max-by .
{ -1 4 }

Here is my definition:

: max-by ( seq quot -- result ) [ unclip-slice ] dip '[ [ [ _ call ] bi@
> ] 2keep ? ] reduce ; inline


Using fry for such a general word would be a bit of an overkill but for
demonstration purposes it's OK I guess.

Is there no standard word that already does something similar? Or is
there a way to write this with the existing sequence combinators that
it's so short there was no need to create this generic word?


-- 

  Peter Nagy



___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk

Re: [Factor-talk] Shared Object File Not Found - Ubuntu 16.04

[petern]

Use the apt-get command from the wiki

https://concatenative.org/wiki/view/Factor/Requirements

On 2017-05-04 03:59, Kaveh Shahbazian wrote:
> I've found this  issue (
> https://github.com/factor/factor/issues/1241) (marked as closed) which
> suggests running factor like:
> 
> $ ./factor -run=listener
> 
> And it works.
> 
> On Wed, May 3, 2017 at 8:43 AM Kaveh Shahbazian 
> 
> wrote:
> 
>> Done that. Now we get this error:
>> $ ./factor
>> Cannot resolve C library function
>> Library: DLL" /usr/lib/libgtk-x11-2.0.so.0"
>> Symbol: gtk_init
>> DlError: none
>> See http://concatenative.org/wiki/view/Factor/Requirements
>> (U) Quotation: [ c-to-factor -> ]
>> Word: c-to-factor
>> (U) Quotation: [ [ catchstack* push ] dip call -> catchstack* pop* ]
>> (O) Word: command-line-startup
>> (O) Method: M\ gtk-ui-backend (with-ui)
>> (U) Quotation: [
>> OBJ-CURRENT-THREAD special-object error-thread set-global
>> current-continuation -> error-continuation set-global
>> [ original-error set-global ] [ rethrow ] bi
>> ]
>> 
>> 
>> On Tue, May 2, 2017 at 3:12 PM Chris Double 
>> 
>> wrote:
>> 
>>> On Tue, May 2, 2017 at 9:05 PM, Kaveh Shahbazian
>>>  wrote:
>>> >
>>> > Spent some time googling on the topic, so far failed to resolve this
>>> (not a
>>> > *nix Guru).
>>> > It seems fit to add instructions to resolve this for those who like to
>>> give
>>> > factor a spin.
>>> 
>>> I think you need:
>>> 
>>> sudo apt-get install libgtkglext1-dev
>>> 
>>> --
>>> http://bluishcoder.co.nz
>>> 
>>> 
>>> --
>>> Check out the vibrant tech community on one of the world's most
>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>> ___
>>> Factor-talk mailing list
>>> Factor-talk@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/factor-talk
>>> 
>> --
>> Regards,
>> Kaveh Shahbazian
>> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> Factor-talk mailing list
> Factor-talk@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/factor-talk

-- 

   Peter Nagy


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk

Re: [Factor-talk] Startup time

[petern]

On 2017-02-10 01:44, John Benediktsson wrote:
> Right, probably 80-85% of the startup time is in the C++ method
> factor_vm::load_image.
> 
> Breaking it down further:
> 
> - 32% of the startup time is factor_vm::load_data_heap
> - 15% of the startup time is factor_vm::load_code_heap
> - 32% of the startup time is factor_vm::fixup_heaps
> 
> It's on the list to dig into but if you'd like, you could look there 
> and
> see if there are improvements to be made.
> 
> Best,
> John.
> 
> 
> 
> 
> On Sun, Feb 5, 2017 at 7:19 AM,  wrote:
> 
>> On 2017-02-02 09:59, pet...@riseup.net wrote:
>> > On 2017-02-01 22:39, John Benediktsson wrote:
>> >> Feel free to jump in a profile startup and make some patches to
>> >> improve
>> >> things.
>> >>
>> >> Most languages seem to be under 50 milliseconds for the "startup and
>> >> run no
>> >> code" test case, so I would guess that should be fairly achievable.
>> >>
>> >> Best,
>> >> John.
>> >>
>> >> On Wed, Feb 1, 2017 at 2:10 PM,  wrote:
>> >>
>> >>> On 2017-02-01 19:40, Jim Mack wrote:
>> >>> > So why not create a separate small process that passes on its
>> >>> > parameters to
>> >>> > a running factor if it can find it, or starts a new one if it can't?
>> >>> >
>> >>>
>> >>> That's like running a server and sending requests to it. I take
>> >>> several
>> >>> issues with that:
>> >>>
>> >>> 1 - I need one instance to have *all* my libraries, present and
>> >>> future
>> >>> to be pre-loaded. But more importantly:
>> >>> 2 - a typical shell script can call a dozen external executables.
>> >>> Some
>> >>> will be in C, some in bash, some in python, some in perl etc. If
>> >>> every
>> >>> language would need a huge server to run, where would that leave us?
>> >>>
>> >>> > On Wed, Feb 1, 2017 at 7:51 AM, Timothy Hobbs 
>> wrote:
>> >>> >
>> >>> >> Have you tried loading the
>> >>> >> factor interpreter in the background and seeing if factor launches
>> >>> >> quicker while another factor process is running?
>> >>>
>> >>> I did what I think is fair - started it once so everything necessary
>> >>> gets cached in RAM and discard that run. As noted above I don't think
>> >>> running a server for each possible language is a good solution.
>> >>>
>> >>>
>> >>> Feel free to contradict me gentlemen, I'm open to discussion, but I
>> >>> do
>> >>> have my own opinion of what is acceptable and transferable to other
>> >>> PCs
>> >>> / colleagues. I'm not looking for some local hack to speed things up
>> >>> but
>> >>> a general solution that doesn't put any more burden on the end users
>> >>> than it is necessary.
>> >>>
>> >>> --
>> >>> 
>> >>>Peter Nagy
>> >>> 
>> >>>
>> >>> 
>> >>> --
>> >>> Check out the vibrant tech community on one of the world's most
>> >>> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>> >>> ___
>> >>> Factor-talk mailing list
>> >>> Factor-talk@lists.sourceforge.net
>> >>> https://lists.sourceforge.net/lists/listinfo/factor-talk
>> >>>
>> >>
>> >> 
>> --
>> >> Check out the vibrant tech community on one of the world's most
>> >> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>> >> ___
>> >> Factor-talk mailing list
>> >> Factor-talk@lists.sourceforge.net
>> >> https://lists.sourceforge.net/lists/listinfo/factor-talk
>> >
>> > I'd gladly take a look. Could you give me a hint how could I get
>> > profiling statistics of the boot? I guess most of the startup time is
>> > spent in factor code so I'd need to jack in a call to profile
>> > somewhere and rebuild?
>> 
>> I did a quick check with sysdig on the time of system calls - it shows
>> the base image takes ~50ms to load from my 180ms startup time (which
>> isn't too bad for 115MB file I guess). No other interesting syscalls, 
>> so
>> I guess the leftover 130ms are spent in factor. I'll try to take a 
>> look
>> how could I get profiling statistics on the startup of the listener.
>> 
>> --
>> 
>>Peter Nagy
>> 
>> 
>> 
>> --
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>> ___
>> Factor-talk mailing list
>> Factor-talk@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/factor-talk
>> 
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> ___
> Factor-talk mailing list
> Factor-talk@lists.sourceforge.net
> 

Re: [Factor-talk] Startup time

[petern]

On 2017-02-01 22:44, Jim Mack wrote:
> Forgive me if I sounded like I was pushing.  I am working daily in OS
> development where tools like grunt are commonly accepted, and 
> developers
> accept choices that maximize their power/ease over many other concerns. 
>  I
> never heard the slippery slope argument used against node.js, and it 
> would
> surely lose!
> 
> On Wed, Feb 1, 2017 at 2:20 PM, Timothy Hobbs  wrote:
> 
>> Just so you understand the gif, what happens is a client starts up. It
>> grabs a lock on the counter file. It reads the counter file. If the
>> counter is zero, it launches the service. If it is greater than zero,
>> then it connects to the service. After launching or connecting to the
>> service, it increments the counter, and releases the lock on the lock
>> file. When a client closes, it sends a command to the service, which
>> then grabs a lock on the lock file. Decriments the counter, and if the
>> counter is now zero, it shuts down, otherwise it merely releases the
>> lock. There are no race conditions with this method. If two clients
>> start up at the same time, they will have to wait in line for a
>> connection to the lock file. If a client shuts down at the same time 
>> as
>> another client starts up, this to means that the service and the 
>> client
>> will have to wait their turns for the lock, and therefore, no races...
>> 
>> 
>> On 02/01/2017 11:16 PM, Timothy Hobbs wrote:
>> > You can use a dbus on-demand service or your own locking mechanism, if
>> > you, like me, don't like dbus. Here is a gif which describes the process
>> > for starting and stopping a race-free on demand service
>> > http://timothy.hobbs.cz/subuser-client-service/lock-file.gif using
>> > standard lock files. You can modify this method, so that the service
>> > remains running for a certain number of seconds after the client counter
>> > has reached zero, so that in a sequential shell script, you wouldn't be
>> > launching and stopping the service over and over again.
>> >
>> > However, what I was refering to, with shared executable memory, has
>> > nothing to do with background daemons. It is a feature that is built
>> > into most modern OS kernels. Many kernels load memory pages that are
>> > marked as executable as read-only, and share those pages between
>> > processes. This greatly reduces startup times, and also improves
>> > security (marking them read only that is). It has the dissadvantage,
>> > that self modifying code is impossible. Factor, being a weird system
>> > that self-modifies itself, cannot take advantage of this mechanism at
>> > all. So we'll have to do something more advanced, like use criu, which I
>> > linked to previously.
>> >
>> > On 02/01/2017 11:10 PM, pet...@riseup.net wrote:
>> >> On 2017-02-01 19:40, Jim Mack wrote:
>> >>> So why not create a separate small process that passes on its
>> >>> parameters to
>> >>> a running factor if it can find it, or starts a new one if it can't?
>> >>>
>> >> That's like running a server and sending requests to it. I take several
>> >> issues with that:
>> >>
>> >> 1 - I need one instance to have *all* my libraries, present and future
>> >> to be pre-loaded. But more importantly:
>> >> 2 - a typical shell script can call a dozen external executables. Some
>> >> will be in C, some in bash, some in python, some in perl etc. If every
>> >> language would need a huge server to run, where would that leave us?
>> >>
>> >>> On Wed, Feb 1, 2017 at 7:51 AM, Timothy Hobbs 
>> wrote:
>> >>>
>>  Have you tried loading the
>>  factor interpreter in the background and seeing if factor launches
>>  quicker while another factor process is running?
>> >> I did what I think is fair - started it once so everything necessary
>> >> gets cached in RAM and discard that run. As noted above I don't think
>> >> running a server for each possible language is a good solution.
>> >>
>> >>
>> >> Feel free to contradict me gentlemen, I'm open to discussion, but I do
>> >> have my own opinion of what is acceptable and transferable to other PCs
>> >> / colleagues. I'm not looking for some local hack to speed things up but
>> >> a general solution that doesn't put any more burden on the end users
>> >> than it is necessary.
>> >>
>> >
>> > 
>> --
>> > Check out the vibrant tech community on one of the world's most
>> > engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>> > ___
>> > Factor-talk mailing list
>> > Factor-talk@lists.sourceforge.net
>> > https://lists.sourceforge.net/lists/listinfo/factor-talk
>> 
>> 
>> 
>> --
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>> ___
>> Factor-talk 

Re: [Factor-talk] Startup time

[petern]

On 2017-02-01 22:39, John Benediktsson wrote:
> Feel free to jump in a profile startup and make some patches to improve
> things.
> 
> Most languages seem to be under 50 milliseconds for the "startup and 
> run no
> code" test case, so I would guess that should be fairly achievable.
> 
> Best,
> John.
> 
> On Wed, Feb 1, 2017 at 2:10 PM,  wrote:
> 
>> On 2017-02-01 19:40, Jim Mack wrote:
>> > So why not create a separate small process that passes on its
>> > parameters to
>> > a running factor if it can find it, or starts a new one if it can't?
>> >
>> 
>> That's like running a server and sending requests to it. I take 
>> several
>> issues with that:
>> 
>> 1 - I need one instance to have *all* my libraries, present and future
>> to be pre-loaded. But more importantly:
>> 2 - a typical shell script can call a dozen external executables. Some
>> will be in C, some in bash, some in python, some in perl etc. If every
>> language would need a huge server to run, where would that leave us?
>> 
>> > On Wed, Feb 1, 2017 at 7:51 AM, Timothy Hobbs  wrote:
>> >
>> >> Have you tried loading the
>> >> factor interpreter in the background and seeing if factor launches
>> >> quicker while another factor process is running?
>> 
>> I did what I think is fair - started it once so everything necessary
>> gets cached in RAM and discard that run. As noted above I don't think
>> running a server for each possible language is a good solution.
>> 
>> 
>> Feel free to contradict me gentlemen, I'm open to discussion, but I do
>> have my own opinion of what is acceptable and transferable to other 
>> PCs
>> / colleagues. I'm not looking for some local hack to speed things up 
>> but
>> a general solution that doesn't put any more burden on the end users
>> than it is necessary.
>> 
>> --
>> 
>>Peter Nagy
>> 
>> 
>> 
>> --
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>> ___
>> Factor-talk mailing list
>> Factor-talk@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/factor-talk
>> 
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> ___
> Factor-talk mailing list
> Factor-talk@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/factor-talk

I'd gladly take a look. Could you give me a hint how could I get 
profiling statistics of the boot? I guess most of the startup time is 
spent in factor code so I'd need to jack in a call to profile somewhere 
and rebuild?

-- 

   Peter Nagy


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk

Re: [Factor-talk] Startup time

[petern]

On 2017-02-01 19:40, Jim Mack wrote:
> So why not create a separate small process that passes on its 
> parameters to
> a running factor if it can find it, or starts a new one if it can't?
> 

That's like running a server and sending requests to it. I take several 
issues with that:

1 - I need one instance to have *all* my libraries, present and future 
to be pre-loaded. But more importantly:
2 - a typical shell script can call a dozen external executables. Some 
will be in C, some in bash, some in python, some in perl etc. If every 
language would need a huge server to run, where would that leave us?

> On Wed, Feb 1, 2017 at 7:51 AM, Timothy Hobbs  wrote:
> 
>> Have you tried loading the
>> factor interpreter in the background and seeing if factor launches
>> quicker while another factor process is running?

I did what I think is fair - started it once so everything necessary 
gets cached in RAM and discard that run. As noted above I don't think 
running a server for each possible language is a good solution.


Feel free to contradict me gentlemen, I'm open to discussion, but I do 
have my own opinion of what is acceptable and transferable to other PCs 
/ colleagues. I'm not looking for some local hack to speed things up but 
a general solution that doesn't put any more burden on the end users 
than it is necessary.

-- 

   Peter Nagy


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk

Re: [Factor-talk] Startup time

[petern]

On 2017-02-01 15:16, John Benediktsson wrote:
> I did that and it's not a huge amount, maybe 10-20%. We need to
> profile and see what else it's doing. Like I said, low hanging fruit
> are aplenty.
> 
> 
>> On Feb 1, 2017, at 12:37 AM, pet...@riseup.net wrote:
>> 
>>> On 2017-02-01 00:19, John Benediktsson wrote:
>>> Part of the startup time is calling all of our "startup hooks", of
>>> which I
>>> have 35 right now in my Factor instance, in all of these 
>>> vocabularies:
>>> 
>>> {
>>>"alien"
>>>"destructors"
>>>"alien.strings"
>>>"io.backend"
>>>"source-files.errors"
>>>"compiler.units"
>>>"vocabs"
>>>"io.files"
>>>"vocabs.loader"
>>>"command-line"
>>>"threads"
>>>"cpu.x86.features"
>>>"io.thread"
>>>"core-foundation.run-loop"
>>>"environment"
>>>"io.backend.unix:signal-pipe-thread"
>>>"io.launcher"
>>>"random.unix"
>>>"bootstrap.random"
>>>"io.sockets:ipv6-supported?"
>>>"openssl"
>>>"tools.crossref"
>>>"cocoa"
>>>"io.files.temp"
>>>"tools.deprecation"
>>>"core-foundation"
>>>"vocabs.cache"
>>>"vocabs.refresh.monitor"
>>>"opengl.gl"
>>>"opengl"
>>>"ui"
>>>"core-text.fonts"
>>>"core-text"
>>>"tools.errors.model"
>>>"ui.tools.error-list"
>>> }
>>> 
>>> I'm sure a lot of that could be delayed in smart ways to make startup
>>> time
>>> faster, that's what I meant by low hanging fruit, but also not
>>> something
>>> that anyone is working on right now.
>>> 
>>> Best,
>>> John.
>>> 
 On Mon, Jan 30, 2017 at 12:40 PM,  wrote:
 
> On 2017-01-30 18:14, Timothy Hobbs wrote:
> If you are on linux, you might try this very new feature:
> https://criu.org/Main_Page
> 
>> On 01/30/2017 17:29, Jim Mack wrote:
>> You could also set up factor as a local web server, and send bash
>> requests to it through a web page, and it would be able to 
>> accomplish
>> them locally (or remotely) and report you the results.  It would 
>> be a
>> quick round-trip cycle with great feedback and the ability to 
>> bookmark
>> commands.
>> 
>> On Sun, Jan 29, 2017 at 1:39 PM, > > wrote:
>> 
>>>On 2017-01-29 16:20, John Benediktsson wrote:
>>> It would be nice to improve startup time. I imagine some low
>> hanging
>>> fruits if we looked deeply into it. I'll make a note to get that
>>on my
>>> list, unless someone looks into it sooner.
>>> 
>>> 
>>> 
 On Jan 28, 2017, at 2:25 PM, pet...@riseup.net
>> wrote:
 
 I spend a lot of time writing small scripts, often in bash.
>> That is
 becoming a more and more painful task as bash is a nice-enough
 language
 on the first look but in the end it seems to go out of it's way
>> to
 trip
 you up with every character you type. I don't mean to rant, I
>> know
 it's
 an old language that has to keep a lot of backward
>>compatibility and
 it
 actually still serves very well for what it was designed for,
>>as long
 as
 your script stays <100 lines. Still, there's a lot to be
>> desired.
 
 Factor is a cool language, it's very expressive, mature, has a
>>lot of
 libraries and has all sorts of tricks up its sleaves to bend it
>> to
 your
 will (much like lisp in that regard). It would be a fun
>>experiment to
 write a library or EDSL (embedded DSL) for bash-like scripting.
 However
 there's a bone to be picked:
 
 $ time bash -c ''
 bash -c ''  0.00s user 0.00s system 94% cpu 0.004 total
 
 $ time factor-vm -e=''
 factor-vm -e=''  0.12s user 0.05s system 99% cpu 0.178 total
 
 I know one can create a custom image and maybe cut down on the
>>startup
 a
 bit, but my question is - would it be possible to cut it down
>> to
 bash's
 startup time *and* still have all the necessary vocabularies in
>>it? I
 don't want to know the startup time with a small image that has
>>like
 nothing in it, I can quickly test that myself. I'd need help to
>>answer
 -
 if you imagine the use case I'm talking about, include all the
>>vocabs
 that use case would need and make all other possible
>>optimizations (if
 there are) without sacrificing too much, can the startup reach
>>similar
 times?
 
 --
 
  Peter Nagy
 
 
 
>> 
>> 
 --
 Check out the vibrant tech community on 

Re: [Factor-talk] Startup time

[petern]

On 2017-02-01 00:19, John Benediktsson wrote:
> Part of the startup time is calling all of our "startup hooks", of 
> which I
> have 35 right now in my Factor instance, in all of these vocabularies:
> 
> {
> "alien"
> "destructors"
> "alien.strings"
> "io.backend"
> "source-files.errors"
> "compiler.units"
> "vocabs"
> "io.files"
> "vocabs.loader"
> "command-line"
> "threads"
> "cpu.x86.features"
> "io.thread"
> "core-foundation.run-loop"
> "environment"
> "io.backend.unix:signal-pipe-thread"
> "io.launcher"
> "random.unix"
> "bootstrap.random"
> "io.sockets:ipv6-supported?"
> "openssl"
> "tools.crossref"
> "cocoa"
> "io.files.temp"
> "tools.deprecation"
> "core-foundation"
> "vocabs.cache"
> "vocabs.refresh.monitor"
> "opengl.gl"
> "opengl"
> "ui"
> "core-text.fonts"
> "core-text"
> "tools.errors.model"
> "ui.tools.error-list"
> }
> 
> I'm sure a lot of that could be delayed in smart ways to make startup 
> time
> faster, that's what I meant by low hanging fruit, but also not 
> something
> that anyone is working on right now.
> 
> Best,
> John.
> 
> On Mon, Jan 30, 2017 at 12:40 PM,  wrote:
> 
>> On 2017-01-30 18:14, Timothy Hobbs wrote:
>> > If you are on linux, you might try this very new feature:
>> > https://criu.org/Main_Page
>> >
>> > On 01/30/2017 17:29, Jim Mack wrote:
>> >> You could also set up factor as a local web server, and send bash
>> >> requests to it through a web page, and it would be able to accomplish
>> >> them locally (or remotely) and report you the results.  It would be a
>> >> quick round-trip cycle with great feedback and the ability to bookmark
>> >> commands.
>> >>
>> >> On Sun, Jan 29, 2017 at 1:39 PM, > >> > wrote:
>> >>
>> >> On 2017-01-29 16:20, John Benediktsson wrote:
>> >> > It would be nice to improve startup time. I imagine some low
>> >> hanging
>> >> > fruits if we looked deeply into it. I'll make a note to get that
>> >> on my
>> >> > list, unless someone looks into it sooner.
>> >> >
>> >> >
>> >> >
>> >> >> On Jan 28, 2017, at 2:25 PM, pet...@riseup.net
>> >>  wrote:
>> >> >>
>> >> >> I spend a lot of time writing small scripts, often in bash.
>> >> That is
>> >> >> becoming a more and more painful task as bash is a nice-enough
>> >> >> language
>> >> >> on the first look but in the end it seems to go out of it's way
>> >> to
>> >> >> trip
>> >> >> you up with every character you type. I don't mean to rant, I
>> >> know
>> >> >> it's
>> >> >> an old language that has to keep a lot of backward
>> >> compatibility and
>> >> >> it
>> >> >> actually still serves very well for what it was designed for,
>> >> as long
>> >> >> as
>> >> >> your script stays <100 lines. Still, there's a lot to be
>> >> desired.
>> >> >>
>> >> >> Factor is a cool language, it's very expressive, mature, has a
>> >> lot of
>> >> >> libraries and has all sorts of tricks up its sleaves to bend it
>> >> to
>> >> >> your
>> >> >> will (much like lisp in that regard). It would be a fun
>> >> experiment to
>> >> >> write a library or EDSL (embedded DSL) for bash-like scripting.
>> >> >> However
>> >> >> there's a bone to be picked:
>> >> >>
>> >> >> $ time bash -c ''
>> >> >> bash -c ''  0.00s user 0.00s system 94% cpu 0.004 total
>> >> >>
>> >> >> $ time factor-vm -e=''
>> >> >> factor-vm -e=''  0.12s user 0.05s system 99% cpu 0.178 total
>> >> >>
>> >> >> I know one can create a custom image and maybe cut down on the
>> >> startup
>> >> >> a
>> >> >> bit, but my question is - would it be possible to cut it down
>> >> to
>> >> >> bash's
>> >> >> startup time *and* still have all the necessary vocabularies in
>> >> it? I
>> >> >> don't want to know the startup time with a small image that has
>> >> like
>> >> >> nothing in it, I can quickly test that myself. I'd need help to
>> >> answer
>> >> >> -
>> >> >> if you imagine the use case I'm talking about, include all the
>> >> vocabs
>> >> >> that use case would need and make all other possible
>> >> optimizations (if
>> >> >> there are) without sacrificing too much, can the startup reach
>> >> similar
>> >> >> times?
>> >> >>
>> >> >> --
>> >> >> 
>> >> >>   Peter Nagy
>> >> >> 
>> >> >>
>> >> >>
>> >>
>> >> 
>> --
>> >> >> Check out the vibrant tech community on one of the world's most
>> >> >> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>> >> >> ___
>> >> >> Factor-talk mailing 

Re: [Factor-talk] Startup time

[petern]

On 2017-01-29 16:20, John Benediktsson wrote:
> It would be nice to improve startup time. I imagine some low hanging
> fruits if we looked deeply into it. I'll make a note to get that on my
> list, unless someone looks into it sooner.
> 
> 
> 
>> On Jan 28, 2017, at 2:25 PM, pet...@riseup.net wrote:
>> 
>> I spend a lot of time writing small scripts, often in bash. That is
>> becoming a more and more painful task as bash is a nice-enough 
>> language
>> on the first look but in the end it seems to go out of it's way to 
>> trip
>> you up with every character you type. I don't mean to rant, I know 
>> it's
>> an old language that has to keep a lot of backward compatibility and 
>> it
>> actually still serves very well for what it was designed for, as long 
>> as
>> your script stays <100 lines. Still, there's a lot to be desired.
>> 
>> Factor is a cool language, it's very expressive, mature, has a lot of
>> libraries and has all sorts of tricks up its sleaves to bend it to 
>> your
>> will (much like lisp in that regard). It would be a fun experiment to
>> write a library or EDSL (embedded DSL) for bash-like scripting. 
>> However
>> there's a bone to be picked:
>> 
>> $ time bash -c ''
>> bash -c ''  0.00s user 0.00s system 94% cpu 0.004 total
>> 
>> $ time factor-vm -e=''
>> factor-vm -e=''  0.12s user 0.05s system 99% cpu 0.178 total
>> 
>> I know one can create a custom image and maybe cut down on the startup 
>> a
>> bit, but my question is - would it be possible to cut it down to 
>> bash's
>> startup time *and* still have all the necessary vocabularies in it? I
>> don't want to know the startup time with a small image that has like
>> nothing in it, I can quickly test that myself. I'd need help to answer 
>> -
>> if you imagine the use case I'm talking about, include all the vocabs
>> that use case would need and make all other possible optimizations (if
>> there are) without sacrificing too much, can the startup reach similar
>> times?
>> 
>> --
>> 
>>   Peter Nagy
>> 
>> 
>> --
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>> ___
>> Factor-talk mailing list
>> Factor-talk@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/factor-talk
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> ___
> Factor-talk mailing list
> Factor-talk@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/factor-talk

Hi John,

you think you can reach similar startup times? That would be really 
cool. I guess your todo list is rather long though.

As a side note, are there other concatenative languages you know I could 
look at?

-- 

   Peter Nagy


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk

[Factor-talk] Startup time

[petern]

I spend a lot of time writing small scripts, often in bash. That is 
becoming a more and more painful task as bash is a nice-enough language 
on the first look but in the end it seems to go out of it's way to trip 
you up with every character you type. I don't mean to rant, I know it's 
an old language that has to keep a lot of backward compatibility and it 
actually still serves very well for what it was designed for, as long as 
your script stays <100 lines. Still, there's a lot to be desired.

Factor is a cool language, it's very expressive, mature, has a lot of 
libraries and has all sorts of tricks up its sleaves to bend it to your 
will (much like lisp in that regard). It would be a fun experiment to 
write a library or EDSL (embedded DSL) for bash-like scripting. However 
there's a bone to be picked:

$ time bash -c ''
bash -c ''  0.00s user 0.00s system 94% cpu 0.004 total

$ time factor-vm -e=''
factor-vm -e=''  0.12s user 0.05s system 99% cpu 0.178 total

I know one can create a custom image and maybe cut down on the startup a 
bit, but my question is - would it be possible to cut it down to bash's 
startup time *and* still have all the necessary vocabularies in it? I 
don't want to know the startup time with a small image that has like 
nothing in it, I can quickly test that myself. I'd need help to answer - 
if you imagine the use case I'm talking about, include all the vocabs 
that use case would need and make all other possible optimizations (if 
there are) without sacrificing too much, can the startup reach similar 
times?

-- 

   Peter Nagy


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk

Re: [Factor-talk] Web scraping

[petern]

Thank you, Bjorn, I found html.parser.analyzer to be the same way, _good 
enough_ .

Cheers

On 2016-11-19 01:42, Björn Lindqvist wrote:
> I think the reason it is parsed into a vector of start and end tags is
> because it is much simpler when not all of the html data is available.
> Or you are dealing with broken html code. There is no real XPath
> support in any Factor vocab as far as I'm aware of. I once wrote a
> half-completed binding for libxml2 (which has XPath support and a lot
> of other goodies) when I also needed it, but then I got side-tracked
> with other things I wanted to build. And the words in
> html.parser.analyzer were "good enough" for my use case. It's not so
> hard to use them to do the same kind of querying you would with XPath.
> 
> So for example, if you have the result of
> "https://news.ycombinator.com/; scrape-html nip on the stack:
> 
> //a//text() ->
> [ name>> "a" = ] find-between-all [ [ name>> text = ] filter
> [ text>> ] map " " join ] map
> 
> //@href ->
> [ "href" attribute ] map sift
> 
> //table[@class="itemlist"]/td[@class="storylink"]/(text() or @href) ->
> [ "itemlist" html-class? ] find-between-all first
> [ "storylink" html-class? ] find-between-all
> [ [ first "href" attribute ] [ second text>> ] bi 2array ] map
> 
> XPath expressions look better, but this works just fine.
> 
> 2016-11-19 0:32 GMT+01:00  :
>> Hello again :)
>> 
>> I'm looking at implemented options of scraping web pages? I've hit 
>> into
>> this
>> 
>> http://re-factor.blogspot.nl/2014/04/scraping-re-factor.html
>> 
>> but that's a json output and I'm looking at pages that only have html. 
>> I
>> see there's parse-html and scrape-html to parse a url into a vector,
>> which seems like an html tree flattened to an (event) stream. I'm left
>> to wonder about the choice as it is unusual to my eyes, but I found
>> there's a bunch of words working with the output in
>> html.parser.analyzer. I've fiddled around with it and found my way
>> around to extract some components I was looking for.
>> 
>> So now I'm wondering - is there anything else I've missed. Is there
>> something that parses html into a tree structure? Is there some 
>> simpler
>> DSL to extract data? The common cases I hit into are XPath and CSS
>> selectors, which are short and to the point, but I'm fine with w/e 
>> that
>> is easy enough and has the same power. So basically I'm just looking 
>> for
>> more tips or options in case I missed something. You guys have a lot 
>> of
>> vocabs :)
>> 
>> --
>> 
>>Peter Nagy
>> 
>> 
>> --
>> ___
>> Factor-talk mailing list
>> Factor-talk@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/factor-talk

-- 

   Peter Nagy


--
___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk

[Factor-talk] Web scraping

[petern]

Hello again :)

I'm looking at implemented options of scraping web pages? I've hit into 
this

http://re-factor.blogspot.nl/2014/04/scraping-re-factor.html

but that's a json output and I'm looking at pages that only have html. I 
see there's parse-html and scrape-html to parse a url into a vector, 
which seems like an html tree flattened to an (event) stream. I'm left 
to wonder about the choice as it is unusual to my eyes, but I found 
there's a bunch of words working with the output in 
html.parser.analyzer. I've fiddled around with it and found my way 
around to extract some components I was looking for.

So now I'm wondering - is there anything else I've missed. Is there 
something that parses html into a tree structure? Is there some simpler 
DSL to extract data? The common cases I hit into are XPath and CSS 
selectors, which are short and to the point, but I'm fine with w/e that 
is easy enough and has the same power. So basically I'm just looking for 
more tips or options in case I missed something. You guys have a lot of 
vocabs :)

-- 

   Peter Nagy


--
___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk

Re: [Factor-talk] Questions of a newcomer

[petern]

Hi Chris,

On 2016-11-10 11:03, Chris Double wrote:
> With that I see what you are seeing. Both numbers print out after the
> last thread finishes. It looks like it's buffering in this case. If I
> add a 'flush' then I see them printed after 5 seconds then 10 seconds:
> 
> -8<-
> USING: fry prettyprint kernel io namespaces sequences io.launcher
> io.directories io.encodings.utf8 io.files io.files.info io.pathnames
> concurrency.messaging threads math tools.threads accessors calendar ;
> IN: script
> 
> self '[ "bash -c \"sleep 10\"" run-process drop 1 _ send ] "1" spawn
> self '[ "bash -c \"sleep 5\"" run-process drop 2 _ send ] "2" spawn
> receive . flush
> receive . flush
> clear
> -8<-
> 
> Could this fix your issue?

Indeed that's it. Thanks for your help, I should be able to finish it 
from this point :)

-- 

   Peter Nagy


--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk

Re: [Factor-talk] Questions of a newcomer

[petern]

Hi John,

On 2016-11-08 22:38, John Benediktsson wrote:
> There are a lot of ways to solve the problem, but without knowing more
> about what you're looking for, I'll just leave these here.

I spent the last half hour reading your code, running it and examining 
the vocabularies. Very helpful, thank you for your time!

-- 

   Peter Nagy


--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk

Re: [Factor-talk] Questions of a newcomer

[petern]

Hi Chris,

On 2016-11-08 23:58, Chris Double wrote:
> I tried to duplicate the basics of your code with the following:
> 
> self '[ "bash -c \"sleep 10\"" run-process drop 1 _ send ] "1" spawn
> self '[ "bash -c \"sleep 5\"" run-process drop 2 _ send ] "2" spawn
> receive
> 
> This will spawn two threads that run a bash 'sleep'. The 'receive'
> blocks until the shortest sleep finishes and if you run another
> 'receive' then it blocks until the longer one completes. This should
> be what your code does too. Are you sure the git commands aren't all
> completing at the same time?

After a simple test you are right, the threads send the message at the 
same time. I'm still left wondering why. There's e.g. 10 threads running 
a `(thread on stack here) "git pull" run-process "pulled" send`. Now the 
git output comes to stdout intermixed and only after all threads 
finished with the run-process do they get time to send the message. Any 
more ideas why? Is run-process blocking everyone? Is there some FFI call 
like you mentioned? Where could I start to debug this on my own?

> 
> Note the use of the 'fry' quotation to avoid having to curry and swap
> later to pass the 'self' around btw.

Yes looks much cleaner, thanks. I am avoiding fry and locals and similar 
helpers for now to train the shuffling/composing more.

Thank you for your answer, much appreciated.

-- 

   Peter Nagy


--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk

Re: [Factor-talk] Questions of a newcomer

[petern]

Hi John,

On 2016-11-08 16:52, John Benediktsson wrote:
>> 
>> In that case I guess the mailing list makes more sense. Unless there's
>> people reading the IRC logs and not part of the mailing list.
>> 
> 
> The mailing list can be a fine place, or GitHub issues if you run into 
> some
> problems with Factor.  If you are worried about higher volume of
> conversation, you could reach out to me and we could have an offline
> discussion, pulling a couple of the other developers into it as well.

I'm OK with the mailing list, but let me know if you want to take the 
conversation off of it.

> 
> 
>> Thanks, I guess your explanation makes sense, `[ y ] dip` looked a bit
>> weird to me on the first look but I understand the meaning now.
>> 
> 
> There should not be a performance difference, in a simple example of `[ 
> y ]
> dip` versus `y swap``, its based on a sense of which one looks cleaner 
> or
> more closely expresses the purpose of the code.  You can find other 
> similar
> examples, like `[ foo ] keep` versus `[ foo ] [ ] bi` where developers 
> are
> experimenting with and thinking about expressivity in concatenative
> languages.

OK, guess I'll experiment as well, thanks.

> 
> 
>> > I've seen discussions on this mailing list about the extra/cursors
>> > vocabulary about that. I've never used it though. For example Joe
>> > talked
>> > about it in 2009:
>> > https://sourceforge.net/p/factor/mailman/message/23878123
>> >
>> 
>> I see, so nothing that would be alive.
>> 
> 
> Well, possibly thats one way to look at it.  Another is that it is/was 
> an
> experiment in thinking about generalized iterators and has tests and 
> works
> fine for what it is.  You can see the code in extra/cursors or use it
> (`USE: cursors`).  We do need to move towards lazy/infinite
> generator/iterators and hopefully to more easily generalize
> iteration/operations on those as well as sequences (`each`) and assocs
> (`assoc-each`) and dlists (`dlist-each`) and lists (`leach`) and heaps,
> trees, etc etc.  Doug Coleman did some work on this, but we haven't
> finished it yet.

I looked into the cursors vocab, sadly it doesn't have any 
documentation. But I'll dig into it if the need arises.

> 
> In the meantime I finished a small script where I was trying out factor
>> - it goes into my git folders and fetches/pulls updates from origin. I
>> got the first version working pretty quickly, then I wanted to tweak 
>> it
>> so that the updates run in parallel and output updates on the console
>> live. In a final version I wanted to print some progressbar while
>> surpressing git output. For that reason I thought I need message
>> passing. This isn't working as I expect it to. Here is the gist:
>> 
>> https://gist.github.com/xificurC/f4de1993b3218a50dd8936dfc0ec16f2
>> 
>> There are my last 2 attemps. The first, commented out version finishes
>> without waiting for the threads to finish (even with the ugly hack of
>> reading the state>> of the thread) while in the second the receiving
>> thread doesn't read the messages as they arrive, rather its mailbox 
>> gets
>> filled up and only when everyone finishes gets time to work on the
>> messages. What am I doing wrong?
>> 
> 
> I would think it would be easier to use some form of ``parallel-map`` 
> from
> `USE: concurrency.combinators`.  But if you want to build it up from 
> other
> concurrency libraries, you might think of a count-down latch (`USE:
> concurrency.count-downs`) and waiting for it to reach zero, since you 
> know
> how many items to wait for.

I thought of parallel-map first but I wanted to do a bit more than that, 
otherwise I could just write a script that handles one and feed it to 
GNU parallel. I wanted to achieve more than what I can with GNU parallel 
(which is able to parallelize on multiple cores and writes correctly to 
stdout by not mixing the outputs of several tasks). Since I know the 
count ahead I wanted to suppress the git output and instead show a 
progress bar with some results, e.g. if the pull/fetch was successful or 
not. The only way I can imagine that is by message passing. But as I 
explained the threads don't seem to work as I expect, the recieving 
thread doesn't get to "speak up" until it's too long. Is there any way 
one can make a particular thread higher priority or come up right when 
it gets a message? Also, when does factor end? I have a bunch of threads 
that are waiting for I/O or network response and it finishes anyway. My 
problem with threads is that I don't understand what's happening behind 
the scenes, who goes when, how many of them at once, etc.

>  You might also want to limit the number of
> threads running by using a semaphore (`USE: concurrency.semaphores`) if 
> not
> using parallel-map on groups of items.

Would that help to keep the receiver alive? Like if there's a total of 4 
threads running I put a semaphore on 3 and I get one always through? The 
problem with this example concretely is that I'm waiting on 

Re: [Factor-talk] Questions of a newcomer

[petern]

Hi Jon,

On 2016-11-07 22:34, Jon Harper wrote:
> Hi Peter,
> 
> On Mon, Nov 7, 2016 at 3:07 PM,  wrote:
> 
>> Hello,
>> 
>> I am tinkering with factor and was wondering if it is OK to pick your
>> brains here? As I play around with the language questions come up that
>> are probably easy for you to answer. I don't see much action on the
>> #concatenative IRC channel so I thought the maling list might be a
>> better place?
>> 
>> Questions are always welcome. The mailing list or #concatenative on 
>> IRC
> are both good places to ask. Several people read the logs of 
> #concatenative
> and may answer your questions some time after you sent it, so don't 
> give up.

In that case I guess the mailing list makes more sense. Unless there's 
people reading the IRC logs and not part of the mailing list.

> 
> As a starter:
>> 
>> - I see a common pattern in definitions of using `dip` instead of
>> `swap`. Is there some special reason for that? Is it more performant? 
>> I
>> know the words aren't interchangeable but e.g. `with-directory-files`
>> has `[ [ "" directory-files ] ] dip` which as far as I can tell is
>> equivalent to `[ "" directory-files ] swap`. I saw this pattern in 
>> more
>> definitions.
>> 
> I guess it's a matter of personal style. I would argue that the 
> 'meaning'
> of swap ( x y -- y x ) is that there's the same importance on pushing y
> down the stack than on puling x up the stack. Whereas [ y ] dip would 
> focus
> more putting y down the stack.
> 
> Regarding the performance, you can often see for yourself using the
> optimized. word of compiler.tree.debugger. You could even install 
> libudis
> and use the disassemble word of tools.disassembler.
> I would be surprised if the performance of swap vs dip mattered in a 
> real
> application.
> 

Thanks, I guess your explanation makes sense, `[ y ] dip` looked a bit 
weird to me on the first look but I understand the meaning now.

> 
>> 
>> - is there any sequence generator/iterator vocabulary? Something that
>> gives or computes values on demand. One can find it in many languages
>> with a bit different flavor, e.g. Scheme, Rust, Python. I saw that 
>> there
>> is lists.lazy which can serve a similar purpose but is a bit more 
>> heavy
>> weight in some cases. Maybe this isn't needed in factor at all and you
>> use a different pattern to solve a similar problem?
>> 
> I've seen discussions on this mailing list about the extra/cursors
> vocabulary about that. I've never used it though. For example Joe 
> talked
> about it in 2009: 
> https://sourceforge.net/p/factor/mailman/message/23878123
> 

I see, so nothing that would be alive.



In the meantime I finished a small script where I was trying out factor 
- it goes into my git folders and fetches/pulls updates from origin. I 
got the first version working pretty quickly, then I wanted to tweak it 
so that the updates run in parallel and output updates on the console 
live. In a final version I wanted to print some progressbar while 
surpressing git output. For that reason I thought I need message 
passing. This isn't working as I expect it to. Here is the gist:

https://gist.github.com/xificurC/f4de1993b3218a50dd8936dfc0ec16f2

There are my last 2 attemps. The first, commented out version finishes 
without waiting for the threads to finish (even with the ugly hack of 
reading the state>> of the thread) while in the second the receiving 
thread doesn't read the messages as they arrive, rather its mailbox gets 
filled up and only when everyone finishes gets time to work on the 
messages. What am I doing wrong?

On the topic of factor's cooperative threads - do they run multi- or 
single-core?

If you have some other tips on the code I'll be glad, I feel like I'm 
doing more shuffling then I might need.

Thank you for your answer.

> Cheers,
> Jon
> 
> --
> Developer Access Program for Intel Xeon Phi Processors
> Access to Intel Xeon Phi processor-based developer platforms.
> With one year of Intel Parallel Studio XE.
> Training and support from Colfax.
> Order your platform today. http://sdm.link/xeonphi
> ___
> Factor-talk mailing list
> Factor-talk@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/factor-talk

-- 

   Peter Nagy


--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk

[Factor-talk] Questions of a newcomer

[petern]

Hello,

I am tinkering with factor and was wondering if it is OK to pick your 
brains here? As I play around with the language questions come up that 
are probably easy for you to answer. I don't see much action on the 
#concatenative IRC channel so I thought the maling list might be a 
better place?

As a starter:

- I see a common pattern in definitions of using `dip` instead of 
`swap`. Is there some special reason for that? Is it more performant? I 
know the words aren't interchangeable but e.g. `with-directory-files` 
has `[ [ "" directory-files ] ] dip` which as far as I can tell is 
equivalent to `[ "" directory-files ] swap`. I saw this pattern in more 
definitions.

- is there any sequence generator/iterator vocabulary? Something that 
gives or computes values on demand. One can find it in many languages 
with a bit different flavor, e.g. Scheme, Rust, Python. I saw that there 
is lists.lazy which can serve a similar purpose but is a bit more heavy 
weight in some cases. Maybe this isn't needed in factor at all and you 
use a different pattern to solve a similar problem?

All in all the language is quite fun, I like it when one has to fight 
his own brain to get something done because one often learns so much 
along the way.

Any tips welcome, thank you

-- 

   Peter Nagy


--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk