Re: [9fans] command repetition in sam/acme
Sam and Acme use a simple, pure form of regular expressions. If they
had the counting operations, this would be a trivial task, but to add
them would open the door to the enormous, ill-conceived complexity of
(no longer) regular expressions as the open source community thinks of
them.
Is it really so? R. Cox (Regular Expression Matching Can Be Simple And
Fast), I think, shows, that repetition can be first expanded and then
used even by the nice (non-backtracking) algorithms, like this:
e{3} -- eee
e{3,5} -- ?e?
e{3,} -- eee+
where would the problem arise?
Thanks
Ruda
So yes: use other tools, with my apologies.
-rob
Re: [9fans] threads vs forks
John Barham wrote: On Tue, Mar 3, 2009 at 3:52 AM, hugo rivera uai...@gmail.com wrote: I have to launch many tasks running in parallel (~5000) in a cluster running linux. Each of the task performs some astronomical calculations and I am not pretty sure if using fork is the best answer here. First of all, all the programming is done in python and c... Take a look at the multiprocessing package (http://docs.python.org/library/multiprocessing.html), newly introduced with Python 2.6 and 3.0: multiprocessing is a package that supports spawning processes using an API similar to the threading module. The multiprocessing package offers both local and remote concurrency, effectively side-stepping the Global Interpreter Lock by using subprocesses instead of threads. It should be a quick and easy way to set up a cluster-wide job processing system (provided all your jobs are driven by Python). Better: use parallelpython (www.parallelpython.org). Afaik multiprocessing is geared towards multi-core systems (one machine), while pp is also suitable for real clusters with more pc's. No special cluster software needed. It will start (here's your fork) a (some) python interpreters on each node, and then you can submit jobs to those 'workers'. The interpreters are kept alive between jobs, so the startup penalty becomes neglectibly when the number of jobs is large enough. Using it here to process massive amounts of satellite data, works like a charm. Vincent. It also looks like it's been (partially?) back-ported to Python 2.4 and 2.5: http://pypi.python.org/pypi/processing. John
Re: [9fans] command repetition in sam/acme
Not all the features adapt as easily. -rob
Re: [9fans] threads vs forks
Thanks for the advice. Nevertheless I am in no position to decide what pieces of software the cluster will run, I just have to deal with what I have, but anyway I can suggest other possibilities. 2009/3/4, Vincent Schut sc...@sarvision.nl: John Barham wrote: On Tue, Mar 3, 2009 at 3:52 AM, hugo rivera uai...@gmail.com wrote: I have to launch many tasks running in parallel (~5000) in a cluster running linux. Each of the task performs some astronomical calculations and I am not pretty sure if using fork is the best answer here. First of all, all the programming is done in python and c... Take a look at the multiprocessing package (http://docs.python.org/library/multiprocessing.html), newly introduced with Python 2.6 and 3.0: multiprocessing is a package that supports spawning processes using an API similar to the threading module. The multiprocessing package offers both local and remote concurrency, effectively side-stepping the Global Interpreter Lock by using subprocesses instead of threads. It should be a quick and easy way to set up a cluster-wide job processing system (provided all your jobs are driven by Python). Better: use parallelpython (www.parallelpython.org). Afaik multiprocessing is geared towards multi-core systems (one machine), while pp is also suitable for real clusters with more pc's. No special cluster software needed. It will start (here's your fork) a (some) python interpreters on each node, and then you can submit jobs to those 'workers'. The interpreters are kept alive between jobs, so the startup penalty becomes neglectibly when the number of jobs is large enough. Using it here to process massive amounts of satellite data, works like a charm. Vincent. It also looks like it's been (partially?) back-ported to Python 2.4 and 2.5: http://pypi.python.org/pypi/processing. John -- Hugo
Re: [9fans] threads vs forks
hugo rivera wrote: Thanks for the advice. Nevertheless I am in no position to decide what pieces of software the cluster will run, I just have to deal with what I have, but anyway I can suggest other possibilities. Well, depends on how you define 'software the cluster will run'. Do you mean cluster management software, or really any program or script or python module that needs to be installed on each node? Because for pp, you won't need any cluster software. pp is just some python module and helper scripts. You *do* need to install this (pure python) module on each node, yes, but that's it, nothing else needed. Btw, you said 'it's a small cluster, about 6 machines'. Now I'm not an expert, but I don't think you can do threading/forking from one machine to another (on linux). So I suppose there already is some cluster management software involved? And while you appear to be in no position to decide what pieces of software the cluster will run, you might want to enlighten us on what this cluster /will/ run? Your best solution might depend on that... Cheers, Vincent.
Re: [9fans] threads vs forks
hugo rivera wrote: The cluster has torque installed as the resource manager. I think it runs of top of pbs (an older project). As far as I know now I just have to call a qsub command to submit my jobs on a queue, then the resource manager allocates a processor in the cluster for my process to run till is finished. Well, I don't know torque neither pbs, but I'm guessing that when you submit a job, this job will be some program or script that is run on the allocated processor? If so, your initial question of forking vs threading is bogus. Your cluster manager will run (exec) your job, which if it is a python script will start a python interpreter for each job. I guess that's the overhead you get when running a flexible cluster system, flexible meaning that it can run any type of job (shell script, binary executable, python script, perl, etc.). However, your overhead of starting new python processes each time may seem significant when viewed in absolute terms, but if each job processes lots of data and takes, as you said, 5 min to run on a decent processor, don't you think the startup time for the python process would become non-significant? For example, on a decent machine here, the first time python takes 0.224 secs to start and shutdown immediately, and consequetive starts take only about 0.009 secs because everything is still in memory. Let's take the 0.224 secs for a worst case scenario. That would be approx 0.075 percent of your job execution time. Now lets say you have 6 machines with 8 cores each and perfect scaling, all your jobs would take 6000 / (6*8) *5min = 625 minutes (10 hours 25 mins) without python starting each time, and 625 minutes and 28 seconds with python starting anew each job. Don't you think you could just live with these 28 seconds more? Just reading this message might already have taken you more than those 28 seconds... Vincent. And I am not really sure if I have access to all the nodes, so I can install pp on each one of them. 2009/3/4, Vincent Schut sc...@sarvision.nl: hugo rivera wrote: Thanks for the advice. Nevertheless I am in no position to decide what pieces of software the cluster will run, I just have to deal with what I have, but anyway I can suggest other possibilities. Well, depends on how you define 'software the cluster will run'. Do you mean cluster management software, or really any program or script or python module that needs to be installed on each node? Because for pp, you won't need any cluster software. pp is just some python module and helper scripts. You *do* need to install this (pure python) module on each node, yes, but that's it, nothing else needed. Btw, you said 'it's a small cluster, about 6 machines'. Now I'm not an expert, but I don't think you can do threading/forking from one machine to another (on linux). So I suppose there already is some cluster management software involved? And while you appear to be in no position to decide what pieces of software the cluster will run, you might want to enlighten us on what this cluster /will/ run? Your best solution might depend on that... Cheers, Vincent.
Re: [9fans] command repetition in sam/acme
2009/3/4 Rob Pike robp...@gmail.com:
Not all the features adapt as easily.
-rob
By counted repetittion I've always meant just the mentioned, i.e.
{n}
{n,}
{,m}
{n,m}
.
What feature do you have on mind?
Thanks
Ruda
Re: [9fans] threads vs forks
you are right. I was totally confused at the beggining. Thanks a lot. 2009/3/4, Vincent Schut sc...@sarvision.nl: hugo rivera wrote: The cluster has torque installed as the resource manager. I think it runs of top of pbs (an older project). As far as I know now I just have to call a qsub command to submit my jobs on a queue, then the resource manager allocates a processor in the cluster for my process to run till is finished. Well, I don't know torque neither pbs, but I'm guessing that when you submit a job, this job will be some program or script that is run on the allocated processor? If so, your initial question of forking vs threading is bogus. Your cluster manager will run (exec) your job, which if it is a python script will start a python interpreter for each job. I guess that's the overhead you get when running a flexible cluster system, flexible meaning that it can run any type of job (shell script, binary executable, python script, perl, etc.). However, your overhead of starting new python processes each time may seem significant when viewed in absolute terms, but if each job processes lots of data and takes, as you said, 5 min to run on a decent processor, don't you think the startup time for the python process would become non-significant? For example, on a decent machine here, the first time python takes 0.224 secs to start and shutdown immediately, and consequetive starts take only about 0.009 secs because everything is still in memory. Let's take the 0.224 secs for a worst case scenario. That would be approx 0.075 percent of your job execution time. Now lets say you have 6 machines with 8 cores each and perfect scaling, all your jobs would take 6000 / (6*8) *5min = 625 minutes (10 hours 25 mins) without python starting each time, and 625 minutes and 28 seconds with python starting anew each job. Don't you think you could just live with these 28 seconds more? Just reading this message might already have taken you more than those 28 seconds... Vincent. And I am not really sure if I have access to all the nodes, so I can install pp on each one of them. 2009/3/4, Vincent Schut sc...@sarvision.nl: hugo rivera wrote: Thanks for the advice. Nevertheless I am in no position to decide what pieces of software the cluster will run, I just have to deal with what I have, but anyway I can suggest other possibilities. Well, depends on how you define 'software the cluster will run'. Do you mean cluster management software, or really any program or script or python module that needs to be installed on each node? Because for pp, you won't need any cluster software. pp is just some python module and helper scripts. You *do* need to install this (pure python) module on each node, yes, but that's it, nothing else needed. Btw, you said 'it's a small cluster, about 6 machines'. Now I'm not an expert, but I don't think you can do threading/forking from one machine to another (on linux). So I suppose there already is some cluster management software involved? And while you appear to be in no position to decide what pieces of software the cluster will run, you might want to enlighten us on what this cluster /will/ run? Your best solution might depend on that... Cheers, Vincent. -- Hugo
[9fans] acme Put doesn't save
Hello I am running p9p acme. I open a directory, so in the tag line I have sth. ending with '/' and in the window I have the list of files in that directory. Now I go to the tag line and append a name, say 'a', to the existing path, i.e. now I have there e.g. '/home/ruda/a'. Then I write Put somewhere and try to middle click it thinking the window contents gets saved to '/home/ruda/a'. But it does not. Why? Thanks Ruda
Re: [9fans] 8c, 8l, and empty files
Garbage-in, Garbage-out What does it happen when you try to link an object file that does not have main() defined? And when you try to link a malformed object file? In these cases, there is not any garbage-out for the garbage-in. There is an error, as expected. two things: the linker doesn't only produce binaries, it has options for producing other output in which a null object file may be applicable; In most cases it is used to create a binary. furthermore, it takes more than a single file, so you can see how a #ifdef-ed C file compiles to nothing (even if it's bad practice) but then is linked with other parts of the program just fine. In the example, I was linking a *unique* empty object file. To create a 0 byte exec file, that is, a binary without a header, is a nonsense. Regards, Q.
Re: [9fans] 8c, 8l, and empty files
viewing the source code, i suspect it's an oversight rather than essential design.
Re: [9fans] acme Put doesn't save
2009/3/4 Rudolf Sykora rudolf.syk...@gmail.com: Hello I am running p9p acme. I open a directory, so in the tag line I have sth. ending with '/' and in the window I have the list of files in that directory. Now I go to the tag line and append a name, say 'a', to the existing path, i.e. now I have there e.g. '/home/ruda/a'. Then I write Put somewhere and try to middle click it thinking the window contents gets saved to '/home/ruda/a'. But it does not. Why? Thanks Ruda Because at the beginning of the put function in exec.c you have: if(et==nil || et-w==nil || et-w-isdir) return; and et-w-isdir is true. But there are workarounds if you want to save the contents of the window. -- - yiyus || JGL .
Re: [9fans] acme Put doesn't save
i reckon this could be classed as a bug. you can't change a directory window to a file window. 2009/3/4 Rudolf Sykora rudolf.syk...@gmail.com: Hello I am running p9p acme. I open a directory, so in the tag line I have sth. ending with '/' and in the window I have the list of files in that directory. Now I go to the tag line and append a name, say 'a', to the existing path, i.e. now I have there e.g. '/home/ruda/a'. Then I write Put somewhere and try to middle click it thinking the window contents gets saved to '/home/ruda/a'. But it does not. Why? Thanks Ruda
Re: [9fans] acme Put doesn't save
Because at the beginning of the put function in exec.c you have: if(et==nil || et-w==nil || et-w-isdir) return; and et-w-isdir is true. But there are workarounds if you want to save the contents of the window. Well, I had some text in in-this-way created window and I only grabbed the window trying to place it in another column and suddenly it was all empty... :( All my text not only not saved, but lost completely... This isn't nice. So is there a workaround other than 1) make a new window, 2) copy past the text there 3) save from the new window? I thought that the decision when it saves and when not should be directed by the name format: ending with / then not save, ending with sth sane, save... Ruda
Re: [9fans] acme Put doesn't save
So is there a workaround other than 1) make a new window, 2) copy past the text there 3) save from the new window? 4) New 5) B. - erik
Re: [9fans] threads vs forks
What about xcpu? On Wed, Mar 4, 2009 at 12:33 PM, hugo rivera uai...@gmail.com wrote: you are right. I was totally confused at the beggining. Thanks a lot. 2009/3/4, Vincent Schut sc...@sarvision.nl: hugo rivera wrote: The cluster has torque installed as the resource manager. I think it runs of top of pbs (an older project). As far as I know now I just have to call a qsub command to submit my jobs on a queue, then the resource manager allocates a processor in the cluster for my process to run till is finished. Well, I don't know torque neither pbs, but I'm guessing that when you submit a job, this job will be some program or script that is run on the allocated processor? If so, your initial question of forking vs threading is bogus. Your cluster manager will run (exec) your job, which if it is a python script will start a python interpreter for each job. I guess that's the overhead you get when running a flexible cluster system, flexible meaning that it can run any type of job (shell script, binary executable, python script, perl, etc.). However, your overhead of starting new python processes each time may seem significant when viewed in absolute terms, but if each job processes lots of data and takes, as you said, 5 min to run on a decent processor, don't you think the startup time for the python process would become non-significant? For example, on a decent machine here, the first time python takes 0.224 secs to start and shutdown immediately, and consequetive starts take only about 0.009 secs because everything is still in memory. Let's take the 0.224 secs for a worst case scenario. That would be approx 0.075 percent of your job execution time. Now lets say you have 6 machines with 8 cores each and perfect scaling, all your jobs would take 6000 / (6*8) *5min = 625 minutes (10 hours 25 mins) without python starting each time, and 625 minutes and 28 seconds with python starting anew each job. Don't you think you could just live with these 28 seconds more? Just reading this message might already have taken you more than those 28 seconds... Vincent. And I am not really sure if I have access to all the nodes, so I can install pp on each one of them. 2009/3/4, Vincent Schut sc...@sarvision.nl: hugo rivera wrote: Thanks for the advice. Nevertheless I am in no position to decide what pieces of software the cluster will run, I just have to deal with what I have, but anyway I can suggest other possibilities. Well, depends on how you define 'software the cluster will run'. Do you mean cluster management software, or really any program or script or python module that needs to be installed on each node? Because for pp, you won't need any cluster software. pp is just some python module and helper scripts. You *do* need to install this (pure python) module on each node, yes, but that's it, nothing else needed. Btw, you said 'it's a small cluster, about 6 machines'. Now I'm not an expert, but I don't think you can do threading/forking from one machine to another (on linux). So I suppose there already is some cluster management software involved? And while you appear to be in no position to decide what pieces of software the cluster will run, you might want to enlighten us on what this cluster /will/ run? Your best solution might depend on that... Cheers, Vincent. -- Hugo
Re: [9fans] acme Put doesn't save
2009/3/4 Rudolf Sykora rudolf.syk...@gmail.com: So is there a workaround other than 1) make a new window, 2) copy past the text there 3) save from the new window? The easier solution that comes to my mind now is Edit w filename. I haven't looked into it, but I think you lost the content of your window because when directory windows are resized acme columnates them again (which is a nice thing, if you ask me). It would be good that the isdir flag of a window changed when you edit its name. I will have a look (playing with acme source is like my new hobby...). It is not too consistent now: you can remove the basename and then do 'Get', but you cannot add it again and do 'Put' (I'm talking by memory. BTW, everything works as expected in acme-sac, the only acme I can use at work). -- - yiyus || JGL .
Re: [9fans] command repetition in sam/acme
From earlier in this thread:
Sam and Acme use a simple, pure form of regular expressions. If they
had the counting operations, this would be a trivial task, but to add
them would open the door to the enormous, ill-conceived complexity of
(no longer) regular expressions as the open source community thinks of
them.
So, if you want counting, you can always write a patch, who knows, it
might even be accepted.
But I think that anyone not under the influence of psychedelic
substances that has suffered PCR, will agree we don't want to move in
that direction, and even if small, counting is a step in that
direction.
I personally rarely have use for it, and when I do, it is trivial to
write a script to generate the desired regexp, and I'm eternally
grateful for being free from Perl-induced psychosis.
uriel
On Wed, Mar 4, 2009 at 12:32 PM, Rudolf Sykora rudolf.syk...@gmail.com wrote:
2009/3/4 Rob Pike robp...@gmail.com:
Not all the features adapt as easily.
-rob
By counted repetittion I've always meant just the mentioned, i.e.
{n}
{n,}
{,m}
{n,m}
.
What feature do you have on mind?
Thanks
Ruda
Re: [9fans] command repetition in sam/acme
Sam and Acme use a simple, pure form of regular expressions. If they had the counting operations, this would be a trivial task, but to add them would open the door to the enormous, ill-conceived complexity of (no longer) regular expressions as the open source community thinks of them. So, if you want counting, you can always write a patch, who knows, it might even be accepted. But I think that anyone not under the influence of psychedelic substances that has suffered PCR, will agree we don't want to move in that direction, and even if small, counting is a step in that direction. I personally rarely have use for it, and when I do, it is trivial to write a script to generate the desired regexp, and I'm eternally grateful for being free from Perl-induced psychosis. uriel Isn't it just easier to answer the questions? ... this is just like bla... bla... bla... sorry, but being such doesn't shed any light on the subject... Ruda
Re: [9fans] acme Put doesn't save
While learning some of acme's basic commands I was also shocked by this, and I agree with Ruda that executing a Put with a non-existent file (e.g. '/home/ruda/a' ;-) should save the contents of the window in that file. Probably there are other ways to do it, like Edit w /home/ruda/a, but the ones I've seen so far involve a little more typing. Is it just out of the question to implement Ruda's suggestion? (if it was a suggestion at all) It is just a little more comfortable, and it fits very natural into acme, if you ask me. 2009/3/4, erik quanstrom quans...@quanstro.net: So is there a workaround other than 1) make a new window, 2) copy past the text there 3) save from the new window? 4) New 5) B. - erik -- Hugo
Re: [9fans] command repetition in sam/acme
this is plan 9. we don't ask if new feature x would not cause a problem, we ask if x would make plan 9 a better system. well, no-one disputes the claim, if you read twice. i'm quite sure one would be wrong in assuming that plan 9's designers did not know about repetition. i think it would be safer to assume they were not keen on the idea. I don't assume anything, while you do. What if they just didn't need it and followed the idea of simplicity? The result of this discussion basically has been: neither acme nor sam is suited for the original problem, there is no simple way present in plan9 allowing you to edit such files with long lines, which are quite commonly and with justification present in the world. Only Vim, which was ported to plan9, with its regexps, can do it gracefully. Thus if you want to edit a file, you are forced to use bloated regexps present in Vim. Isn't it sad being in plan9?! Things should be simple, but not simpler than that. Ruda
Re: [9fans] acme Put doesn't save
On Wed Mar 4 09:12:08 EST 2009, uai...@gmail.com wrote: While learning some of acme's basic commands I was also shocked by this, and I agree with Ruda that executing a Put with a non-existent file (e.g. '/home/ruda/a' ;-) should save the contents of the window in that file. this isn't true. if you click on New and then type something for a filename you will notice that the tag line box becomes filled and then click on Put, the file *will* be saved, even if the file does not exist. if on the other hand you try to modify the name of a directory listing frame, the tag line box is not filled. this is a hint that you're doing something wrong. also there is no Put option. this is a bigger hint that you're doing something wrong. the reason that directories behave differently is that in acme programs are allowed to take over a window. while the directory listing is built in, it operates on the same principle, and it does not make sense to write a directory through acme's interface. - erik
Re: [9fans] command repetition in sam/acme
The result of this discussion basically has been: neither acme nor sam is suited for the original problem, there is no simple way present in plan9 allowing you to edit such files with long lines, which are quite commonly and with justification present in the world. Only Vim, which was ported to plan9, with its regexps, can do it gracefully. Thus if you want to edit a file, you are forced to use bloated regexps present in Vim. Isn't it sad being in plan9?! Things should be simple, but not simpler than that. i disagree with your premise that only vim has the vigor to modify your super special file. all you need is f and f*. f transforms your source into something easy to edit in acme. f* transforms it back into the original form. easy peasy. or, you can write a simple program that edits the file directly. there are so many ways to do this, and i'm just too lazy to list them all. - erik
Re: [9fans] threads vs forks
On Wed, Mar 4, 2009 at 2:30 AM, Vincent Schut sc...@sarvision.nl wrote: hugo rivera wrote: Now I'm not an expert, but I don't think you can do threading/forking from one machine to another (on linux). You can with bproc, but it's not supported past 2.6.21 or so. ron
Re: [9fans] command repetition in sam/acme
i disagree with your premise that only vim has the vigor to modify your super special file. all you need is f and f*. f transforms your source into something easy to edit in acme. f* transforms it back into the original form. easy peasy. or, you can write a simple program that edits the file directly. there are so many ways to do this, and i'm just too lazy to list them all. - erik Alright. Such 'f, f*' has already been proposed and shown, most clearly, by yy at the beginning of this thread. I agree it is functional and perhaps sufficient for now. Let's consider this finished. (Until I come up with some even more compelling argument in favor of counted repetition...) Ruda
Re: [9fans] acme Put doesn't save
if on the other hand you try to modify the name of a directory listing frame, the tag line box is not filled. this is a hint that you're doing something wrong. also there is no Put option. this is a bigger hint that you're doing something wrong. You may call it a hint. I noticed the 'Put' doesn't appear there. But (non-)existence of sth in the tag line doesn't mean it can't be issued. Moreover, I still can't see why it can't work. If I change the tag line by appending a name (which is not, say, a name of a directory within the current directory), I don't understand why this couldn't be saved... and the window couldn't represent the new (or changed) file from now on. the reason that directories behave differently is that in acme programs are allowed to take over a window. while the directory listing is built in, it operates on the same principle, and it does not make sense to write a directory through acme's interface. - erik I don't get what you mean... Ruda
Re: [9fans] acme Put doesn't save
2009/3/4 erik quanstrom quans...@quanstro.net: if on the other hand you try to modify the name of a directory listing frame, the tag line box is not filled. this is a hint that you're doing something wrong. also there is no Put option. this is a bigger hint that you're doing something wrong. not really - +Errors windows fulfil both of those criteria, but Put works fine in those. the reason that directories behave differently is that in acme programs are allowed to take over a window. while the directory listing is built in, it operates on the same principle, and it does not make sense to write a directory through acme's interface. i agree it doesn't make sense to write a directory, but you can't anyway, even if the interface allowed it, and the resulting error message would be self-expanatory. personally, i think that Put should work on any non-application window, and that re-columnation should only take place if the textual content hasn't been modified by the user. (and probably also that if you change the name of a window to a directory name and do Get, that it would get a directory listing). but YMMV as always.
Re: [9fans] 8c, 8l, and empty files
On Wed, Mar 4, 2009 at 12:43 AM, Anthony Sorace ano...@gmail.com wrote: i could see this going either way, but from my perspective the linker did what you told it. The linker has NOT done what it is supposed to do. From the man page: These commands load the named files into executable files for the corresponding architectures And, in this case, it has not produced an executable file. Furthermore, 8l should give an error when there is no function main(). In fact, it shows that error unless there is no function at all in any .8 input file. -- Juan Cespedes
Re: [9fans] threads vs forks
On Tue, 2009-03-03 at 23:24 -0600, blstu...@bellsouth.net wrote: it's interesting that parallel wasn't cool when chips were getting noticably faster rapidly. perhaps the focus on parallelization is a sign there aren't any other ideas. Gotta do something will all the extra transistors. After all, Moore's law hasn't been repealed. And pipelines and traditional caches are pretty good examples of dimishing returns. So multiple cores seems a pretty straightforward approach. Our running joke circa '05 was that the industry was suffering from the transistor overproduction crisis. One only needs to look at other overproduction crisis (especially the food industry) to appreciate the similarities. Now there is another use that would at least be intellectually interesting and possible useful in practice. Use the transistors for a really big memory running at cache speed. But instead of it being a hardware cache, manage it explicitly. In effect, we have a very high speed main memory, and the traditional main memory is backing store. It'd give a use for all those paging algorithms that aren't particularly justified at the main memory-disk boundary any more. And you can fit a lot of Plan 9 executable images in a 64MB on-chip memory space. Obviously, it wouldn't be a good fit for severely memory-hungry apps, and it might be a dead end overall, but it'd at least be something different... One could argue that transactional memory model is supposed to be exactly that. Thanks, Roman.
Re: [9fans] command repetition in sam/acme
Isn't it sad being in plan9?! Things should be simple, but not simpler than that. I am not sad being in plan9 [sic]. I have used it as my main OS for about eight years and I have used sam exclusively for ten. During that time I cannot remember ever needing or wanting repeat counts on regular expressions. Structural regular expressions have intrigued me since the paper was published but they still await somone with the time and inclination. -Steve (happy in plan9).
Re: [9fans] command repetition in sam/acme
But I think that anyone not under the influence of psychedelic substances that has suffered PCR, will agree we don't want to move in that direction, and even if small, counting is a step in that direction. Your feelings are understandable, given the horror of pcre, but in this case they are irrational. pcre is an atrocity for two reasons: (1) it is badly implemented and (2) it changes the family of languages which can be matched by regular expressions. In fact (1) is more or less a consequence of (2). Adding counts would do neither of these, it just makes it a bit easier to and more natural to describe some languages in the class. There are sane objections to adding counts, but I don't think this is really one of them. -- John Stalker School of Mathematics Trinity College Dublin tel +353 1 896 1983 fax +353 1 896 2282
Re: [9fans] command repetition in sam/acme
I still don't get the discussion. This is a research system. People want something. So implement that feature and see how it goes! Report when done :-) ron
Re: [9fans] threads vs forks
On Wed, Mar 4, 2009 at 12:50 AM, erik quanstrom quans...@quanstro.net wrote: Both AMD and Intel are looking at I/O because it is and will be a limiting factor when scaling to higher core counts. i/o starts sucking wind with one core. that's why we differentiate i/o from everything else we do. And soon hard disk latencies are really going to start hurting (they already are hurting some, I'm sure), and I'm not convinced of the viability of SSDs. i'll assume you mean throughput. hard drive latency has been a big deal for a long time. tanenbaum integrated knowledge of track layout into his minix elevator algorithm. Yes, sorry. i think the gap between cpu performance and hd performance is narrowing, not getting wider. i don't have accurate measurements on how much real-world performance difference there is between a core i7 and an intel 5000. it's generally not spectacular, clock-for-clock. on the other hand, when the intel 5000-series was released, the rule of thumb for a sata hd was 50mb/s. it's not too hard to find regular sata hard drives that do 110mb/s today. the ssd drives we've (coraid) tested have been spectacular --- reading at 200mb/s. if you want to talk latency, ssds can deliver 1/100th the latency of spinning media. there's no way that the core i7 is 100x faster than the intel 5000. For the costs (in terms of power and durability) hard drives are really a pain, not just for some of the companies I've talked to that are burning out terabyte drives in a matter of weeks, but for mere mortals as well. And I'm sorry but the performance of hard drives is *not* very good, despite it improving. Every time I do something on a large directory tree, my drive (which is a model from last year) grinds and moans and takes, IMO, too long to do things. Putting 4GB of RAM in my computer helped, but the buffering algorithms aren't psychic, so I still pay a penalty the first time I use certain directories. Now I haven't tested an SSD for performance, but I know they are better. If I got one, this problem would likely subside, but I'm not convinced that SSDs are durable enough, despite what the manufacturers say. I haven't seen many torture tests on them, but the fact that erasing a block destroys it a little bit is scary. I do a lot of sustained writes with my typical desktop workload over the same files, and I'd rather not trust them to something that is delicate enough to need filesystem algorithms to be optimized for so they don't wear out. I guess, in essence, I just want my flying car today. - erik
Re: [9fans] command repetition in sam/acme
Is it really so? R. Cox (Regular Expression Matching Can Be Simple And
Fast), I think, shows, that repetition can be first expanded and then
used even by the nice (non-backtracking) algorithms, like this:
e{3} -- eee
e{3,5} -- ?e?
e{3,} -- eee+
where would the problem arise?
The problem arises mainly not in this construct
but in the tremendous number of other
constructs that regexp libraries with counted
repetition usually throw in along with it.
That said, even counted repetition is not free.
Even if sam/acme had counted repetition,
it would not handle x{1000} particularly well,
since the expansion you give above would
end up being a very long regular expression
for non-trivial x.
R. Cox
Re: [9fans] threads vs forks
On Wed, Mar 4, 2009 at 8:52 AM, J.R. Mauro jrm8...@gmail.com wrote: Now I haven't tested an SSD for performance, but I know they are better. Well that I don't understand at all. Is this faith-based performance measurement? :-) I have a friend who is doing lots of SSD testing and they're not always better. For some cases, you pay a whole lot more for 2x greater throughput. it's not as simple as know they are better. If I got one, this problem would likely subside, but I'm not convinced that SSDs are durable enough, despite what the manufacturers say. I haven't seen many torture tests on them, but the fact that erasing a block destroys it a little bit is scary. I do a lot of sustained writes with my typical desktop workload over the same files, and I'd rather not trust them to something that is delicate enough to need filesystem algorithms to be optimized for so they don't wear out. in most cases write leveling is not in the file system. It's in the hardware or in a powerpc that is in the SSD controller. It's worth your doing some reading here. That said, I sure would like to have a fusion IO card for venti. From what my friend is telling me the fusion card would be ideal for venti -- as long as we keep only the arenas on it. ron
Re: [9fans] threads vs forks
That said, I sure would like to have a fusion IO card for venti. From what my friend is telling me the fusion card would be ideal for venti -- as long as we keep only the arenas on it. even better for ken's fs. i would imagine the performance difference between the fusion i/o card and mass storage is similar to that between wrens and the jukebox. - erik
Re: [9fans] netbook ( no cd ) install help
trying now... On Mar 3, 2009, at 8:06 AM, Latchesar Ionkov wrote: You can try /n/sources/contrib/lucho/usbinst9.img.gz. Just dd it to a USB flash drive and try booting from it. Thanks, Lucho On Sun, Mar 1, 2009 at 11:37 PM, Ben Calvert b...@flyingwalrus.net wrote: ya, that would be great On Mar 1, 2009, at 2:45 PM, Latchesar Ionkov wrote: Booting from a USB flash drive is possible (if the BIOS supports booting from USB), but a bit tricky. I had to make few small changes in 9load. I have an image somewhere, if anybody is interested in trying it I can try to find it. Thanks, Lucho On Sun, Mar 1, 2009 at 6:27 AM, Steve Simon st...@quintile.net wrote: You can install from a local fat partition if you put the plan9.iso file in the partition - don't unpack it, just put the single big file there and the installer scripts will allow you to chose it as a source for the full install. A bigger problem is you have to boot the installer kernel, normally this comes from either a floppy disk or the install CDROM (which contains a floppy disk inage). If you don't have either of these you may be able to do some tricks by creating a bootable partition by hand on your hard disk, though this is going to get painful. Maybe somone else can think of some other technique. -Steve
Re: [9fans] netbook ( no cd ) install help
ok - this boots and launches into the installer. I need to back track and free up some disk space, I'll report back when i've accomplished that Ben On Mar 4, 2009, at 10:32 AM, Ben Calvert wrote: trying now... On Mar 3, 2009, at 8:06 AM, Latchesar Ionkov wrote: You can try /n/sources/contrib/lucho/usbinst9.img.gz. Just dd it to a USB flash drive and try booting from it. Thanks, Lucho On Sun, Mar 1, 2009 at 11:37 PM, Ben Calvert b...@flyingwalrus.net wrote: ya, that would be great On Mar 1, 2009, at 2:45 PM, Latchesar Ionkov wrote: Booting from a USB flash drive is possible (if the BIOS supports booting from USB), but a bit tricky. I had to make few small changes in 9load. I have an image somewhere, if anybody is interested in trying it I can try to find it. Thanks, Lucho On Sun, Mar 1, 2009 at 6:27 AM, Steve Simon st...@quintile.net wrote: You can install from a local fat partition if you put the plan9.iso file in the partition - don't unpack it, just put the single big file there and the installer scripts will allow you to chose it as a source for the full install. A bigger problem is you have to boot the installer kernel, normally this comes from either a floppy disk or the install CDROM (which contains a floppy disk inage). If you don't have either of these you may be able to do some tricks by creating a bootable partition by hand on your hard disk, though this is going to get painful. Maybe somone else can think of some other technique. -Steve
Re: [9fans] Porter-Duff alpha blending
that's quite interesting, and i suspect you've discovered a bug.
actually two bugs, maybe. one has to deal with drawing alpha-blended
images, the other with drawing using RGBA32...
the simplest way to trigger the second bug, which may or may not be
related to the first, is to draw using black, RGBA32 channel and full
alpha:
the following draws fine on the screen:
b = allocimage(display, Rect(0, 0, Dx(screen-r), Dy(screen-r)),
ARGB32, 0, DBlack);
draw(screen, screen-r, b, nil, ZP);
but the following draws incorrectly as blue:
allocimage(display, Rect(0, 0, Dx(screen-r), Dy(screen-r)), RGBA32,
0, DBlack);
draw(screen, screen-r, b, nil, ZP);
at this point i would suspect an endianness issue. setalpha doesn't
care about endianness, but libmemdraw does.
to add a bit more information i've taken the liberty of modifying your
program. the code attached exhibits all sorts of issues with alpha
blending and illustrates them more fully (i've also used a few of the
tricks in /sys/src/libdraw/arith.c to show the correct usage of Pt(),
ZP, color names and the like)
in this example everything is drawn over a black screen image to avoid
any effects of conversion to the real bpp of the screen. the base
image's channel type is fixed at ABGR32 because this particular
channel does not exhibit the behaviour of switching from black to blue
described at the beginning of this email. i've included a screencap
with the black image using RGBA32 so you can see the difference.
the program shows 4 different columns of 256 lines with varying levels
of alpha-blendedness (from 0xFF on top to 0x00 on bottom). the first
column draws white, second red, third green and fourth blue.
below the column a single line is drawn twice: once on the black
screen and once on the display screen directly. in the two attached
snapshots the line (created with ARGB23) is drawn correctly both on
the black image and on the display when the black image is ARGB32, but
if we switch to RGBA32 for the black image the first line gets mangled
when the whole image is displayed.
i'll leave the bit/byte swap analysis to the readers :)
ps: are you using this on native terminal? it would be very
interesting to test the program there for different values of ARGB32,
RGBA32 and ABGR32 (there's no BGRA32) and possibly under display depth
of 32bpp. i only have 9vx and drawterm to test with, unfortunately,
although i will try with osx's drawterm when i get home: that one runs
in 32bpp at all times.
cheers!
#include u.h
#include libc.h
#include draw.h
#include cursor.h
Image *col;
Image *black;
void
main(void) {
int i;
ulong type = ARGB32;
newwindow(-r 0 0 800 400);
if(initdraw(nil, nil, tri) 0)
exits(initdraw);
black = allocimage(display, Rect(0, 0, Dx(screen-r), Dy(screen-r)), ARGB32, 0, DBlack);
for(i = 0; i 0xff; i++) {
col = allocimage(display, Rect(0,0,1,1), type, 1, setalpha(DWhite, 0xff-i));
draw(black, Rect(0,i,200,i+1), col, nil, ZP);
freeimage(col);
col = allocimage(display, Rect(0,0,1,1), type, 1, setalpha(DRed, 0xff-i));
draw(black, Rect(200,i,400,i+1), col, nil, ZP);
freeimage(col);
col = allocimage(display, Rect(0,0,1,1), type, 1, setalpha(DGreen, 0xff-i));
draw(black, Rect(400,i,600,i+1), col, nil, ZP);
freeimage(col);
col = allocimage(display, Rect(0,0,1,1), type, 1, setalpha(DBlue, 0xff-i));
draw(black, Rect(600,i,800,i+1), col, nil, ZP);
freeimage(col);
}
col = allocimage(display, Rect(0, 0, 1, 1), type, 1, DBlack);
draw(black, Rect(0, 260, 800, 300), col, nil, ZP);
draw(screen, screen-r, black, nil, ZP);
draw(screen, Rect(0, 360, 800, 400), col, nil, ZP);
flushimage(display, 1);
sleep(1);
closedisplay(display);
}
attachment: RGBA32.pngattachment: ARGB32.png
Re: [9fans] Porter-Duff alpha blending
it looks fine on a native plan 9 386 terminal. - erikattachment: α.png
Re: [9fans] Porter-Duff alpha blending
but it doesn't: the red is drawn as blue, the green switches to pink in the middle, and blue is drawn as red and cyan... only the white is alpha-blended correctly. On Wed, Mar 4, 2009 at 2:32 PM, erik quanstrom quans...@coraid.com wrote: it looks fine on a native plan 9 386 terminal. - erik
Re: [9fans] Porter-Duff alpha blending
to see how it should look when drawn correctly use type = ARGB32 and RGB24 for black's allocimage(). see attached. the alpha blending still works because when downgrading from ARGB32-RGB24 (for drawing onto black) the library still takes the source color from black without issues. the bugs appear when 32-bit channels are being used for source and destination. attachment: RGB24.png
Re: [9fans] Porter-Duff alpha blending
i didn't do a hex dump. it must be displayed differently outside of plan 9. - erik
Re: [9fans] Porter-Duff alpha blending
are you saying that you see the correct image but wee see the png differently? can you convert it to gif instead of png? On Wed, Mar 4, 2009 at 2:44 PM, erik quanstrom quans...@coraid.com wrote: i didn't do a hex dump. it must be displayed differently outside of plan 9. - erik
Re: [9fans] Porter-Duff alpha blending
this is a picture of what this looks like on my screen. your attached png is 4 stripes of white-blue blue-blue magenta-blue cyan-blue - erikattachment: αpic.jpg
Re: [9fans] Porter-Duff alpha blending
the jpg file you attached doesn't look like a jpg file :( On Wed, Mar 4, 2009 at 2:53 PM, erik quanstrom quans...@coraid.com wrote: this is a picture of what this looks like on my screen. your attached png is 4 stripes of white-blue blue-blue magenta-blue cyan-blue - erik
[9fans] GSoC 2009
So, the web site's up, program is announced, and so on. Was anyone planning on doing a Plan 9 application? I'm willing to help, if anyone was planning on it, or lead, if not. In the later case, I'd appreciate help (any of advice, materials, or labor) from people who were involved in our last two applications. Anthony
Re: [9fans] Porter-Duff alpha blending
ok, those all exhibit the incorrect blending behaviour, would you not agree? here's a .jpg that shows the correct behaviour and that should render properly on Plan9. I just tried RGB24.png and it indeed renders incorrectly in 9vx the way you're describing it. the jpg should be the benchmark for correct display. attachment: RGB24.jpg
Re: [9fans] Porter-Duff alpha blending
On Wed Mar 4 17:48:12 EST 2009, mirtchov...@gmail.com wrote: Erik, I think you're running in 32bpp mode, right? yes. iirc my nvidia card was not willing to do 24bpp. To have 'png' render RGB24.png correctly make the following change to /sys/src/cmd/jpg/png.c: however it won't fix the program that you originally sent. - erik
Re: [9fans] Porter-Duff alpha blending
however it won't fix the program that you originally sent. the fix (if you can call sidestepping the problem :a fix) for the original program is to use RGB24 as the channel for black's allocimage(): black = allocimage(display, Rect(0, 0, Dx(screen-r), Dy(screen-r)), RGB24, 0, DBlack);
Re: [9fans] Porter-Duff alpha blending
On Thu, Mar 5, 2009 at 10:31 AM, maht mattmob...@proweb.co.uk wrote: I decided to try some draw(2) exploration and I'm not getting what I expected from alpha blending. If I apply alpha to RGB24 images they change as expected but if I use RGBA32 images and try to apply alpha it's going weird on me (or I am weird on it). If I apply setalpha(DRed, 0x7F) and then draw() it to screen instead of being less red it goes blue! I've posted some sample code and the outputs at http://maht0x0r.blogspot.com/2009/03/o-that-way-madness-lies.html I can't tell from your post which images you think are wrong. You have three images after the words But when you apply alpha to RGBA32 images something unexpected happens:. The first one looks fine. The second one does not specify what your initial background is. It matters, because you are drawing a 1/2-translucent image and then a second one, so the result is 1/2-translucent in some places and 1/4-translucent in others. Whatever is underneath will shine through. Note the difference in the bottom right image in the white half vs the black half in the attached alpha.png. (View the PNG using something other than Plan 9, if necessary.) Andrey's bug feels like a variation on that theme, but I cannot pin it down just now. Russ attachment: alpha.png
Re: [9fans] Porter-Duff alpha blending
from 32-bit image to a 32-bit display. a 32-bit display? what's the result of cat /dev/draw/new?
Re: [9fans] Porter-Duff alpha blending
On Wed Mar 4 18:42:46 EST 2009, fors...@terzarima.net wrote: from 32-bit image to a 32-bit display. a 32-bit display? what's the result of cat /dev/draw/new? 1: !cat /dev/draw/new 9 0x8r8g8b8 0 0 0 16001200 0 016001200 cat: erro - erik
Re: [9fans] Porter-Duff alpha blending
in the case of 9vx running on Linux it's x8r8g8b8, drawterm on osx is also x8r8g8b8. here's how it breaks down drawing to display: sourcedestination error XRGB32 XRGB32 no XBGR32 XRGB32 no ARGB32 XRGB32 yes (see attached) RGBA32 XRGB32 yes i've done a bit more testing and will hopefully have a full table for 32-bit images later tonight: attachment: xrgb-XRGB32.png
Re: [9fans] Porter-Duff alpha blending
just for fun, the attached file should run the whole gamut of
allocimage chan options in the 32-bit range (with RGB24 thrown in for
good measure) and will create files in /tmp for each possible
combination without involving drawing to the screen display (ideally
this should be done in memdraw, i suppose).
i think page will display them honestly, but i'm not sure. i'll leave
to you to convert the resulting images for outside viewing :)
#include u.h
#include libc.h
#include draw.h
#include cursor.h
Image *col;
Image *back;
ulong types[] = {
RGBA32,
ARGB32,
XRGB32,
XBGR32,
RGB24,
};
char *descs[] = {
rgba32,
argb32,
xrgb32,
xbgr32,
rgb24,
};
void
main(void) {
int fd, i, j, k;
char *str;
ulong type = RGBA32;
if(initdraw(nil, nil, tri) 0)
exits(initdraw);
for(j = 0; j 5; j++) {
for(k = 0; k 5; k++) {
str= smprint(/tmp/%s-over-%s.1, descs[k], descs[j]);
print(creating: %s\n, str);
back = allocimage(display, Rect(0, 0, 800, 300),
types[j], 0, DBlack);
for(i = 0; i 0xff; i++) {
col = allocimage(display, Rect(0,0,1,1),
types[k], 1, setalpha(DWhite, 0xff-i));
draw(back, Rect(0,i,200,i+1), col, nil, ZP);
freeimage(col);
col = allocimage(display, Rect(0,0,1,1),
types[k], 1, setalpha(DRed, 0xff-i));
draw(back, Rect(200,i,400,i+1), col, nil, ZP);
freeimage(col);
col = allocimage(display, Rect(0,0,1,1),
types[k], 1, setalpha(DGreen, 0xff-i));
draw(back, Rect(400,i,600,i+1), col, nil, ZP);
freeimage(col);
col = allocimage(display, Rect(0,0,1,1),
types[k], 1, setalpha(DBlue, 0xff-i));
draw(back, Rect(600,i,800,i+1), col, nil, ZP);
freeimage(col);
}
fd = create(str, OWRITE, 0664);
if(fd 0)
sysfatal(create);
writeimage(fd, back, 0);
close(fd);
}
}
}
Re: [9fans] Porter-Duff alpha blending
xgbr32 over xgbr32 or xrgb32 looks fine. but xgbr32 over agbr32, for example, results in a fade-to-blue and an extremely slow display time. seems like the channels are getting mixed up. oddly, xrgb32 over argb32 works (but slowly). - erik
Re: [9fans] threads vs forks
On Wed, Mar 4, 2009 at 12:14 PM, ron minnich rminn...@gmail.com wrote: On Wed, Mar 4, 2009 at 8:52 AM, J.R. Mauro jrm8...@gmail.com wrote: Now I haven't tested an SSD for performance, but I know they are better. Well that I don't understand at all. Is this faith-based performance measurement? :-) No, I have seen several benchmarks. The benchmarks I haven't seen are ones for how long does it take to actually break these drives? from anyone other than the manufacturer. I have a friend who is doing lots of SSD testing and they're not always better. For some cases, you pay a whole lot more for 2x greater throughput. it's not as simple as know they are better. What types of things degrade their performance? I'm interested in seeing other data than the handful of benchmarks I've seen. I imagine writes would be the culprit since you have to erase a whole block first? If I got one, this problem would likely subside, but I'm not convinced that SSDs are durable enough, despite what the manufacturers say. I haven't seen many torture tests on them, but the fact that erasing a block destroys it a little bit is scary. I do a lot of sustained writes with my typical desktop workload over the same files, and I'd rather not trust them to something that is delicate enough to need filesystem algorithms to be optimized for so they don't wear out. in most cases write leveling is not in the file system. It's in the hardware or in a powerpc that is in the SSD controller. It's worth your doing some reading here. I've seen a lot about optimizing the next-generation filesystems for flash. Despite the claims that the hardware-based solutions will be satisfactory, there are a lot of people interested in making existing filesystems smarter about SSDs, both for wear and for optimizing read/write. Beyond that, though, I feel very shaky just hearing the term wear leveling. I've had more flash-based devices fail on me than hard drives, but maybe I'm just crazy and the technology has gotten decent enough in the past couple years to allay my worrying. It would just be nice to see a bit stronger alternative being pushed as hard as SSDs. That said, I sure would like to have a fusion IO card for venti. From what my friend is telling me the fusion card would be ideal for venti -- as long as we keep only the arenas on it. ron
Re: [9fans] threads vs forks
On Wed, Mar 04, 2009 at 10:32:55PM -0500, J.R. Mauro wrote: What types of things degrade their performance? I'm interested in seeing other data than the handful of benchmarks I've seen. I imagine writes would be the culprit since you have to erase a whole block first? Being full. Small random writes, too, although much more so for run-of-the-mill SSDs than for FusionIO. [citation needed] - erik
Re: [9fans] mounting plumber from p9p to 9vx
2008/11/24 Russ Cox r...@swtch.com: How do I mount my p9p plumber on to my 9vx session? % ls /mnt/plumb % bind '#Z' /mnt/term % ls /mnt/term/tmp/ns.rsc* /mnt/term/tmp/ns.rsc.wreck/acme /mnt/term/tmp/ns.rsc.wreck/plumb % mount /mnt/term/tmp/ns.rsc.wreck/plumb /mnt/plumb % ls /mnt/plumb /mnt/plumb/edit /mnt/plumb/image /mnt/plumb/msword /mnt/plumb/openoffice /mnt/plumb/postscript /mnt/plumb/rules /mnt/plumb/seemail /mnt/plumb/send /mnt/plumb/sendmail /mnt/plumb/showmail /mnt/plumb/web % Can you also write to 9vx's plumber? -- Tom Lieber http://AllTom.com/
Re: [9fans] GSoC 2009
Well, I asked about GSoC 2009 some weeks ago and I got no reply here. I certainly would like to apply as a student, but I am no sure if I qualify (in one hand I am kind of a student, and on the other I probably need to learn a lot more about plan 9). In any case, I would like to see how this evolves. Saludos 2009/3/4, Anthony Sorace ano...@gmail.com: So, the web site's up, program is announced, and so on. Was anyone planning on doing a Plan 9 application? I'm willing to help, if anyone was planning on it, or lead, if not. In the later case, I'd appreciate help (any of advice, materials, or labor) from people who were involved in our last two applications. Anthony -- Hugo
