Also, on a somewhat related note, does catching SIGSEGV not work for
some reason?
I can see no reason for why it would not work for this specific
purpose.
Still, doing the checks that are done now in mmap does not in any way
provide protection against sigsegvs.
Well.
I mean the fact that you have to have a ; after } where } define a
function in functions (well, that's not really why, but it's where it
shows up).
>Your example did not contain any definitions of functions in
>functions.
That's sort of debatable, since x(){} is just a shorthand syntax for
x(lambda(){});, which does define a local function. :)
I do see the conflict, however.
>I think the biggest benefit is that your async programming becomes/or
>looks more sequential/synchroneous which tend to be easier to follow,
>or in other words less spaghetti-ish.
And I definitely beg to differ that.
Take this future-less code from a chat server:
| void
Yes. Inherit the buffer, then implement a function
bool range_error( int bytes ) {
if( more data added to buffer )
return true;
return false;
}
Something I would recommend, from having written heaps of asynchronous
object based systems, with or without remote computers involved, is to
treat all references to the promise object going away as an error, and
avoid un-needed circular references.
This makes it easy to detect when you
The write calback will only be called when the buffer is empty.
So it has a slightly different purpose. But it is not needed. You can
use the filling callback in the buffer object instead, as an example.
Well.
I think that is sort of the wrong question, what do you really want?
But:
As long as you have a multiset (and, in 8.0+ probably mapping) and do
not change it's content the iteration order tends to be stable (modulo
gc of destructed things in said containers in this case, or if they
have
F_LOOP is rather simple.
is type int?
yes:
is it 0?
yes:
end loop
no:
decrement
call body
no:
call f_loop
Note the absence of "or less than" in the check for 0. I guess there
should be such a check. It's there in the interret_functions.h
version.
I will
It looks good to me.
There might be some other functions that would be useful for external
modules, but this looks like a good start.
I have now run a somewhat more extensive test (run the benchmark for
20 hours, alternating with and without the substring type).
Even with this long a run of the testsuite there seems to be a few
%-agepoints of variance, which is impressive. The code is identical
except for the change to making
Ok, #3 is fixed now. So, the testsuite is on average faster. In some
cases significantly so, for obvious reasons.
However, that does not really say much, except that there is no
drastic slowdown.
C11 is default since gcc 5.2, I think. But C99 seems to have been
skipped (as the default) entirely.
C99 would be nice, there is a lot of nice additions.
And we could remove a lot of the various ATTRIBUTE() style macros.
And there are predefined types for integer-with-x-bits
(fastest-with-or-at-least x bits etc). That removes another bunch of
configure tests.
Also, there is of course a bunch of
Since this change does fix the testcase, should I commit it to 8.1?
Please go ahead and commit it.
Ok.. I accidentally commited about half of the changes to 8.1, without
the other half (I did a 'git push' but aborted almost directly when I
noticed it was pushing 8.1. However, it seems aborting does not stop
the commiting, and in this case a lot of the commits were not actually
included).
To
On a somewhat related note, I noticed that the string structure no
longer has extra padding bytes available, so adding new string types
(substring) is not possible without increasing it's size
As it turs out, by shrinking the size_shift field to 2 bits there are
6 bytes left for types instead of
There is a somewhat major difference between add_string_constant (not
static) and push_constant_text (static + constant.:))
But I can agree that the string function names are rather messy. It is
a rather big job to change them all to be consistent, however.
On a somewhat related note, I noticed
Would it be OK to merge the split string code to pike 8.1?
I have done the merge to a private branch, and now wonder if it would
be OK to push it upstream.
It splits strings into two parts: The pike_string structure and the
actual string.
There are at least two advantages that are available
On my slow arm (about 50% the speed of a pi) it takes about a minute
to load the calendar module. Almost all that time is parsing
timezones.
So, limiting the size of the timezone file for slow devices would
help.
This is somewhat surprising, becase I can not remember it taking
minutes to load
I have tried that, and it is somewhat hard because they are compiled
to objects in several mappines/lists. Adding _encode and _decode and
then using encode/decode value does not really make things all that
much faster.
Protocols.HTTP uses the calendar module to decode http dates.
If everyone in the http land just used the GMT timezone as specified
this would not really be needed, and perhaps now it would be safe to
switch to System.TM or just mktime().
0.18 seconds to load the perl modules - the pi is 50 times slower.
Actually closer to 100x, if you are looking at raw CPU performance.
However, memory speed is probably somewhat important as well, it is
only about 10x slower there.
The information is actually still available, but in historical CHANGES
file.
We do not offer those on the website, and most definitely not marked
up in a nice way, however.
We might want to do that.
Any new version that might work better around?
Not really, no. The thing is, the files seems to have correct
timestamps, so there should be no need to re-generate the
interpret_functions file.
Ω leopard$ ls -l interpret_functions*
91 -rw-rw-r-- 1 per per 92635 okt 20 15:51
Something along the lines of
Buffer sub = b-new_hbuffer();
sub-add...
sub-finish();
looks cleaner, I think.
--
Per Hedbor
It is thread safe. In fact, all buffer methods are.
With one exception: If you have a buffer-is-empty handler function,
that function has to be threadsafe for the buffer object to be so.
--
Per Hedbor
It is thread safe. In fact, all buffer methods are.
overflows the string if it encounters a truncated sequence, e.g. a
trailing % or %u with less than 4 hex chars.
The old code relied on the null character after the last character in
the string, but that changed when it started to accept widestrings.
--
Per Hedbor
I just forked off pike 8.1 from 8.0.
The intent is to fairly soon (for values of soon) release a new stable
from 8.0.
If you have any issues with the current version, please tell us about
it. :)
Also, new features should preferably go to 8.1 now.
--
Per Hedbor
Was having trouble building the latest, but a complete wipe-out (rm
-rf *; git checkout -f) fixed that. Not sure what the issue was,
doesn't much matter.
If I would have to guess it is the precompiler changes. The
dependencies for the precompiler are not quite correct, changes to the
And on a somewhat related note /bin/sh is significantly faster than
bash when it is not an old solaris (before opensolaris)/irix or
similar machine, say, hpux. :)
That is, when /bin/bash is dash or a modern (k)sh it is actually
faster than bash, significantly so in the case of dash (it saves
bout
No. I do know that the SunOS 4 shell is very slow when you have a lot
of variables, however, due to a fixed hashtable size for them.
Unless that has changed it is probably still significantly slower than
/bin/bash.
TCP_DEFER_ACCEPT, TCP_FASTOPEN, newcomers related to TCP_NODELAY.
Thanks for reminding me. :) TCP_FASTOPEN is now supported by pike.
--
Per Hedbor
Mine is an 8 core ~2Ghz with 2Gb of DDR3 RAM. And a very fast ssd.
So it is actually rahter fast at building, around 15 minutes from
scratch using -j8, slightly over 10 of those minutes are configure.
Ideally, IOBuffer should inherit from String.Buffer, for maximum
code reuse.
For the reasons I mentioned in the last mail that is not actually
posible.
Or, well, for sure, I could inherit String.Buffer, but _all_ methods
would have to be overridden, and some do not really make sense, like
`+,
The actual reason I beefed up Buffer a bit lately is *because* I need
to do some protocol decoding of a byte stream.
Now I see IOBuffer arrive. In order to avoid code bloat, wouldn't it
be a better idea to integrate the functionality of IOBuffer into Buffer
and just keep one?
Sorry about
Xenofarm seems to have been out of order since a while, correct?
Indeed, it has been a while since it worked.
I have aquired an ARM build node now (32bit) (do we have anybody
interrested in developing more ARM support? You can get an account if
so).
I looked into setting it up to build with
Once done with that, the final result can/should be a shared
string.
An additional comment: By definition you are almost never actually
done with a IOBuffer.
They are designed to be input and output buffers for IO.
--
Per
I've been tracking IOBuffer extensions back to String.Buffer, I'll present
what I have shortly.
I suspect (but benchmarks will have to tell) that the String.Buffer
implementation is not significantly slower than the current IOBuffer
one (whilst supporting the full range of character widths).
Erm...
We are *in* a Buffer object, so by definition we have one.
So returning a readonly-copy with zero-copy effort is easy.
It basically delays the creation of the shared string as long as possible.
Not really, we are in /a/ buffer object, not the subsection of it that
should be returned.
Then there's of course the separate problem with out of band data...
/grubba
I forgot to comment about this:
Thankfully there is really nothing that uses it currently.
--
Per Hedbor
I haven't been keeping up with the *Buffer stuff. Would it be possible
to make a line iterator on top of this, and would it do any differance
to performance?
I have sticky note on my desk that says I should check why Python is
faster on some basic log parsing of huge ascii logs I do.
Well,
An additional comment: By definition you are almost never actually
done with a IOBuffer.
They are designed to be input and output buffers for IO.
And now the basic support is there to use them for Stdio.File objects.
Stdio.File now has a new nonblocking mode: Buffered I/O
In this mode the
Object `+= lfuns are not called when they should have been.
Well. That depends on how you define should have been.. :)
`+= has never been called except when there is only one reference to
the object, which is why it can be somewhat confusing. It does allow
optimizations since the value can be
Recent changes to how the testsuite is run makes error messages from
expected failures to be printed out. That is not helpful.
Fixed.
Also, we've gone from 0 to 42 failing tests.
Also fixed. I was considering making Parser.HTML be a real html5
parser, by the way, but then I read it. :)
It
- Stdio.IOBuffer is harder to type than ADT.struct. Can we rename it
to Stdio.Buffer? There is already IO in Stdio.
I tried Stdio.Buffer but then all types suddenly became String.Buffer.
I could not, at that time at least, be bothered to fix the
precompiler...
- read_buffer dosn't return
Well, that would work, except for this:
p-finish(t a='b'c='d''e'=\f\)-read();
([ /* 3 elements */
a: b,
c: d,
e: f
])
(3) Result: t a='b'c=d 'e'=\f\
Isn't the old syntax what Roxen uses to mix different quotes in the
same attribute value? I believe it's supposed to work like this:
foo attr=use ' here'use here'/
I do not really think that was ever an encouraged syntax? quote; has
always been preferably to my knowledge. It has been a
No, it was somewhat easy to fix so the attributes work as in HTML,
but now they no longer work as they used to.
Changing what? Hasn't it always been a syntax error? Just that the
parser doesn't actually report it?
No, Parser.HTML has always _explicitly_ allowed quoted attribute names
(it sets the allowed quotes to the same thing for both attribute
values and names).
I know this is fairly odd, but so
Sorry, I had an error in my it will now return mapping.
Actual example:
p-finish(t a='b''c'\d\e)-read();
([ /* 3 elements */
a: b,
c: c,
d: d
e: e
])
I would like something that breaks down an html document to a
datastructure, preferable one with tools like searching.
Well, yes, a HTML tokenizer would be useful. HTML5 has a very readable
specification.
Parser.HTML is created to allow RXML (or similar) parsing with as
little computron usage as
Really? That doesn't even have matched quotes...
You are free to leave out the last endquote...
Do I understand correctly that the old parser allows quoted attributes
and multiple strings in attribute values, but does not allow multiple
attributes without whitespace in between or multiple strings in
attribute values _with_ whitespace in between?
Yes. (it also allows quoted and multiple
Is there an example of something which is valid HTML _and_ which would
(for good reason) be interpreted as something else by the old parser?
Well, the one above. And, for good reason is relative, I guess.
If you present the reason why anyone would write that, then I might be
able to judge
string|String.Buffer is a union type, so using it in conjunction
with an `= lfun should result in an error reported to the
programmer.
That is not possible to know at compile time, however.
And we _never_ pass by value in pike, so it would be totally
unexpected to any pike developer.
It might be that he is using a laptop. Or just has power savings
enabled. I have noticed differences on the order of 70% on my macbook
with _no_ code changes.
The adaptive frequencies is generating issues, even with the power
supply connected. In general it is depressingly hard to benchmark
** remove class equals (too many equals)
This somewhat obscure point is probably the existence of a whole bunch
of x [op]= op-codes (+=, -=, =, %= etc)
If so I have now removed them.
This turned out to be sort of interesting, actually, since +=, -= and
friends did not previously type-check the
On Thu, Aug 14, 2014 at 17:01, Peter Bortas wrote:
You brought it up, so however you define it is the correct definition.
Thanks. :)
I can now inform you about other good news: Automap is more complex!
Now and then it now works to have an automap as the left side of an
assignment.
So, what
lines. The crash happens as you describe if there are at least 9 lines
in history; if there are exactly 8, then pressing up won't crash, but
pressing down afterward will.
Don't know if that helps at all or not.
It did!
The code crashed if you had 8..15, 24..31, 40..48 etc
(when num_entries8
... Not very helpful backtrace though.
On a somewhat related note: It is actually possible to tell gdb about
the pike code.
The API is however somewhat .. interresting, you basically generate an
ELF-file with only debugging sections, in memory, and pass it to GDB:
Can we get a more minimal test case in the testsuite?
Well, sure, but it seems to be a fairly esoteric crash condition, how
often do we write machine code that uses the wrong register, and that
for whatever reason _almost_ works?
It is not all that likely to be in the same place if it happens
Partially done now.
Still left to fix are the very complex rules (define
variable,global,function, local function), the issue is that a lot of
things are renumbered when the optional_stars token is removed, and
since optional_stars is now so optional that it is never ever set I
have not bothered
SET_SVAL only uses one, though, the fast_svalue pointer.
It is however true that reading from the argument passed after the set
might be optimized by gcc, I guess. Although that seems like a rather
odd optimization to do since the relevant code is right there.
Anyway, using an union is not an
Isn't also TYPE_SUBTYPE wrong in the big endian case? It seems to me
that on a 64bit big endian machine both subtype and type would always
end up being 0.
like so?
#define TYPE_SUBTYPE(X,Y) (((Y)|((X)16)) (sizeof(ptrdiff_t) - 32))
Yes, it is currently not correct for 64-bit bit endian.
And I
So. At least we are more complete than any other SSL implementation
around.
Could we perhaps soon pronounce it to be 'good enough' and release
pike 8.0 as the new stable? :)
Personally I think it could be removed entirely. It's not like it's
all that common for people to try to decode .o-files generated by
other versions of pike at all.
Speaking about encode: How about putting the encode code for programs
in a separate .so file, that is only loaded when dumping
There is also no point in including it three times, since you can
happily use a slow version of cpp with one function for each string
witdth. Per Hedbor worked on that a week ago, with very good results
in triming down the size (just getting rid of all the inlined code does
wonders in both
I have a cpp.o version on x86_64 that is 48Kb (normally it's about
200k), which is better, albeit still rather large.
That involved changing the input to a pike-string (size /= 3), and
changing all the macros to functions (most are automatically inlined
by gcc, compiling with -Os saves a few Kb
For reference, the pike compiler would actually fit on a DOS box:
textdata bss
195650 0 312 cpp.o
142355 0 148 language.o
88963 0 720 las.o
88252 48264 peep.o
49165 0 0 lex.o
48132 84 68 pikecode.o
42776 72
However, I still can't entirely shake the notion that we're overdoing it
here. Maybe we could simply make the preprocessor and compiler grok
UTF8 directly and get rid of the special casing. All compiler
input processing would return back to 8-bit only.
Converting everything to utf8 before
The only problem with printf is that it seems to be missing on my
embedded ARM fan controller. Not really important, though, I guess,
since it is also missing a working linker currently. :)
I was looking at the GTK support in Pike and noticed the wrappers
where some sort of pre-files in src/post_modules/GTKx
So are these just a special case of mods that new_build_pgtk.pike
generates and not another supported way of writing cmods?
These are rather special, and are only handled
There is no way to ask if a string is marked as secure or not
though.
Actually... :)
Now that flags are propagated: Add 2Gb of data to the string.
Then time how long time it takes to set it to 0.
Is anyone seriously attached to the current pike -x benchmark
format/output?
I have done some work to
a: Only output relevant information
b: Actually output relevant information.
Additionally I have added a json output mode, and a way to run a
benchmark with an old result as the baseline (and
I have some tools for clusterized benchmarks that rely on the
Shoot.pmod raw output format - but that's probably untouched?
No, sorry, the raw output and API is totally different.
Specifically:
Pike v7.9 release 11 running Hilfe v3.5 (Incremental Pike Frontend)
Tools.Shoot.run_sub(
i thought that was the case, at least my impression from the
discussion about it was that once a string was marked secure, it
would no longer be shared.
No, that is not at all how it works. They are still just as shared as
normal strings.
So it is indeed not really all that useful.
In a
Cool, did indeed fix those benchmarks. On the other hand two of my
three real-world XSLT tests dropped 2% (and one gained as much), but
perhaps that's just differences in CPU cache use or alignment or
similar.
If the tests use widestring it might very well be becase the longest
wide short
Yes, I see. I will merge the new block alloc and keep the new
pike_frame code. I have probably not read it properly, then.
... current 7.9 doesnt work for me, right now.
It does not work with DEBUG enabled currently, I will commit a fix for
that shortly.
If you have some other issue (say,
Binary Trees... 936518/s (43Mb) (3x faster than 7.8!)
For reference, simply replacing the blockalloc with malloc/free
(something I did to try it out yesterday) gives this:
Binary Trees... 818800/s (122Mb)
So the new block-alloc is at least fater (and uses significantly
Using malloc/free as in cc0851de980d46e85e514657d8d9a484fe8be2eb is
certainly much faster than the old worst case behavior. I would
guess the new block allocator is faster than malloc/free, so I
propose to overwrite cc0851de980d46e85e514657d8d9a484fe8be2eb by
this. Any objections?
I am not
Using malloc/free as in cc0851de980d46e85e514657d8d9a484fe8be2eb is
certainly much faster than the old worst case behavior. I would
guess the new block allocator is faster than malloc/free, so I
propose to overwrite cc0851de980d46e85e514657d8d9a484fe8be2eb by
this. Any objections?
To clarify
So no automatic configure to a new system?
I think the idea is to use something like CMake.
--
Per Hedbor
Well, we might generate the cmakefiles with a pike script to cut down
on the amount of information duplication in git. :)
I presume the lookup table code comes from the hex2string/string2hex
code.
I added it there because it significantly speed up operations with
long strings, while keeping compatibility (your shift solution
would probably be faster, though).
The reason it works well there is because the strings in
Actually, a much better solution would be to implement the JIT-debug
info generation supported by gdb (including variable mapping etc).
This should make it possible to simply debug using gdb, with symbols.
This can be done using a plugin with GDB 7.5+ (I think)
That would probably help with viewing of stacktraces and variable data,
but wouldn't opcodes still be needed to actually control stepping and
breakpoints in the app itself?
No, GDB will handle the breakpoints natively.
Which is why it would be so convenient.
Basically, when you tell GDB to
However, if during the 7.8 release cycle, the 32-bit machine code
generator has worked at one point or another, then it should keep
working in this release.
The change is in gcc, not pike.
gcc 4.6+ generates broken code for calls to our (old) x86-32
machine-code.
--
Per
That's exactly what I'm asking... How many places are there where we
explicitly depend on the fact that the address can be used to define
uniqueness?
All places where strings are compared.
Say, a few thousand places in the code, probably?
Most importantly: Mappings and multiset, identifiers
The issue isn't necessarily the hashing but the fact that you need to
have this globally synced instead of e.g. creating a thread-local
string pool.
Well. Yes, but as long as we do not have actual threads that can run
concurrently in pike this is not much of an issue, really.
C-code can (and
If it was not clear, before CRC32, almost 50% of share-string time was
spent in the hash function. But it was still a very small percentage
of the total CPU time used.
I optimized it because it was easy to do, mostly.
We spend significantly more time looking up identifiers in objects, as
an
Do you have some statistics on this? I'd imagine that most time
spent on comparising hash-hits would be on short to medium length
strings, and not really long ones since it's unlikely you'll find
another string with the exact same length in the hash bucket.
I do have some statistics from the
Is it, really, when the only information you get is the checksum of
the file content, anyway?
If you go for crc32 with the iscsi polynominal you can use the
instructions in modern x86 CPU:s. That should make it way fast at
least. :-)
Feel free to replace the 'O' with a fish logo. :-)
As I mentioned, I would like it to restore deleted files as well...
Doesn't it?
Well, no, he wants it to emulate this usecase of 'cvs up':
# discard my changes in foo.pike:
rm foo.pike
cvs up foo.pike
basically what 'git-up;git checkout HEAD foo.pike' does.
My favourite command, 'git up': (save as git-up somewhere in your
PATH):
#!/bin/sh
no_changes () {
git diff-index --quiet --cached HEAD --ignore-submodules --
git diff-files --quiet --ignore-submodules
}
git update-index -q --refresh
POP=1
if no_changes ; then
POP=0
else
echo Saving
Why was this change made? GTK2.FileChooserDialog works fine, it
just requires gtk 2.4. When I change it back to gtk24 from not_now,
it compiles fine and the widget is usable.
Feel free to. It did not work back then, when I got the 'require'
statements working I got compile errors in a few
1 - 100 of 133 matches
Mail list logo
