> quite unexpected to have double values when changing SQL backends.
> What's the rationale behind the commit?
One thing I can think of is if you fetch same column name from
multiple tables you can disambiguate between them in the response
mapping.
Good job!
> o Toole.Standalone.check_http
Small typo: Toole -> Tools
Isn't Calendar supporting a much wider range of years, i.e. not
starting at 1970?
Sounds good. I first wondered what the relationship is to Thread, but
the source code comments mentions that one can block a thread on the
counter value so it probably fits that it's placed in that namespace.
I wonder if RefCount would be a more suitable name though? Not a
strong opinion either
I'm not fully convinced of the syntax proposed by Marcus. If the
expression contains an assignment it leads to multiple "=" signs that
look weird.
In my testing now I see that Pike also supports inline declaration of
a variable in a "return" statement. Not sure if this is by accident,
but it can
Ah, wrong kom (or wrong anguage :-). To recap:
I see Pike 8 garbage collecting this object quite frequently, so my
question is whether there's a leak in the internal search
optimization?
src/pike_search.c:197 _static_modules.Builtin()->Sear... 22
9 -24 148464
Vi pratar en del om GC-körningar i CMS/EP i Slack, och på en i stort
sett obelastad EP hos Kvällsstunden stack detta ut i manuell
GC-körning:
src/pike_search.c:197 _static_modules.Builtin()->Sear... 22
9 -24 148464
Är det någon intern optimering av (sträng?)sökning som
Or just class C(array b) { ... } and remove create() completely,
though I haven't checked if the final modifier can be combined with
that syntax.
Just thinking of Nilsson's trick here:
- if(has_value(str, "\0"))
+ if( String.range(str)[0]==0 )
I was about to suggest moving this optimization to f_has_value(), but
I now see that it's already part of f_search() (called by f_has_value)
for string/string cases so this is just an
Granted more manageble for code I maintain myself, but it doesn't
eliminate the problem in theory.
I think it ultimately depends on what 8.1 is intended to be when it is
released: a drop-in upgrade in 8.0 environments, or a major step like
7.x to 8.0? If it will become Pike 9.0 and introduce a
Not all of our source is public so I doubt you can check that. :-)
Just one example where there could be a problem: filename pattern
matching. We have customers that get news wires with odd filenames,
and the pattern is user-configurable in the import module. It's the [
character that I think
Just saw Per's checkin (781fcde2) in 8.1:
> Extended glob pattern syntax:
> o \ can now be used to quote special characters in the pattern
> o [ can be used for ranges of characters ([bx] [a-c0-9] [^a] etc).
>
> Also changed glob to return the matching glob instead of 1 when an
> array is
Building on OS X was broken since last week but I pushed some fixes to
correct that.
grubba added it, so I guess roxen uses it somewhere?
Our dev branches still run on 7.8 so there shouldn't be any such
dependency.
Here's one of several real-world examples that I found from our CMS
(it's a bit tricky to grep after these constructs...):
#define quote(X) (replace((X)||, ', '\'\'))
[...]
var name='destname' type='string' size='40' default=' +
quote(sbobj-name(id)) + ' /
A sbobj-name() returning 'foo'
Or Parser.HTML5?
poll on OS X uses kqueue internally so it feels rather pointless
unless I'm missing something.
Cool, did indeed fix those benchmarks. On the other hand two of my
three real-world XSLT tests dropped 2% (and one gained as much), but
perhaps that's just differences in CPU cache use or alignment or
similar.
Very nice!
I did compile a fresh 7.9 and then copied all the bench files back to
an old 7.8 and ran both to compare. Although most things were
noticeably improved there are a handful of significant regressions:
7.8:
call_out handling.. 0.097s 0.005s (25) (200123/s)
call_out
One of our customers asked about Pike support for Google Protocol
Buffers [1]. I wasn't aware of that standard before but apparently
it's been incorporated into a wide variety of languages [2]. Has
anyone here experimented with it, and perhaps prototyped a Pike
version?
[1]
Don't know if there could be other peep.in rules that take precedence
or vary randomly. I know that one of the other bugs that I fixed
(tNStr) didn't show up every compile.
The fix I did for F_NEG_NUMBER begs the question whether 0x8000 is
a valid argument for negation. I would assume not and
Which branch? My fix for that was on Arne's branch. If that is stable
I guess it's time to merge it into 7.9.
I saw all of those too but they should be resolved on the of branch.
Good work! Unfortunately I cannot compile the bignum.h macros with
Clang since the expansion order seems different (looks like type2 is
not expanded):
In file included from
/Users/jonasw/hacks/Pike/git/pike79/src/modules/Gmp/mpz_glue.c:37:
/Users/jonasw/hacks/Pike/git/pike79/src/bignum.h:179:1:
Changing the GEN_OF2 definition to use PIKE_CONCAT() simply shifts the
problem to resolving MIN_long (which I guess should say MIN_INT128) so
the expansion troubles aren't trivial to fix.
Ah, this was rather a case where unsigned __int128_t wasn't a valid
type. Commenting out the INT128 support gives a complete build with no
unexpected warnings but math is still broken:
Pike v7.9 release 5 running Hilfe v3.5 (Incremental Pike Frontend)
-0x8000;
(1) Result: 2147483648
Fix committed for the UINT128 problem on the of branch.
Bignums in general work (and they did before too), but there are edge
cases that breaks:
Pike v7.9 release 5 running Hilfe v3.5 (Incremental Pike Frontend)
-0x7fff;
(1) Result: -2147483647
-0x7fff - 1;
(2) Result: 2147483648
-0x7fff - 2;
(3) Result: -2147483649
Yep, it's F_NEG_NUMBER in amd64.c that's broken.
As far as I can see its the latest change, which turned the macro
into fuctions. I think its overall quite mysterious under which
circumstances gcc (or clang) optimize that check away. In particular
because compilers change and something that works now might break
tomorrow.
I don't know for
The macro I'm borrowing renames variables to prevent accidental use.
There are still places where variable usage is conditioned on other
defines (e.g. POLL_DEBUG) and I'll leave them unchanged instead of
creating a more complex wrapper.
Pushed now. If anything breaks please let me know.
This cleanup was kickstarted when I tried to compile 7.9 with Clang. I
still see wrong execution in e.g. integer/bignum handling so I hope to
have more patches in the coming days. One of my other findings was the
commit aabfb4f04c5
Too bad with the name clash, but a Pike-prefixed name would also be
quite annoying due to its length.
Do you mean that my latest changes introduced the gcc overflow issues
or only that you noticed them as of now? I'm also concerned about the
overflow detection method using costly integer division
When compiling Pike with clang I get massive amounts of warnings for
unused variables. This detracts quite a bit from finding more serious
errors and I've got a patch ready to remove them. However, before
pushing that I'd like to hear if people have objections to changing
declarations like in
Grubba is on vacation this week so I don't expect him to see your
message for a few days.
Is the code generation problem a gcc bug that will be fixed in a
future gcc 4.x release, or is it an incompatibility that Pike must
accomodate (like 7.9 has)?
Too bad those pages consistently crash latest Safari (at least for me,
anyone else more successful?).
Seems to be the xlink:href pointing to a page in the page cache that
is the trigger, and since all graphs contain a link to the current
page that condition is always satisfied. Even if we
Nice! Can the unnecessary crossing lines be detected and removed by
reordering the classes? I.e.:
| Class Parser.XML.Tree.SimpleRootNode
| Inherit graph
|
| predef::Parser.XML.Tree.AbstractSimpleNode
| |
| | predef::Parser.XML.Tree.VirtualNode
| | |
|
You can subtract an array from a mapping to remove keys. With automap
it would then become:
mkmapping(a-name, a[*] - ({ name }) );
You can also apply `- if you prefer that:
mkmapping(a-name, map(a, `-, ({ name }) ));
Notice that - automatically applies to all array entries without the
need
The issue isn't necessarily the hashing but the fact that you need to
have this globally synced instead of e.g. creating a thread-local
string pool. Still, I agree with you that fundamental properties of
mappings etc are based on string uniqueness. There are other
low-hanging fruit that should be
I'm not sufficiently versed in the Pike internals to comment on your
alternatives, but I too would welcome any CF/NS runloop integration
improvements.
Incidentally, on the latest releases of Darwin, the default compiler
(with is llvm-based) aborts when compiling mapping.c. I still use
regular gcc, so haven't attempted to report the bug.
I've seen that too. It's normally sufficient to manually compile
mapping.c with gcc and re-run make
Dropping the module from the site configuration is perhaps enough?
I couldn't access it earlier today. Routing problem?
Not sure what happened here but it doesn't seem successful:
dark-castle:pike79 $ git config branch.7.9.rebase true
dark-castle:pike79 $ git status
# On branch 7.9
nothing to commit (working directory clean)
dark-castle:pike79 $ git pull
warning: no common commits
remote: Counting
Here's the config (the rebase flag was set just prior to the
procedure):
dark-castle:pike79 $ less .git/config
[core]
repositoryformatversion = 0
filemode = true
bare = false
logallrefupdates = true
ignorecase = true
[remote origin]
I've already performed reset --hard now so I don't know if forensics
will be successful.
dark-castle:pike79 $ less .git/logs/refs/heads/7.9
37f43e3a6e557681ba693d5a8a587a84bf649414 Jonas Walldén jon...@roxen.com
1291069055 +0100clone: from git-p...@pike-git.lysator.liu.se:pike.git
37f43e3a6e557681ba693d5a8a587a84bf649414
I think I'll pass. :-) Anyway, thanks for the hints.
OS X is case-insensitive but case-preserving.
Well, sounds like a git bug then.
There seems to be a core.ignorecase config option that looks relevant.
Maybe worth trying for anyone on OS X who haven't updated yet.
Yep, seems that pulled in the conflicting branches:
dark-castle:pike79 $ git config --global core.ignorecase true
dark-castle:pike79 $ git pull
From pike-git.lysator.liu.se:pike
+ 108b128...5c8e890 branches/hubbe - origin/branches/hubbe (forced update)
+ 4cff4c1...68b88f5
Indeed, that's odd.
dark-castle:pike79 $ git show-ref branches/hubbe branches/Hubbe
06983fde1434ab5470b2fb656c40db32b2e6a971 refs/remotes/origin/branches/Hubbe
5c8e89060d917cb3b06f831ed7df43e52b125c5c refs/remotes/origin/branches/hubbe
I think the problem is that the original author of DNS_SD was
working from the Mac (Bonjour) APIs and at the time the compat lib
offered a convenient way to offer that support elsewhere.
Correct, when I wrote it I targeted it at OS X.
The last time I looked at Avahi (years ago, though) it
I understand your reasoning but disagree that the worst-case effects
of this type of change warrants a 7.9. But if we get a 7.9 soon I'd be
more positive; maybe we can develop there and backport if it turns out
to be an improvement we want to distribute asap.
Regarding the identifier cache, is
We know what we have
I can't let this one pass. :-) How do you know when the existing
implementation gives repeated collisions? I assume program IDs are
pretty unstable, particularly in Roxen where module loading can happen
anytime. And since collisions cannot be diagnosed from the application
I've brought that up earlier and I still get thumbs-down from other
people regarding such changes in the 7.8 branch. In my opinion a
variant which is overall faster but where the slowest cases happen to
shift to new places is a reasonable optimization even in the current
branch but I'm not making
Site is down now?
Thanks. I wanted to double-check the numbers before saying that you
should switch from total to user in the scraping code. The total
value includes the launch time of Pike itself and is irrelevant to the
particular benchmark that is run.
Good, now the numbers look reasonable. If only the columns were
sorted... :-)
Nice! Does _261 mean that it's based on the Darwin build box? Also,
what determines the column order?
Right, I had considered your first case already and it is safe.
Passing the mapping by reference would however be good anyway since
that would avoid repeating the test for complex content but I never
completed that solution since ONERROR setup/cleanup got messy.
Exactly what I planned, though after Mast's note I might want to wait
for that to be resolved as well.
The first program below illustrates the overhead of array copying and
zeroing:
int fn1()
{
array arg = ({ a, b, c });
return sizeof(arg);
}
int fn2()
{
array arg = allocate(4711);
return sizeof(arg);
}
void main()
{
int dummy;
int t0 = gethrvtime();
Good suggestion, I wasn't aware Pikefarm ran benchmarks all the time.
Are there any tools on the site to chart the results?
I've got a patch for copy_svalues_recursively_no_free() that I'd like
to commit. The biggest gain is that the temporary mapping is only
allocated when recursing into arrays/mappings/multisets with complex
elements, and we now avoid placing simple elements in the mapping
where they serve no
o index an object with the same index you just indexed it with (keep a
small cache for indexing, have assembler code for checking the case,
then a 'slow' fallback version)
I made an attempt at this a while ago but gave up when I noticed that
refcounting rules for objects and functions are
And yes, more revolutionary ideas would be fun to discuss sometimes. I
glanced at the LLVM tutorial for integrating its SSA/code-gen
libraries, and if MacRuby can use it, can't Pike?
I'd like to start a discussion on performance issues where I feel we
can do a lot better with reasonable effort. My test case was initially
the XSLT module in our CMS but with tools such as Shark (an OS X
sampling profiler) I've encountered some interesting results that I've
now reproduced in
Is this malformed format widespread enough to warrant a change in the
library? Why not just fix the string before you feed it into
Standards.URI and maybe report the site/tool that generates it?
Ok, if it's valid then it's a different story.
I just noticed that Locale.Charset treats US-ASCII and Latin-1 as
equivalent encodings. I'd like to implement a separate encoder for the
former so that I can get the custom substitution string/callback
invoked for everything above 127. Any objections or should I reserve a
new name (suggestions
I found these aliases:
ascii us usascii isoir6 iso646us iso646irv1991 367 ansix341968
Looks ok?
Ah, I was thinking bits and thought high half was a bit too much. :-)
Therefore I disabled the MACOSX_DEPLOYMENT_TARGET=10.3 macro. Not
sure what negative impact this has. People with MacOSX experience,
please look into it (I merely tested it on my Leopard MacBook).
We're still building on 10.4 and 10.5 in various combinations of
PPC32, x86 and x86_64 and I'd
...but AvailabilityMacros.h indicates that it should already be done
through MACOSX_DEPLOYMENT_TARGET. Hmm, perhaps the latter isn't set in
enough places during Pike builds?
Actually, we *want* __DARWIN_UNIX03 to be set to 1.
At least on a MacOSX Leopard Macbook we do, because that one returns
102 from the kernel incase of a not-supported error.
But errno.h says there is a libc stub that converts it back to 45 if
it's not set.
Anyway, I don't ask for builds
I'm also unsure about the %O[%d] style. Shouldn't there be a + sign
before the bracket to match how strings are shortened in backtraces?
How do I know if I use services from eureka?
Thanks, that made things clearer!
That doesn't run m_delete on the original mapping but instead copies
all remaining entries to a new one.
That code will copy and rehash data in your in string every
iteration. At some point I implemented %*4711s in sscanf() just to be
able to avoid that bottleneck and it gave significant savings in my
test case (the XML markup repair code in Roxen CMS).
Well, not too bad:
sscanf(data, %* + pos + s..., ...);
It's not impossible to think of an optimization that uses that format
when the programmer writes
sscanf(data[pos..], ..., ...);
as long as the sscanf() return value is adjusted. Maybe Grubba fixed
that already?
I'm happy with Marcus' last change.
I'm confused about the way to refer to CMS. It's got a full XPath
implementation as well as SloppyDOM (which is inspired by XPath and
implements a small subset that should be identical in syntax to XPath
for the features it supports) and that's it. There is no other
semi-XPath or almost-XPath
Roxen CMS was capable of a rather friendly subset of XPath
(including, IIRC, mostly everything except document()
Your memory is not correct. There is no XPath engine in SloppyDOM,
just simple step patterns, node tests and a few predicates. There are
no variables, expressions, functions or data
Query.pike keeps the con variable global in the object so I assume it
is used later. Inside Stdio.File there are checks if the socket was
opened earlier and the behavior changes accoringly.
Passing the remote address to open_socket() would be semantically
quite different from providing a local
Though moving the DNS lookup in Query.pike::thread_request() ought to
fix that since we'd check the IP instead of the host name.
Stdio.File::open_socket() doesn't have such a parameter that I can
find. Which one are you referring to? The host parameter seems
intended for binding local interface only.
If you'd like to rework Query.pike to get rid of open_socket() I see
no problem in changing that. For now I'm inclined to
Avoiding power-of-two is something wikipedia (Hash table) comments
on:
Some older hashes are even worse, requiring table sizes to be a
prime number rather than a power of two, again computing the bucket
index as the hash value modulo the table size. In general, such a
requirement is a sign of
Try this first in your tree:
pike -x rsif 7.7 7.8 `find . -type f -name Repository`
I was going to say that reverse() probably did 32-bit or 64-bit memory
operations but that wasn't true either. Anyway, plenty of
vectorization possibilities in reverse(), search() and string hashing
in general.
By the way, what CPU did you test on? On older hardware a program
could benefit
I'd much rather see a search_reverse() that operates from the end of
the string (or given data type). It would be a bit more code but the
result would be more consistent and predictable.
Though Grubba's commit message didn't reveal so the change is only
related to dmalloc. And speaking of algorithms, check out Introsort
which is a O(n lg n) hybrid of quicksort and heapsort (already used
elsewhere in Pike if I remember correctly).
Time for a follow-up. This bug seems most likely caused by dmalloc
itself not being threadsafe in its handling of internal structures. A
hacked version that Grubba and I put together got rid of the problem
temporarily but I leave it to the dmalloc experts to develop a
long-term solution.
I'd like to enable native Java methods on OS X x86_64. Unfortunately
the current configuration code fails to find ffi headers in Apple's
standard location (/usr/include/ffi/ffi.h) so I've got a small patch
for that:
ceylon:Java $ cvs diff -u configure.in jvm.c
Index: configure.in
I've now tracked down and fixed the bug. Ironically it's Mirar that
caused it (granted a long time ago) by #define unsigned int size_t
which isn't valid on a 64-bit machine. I believe it could have caused
overwriting of as much memory as the resulting JPEG image occupied
outside of the initial
Ok, found and fixed that one as well. But of course fate deals me yet
another one:
#0 debug_fatal (fmt=0x10039ae30 really_free_memloc got invalid pointer %p\n)
at /home/jonasw/pike/7.8/src/error.c:632
#1 0x00010014df83 in really_free_memloc (d=0x114727ac0) at
You added a call to snprintf before checking that in so now the
Windows build is broken:
operators.obj : error LNK2019: unresolved external symbol _snprintf
referenced in function _o_cast_to_string
pike.exe : fatal error LNK1120: 1 unresolved externals
1 - 100 of 133 matches
Mail list logo
