On 7/29/2011 1:05 PM, David Barbour wrote:
On Fri, Jul 29, 2011 at 3:12 AM, BGB <cr88...@gmail.com
<mailto:cr88...@gmail.com>> wrote:
<snip...>
nothing interesting to comment/add...
Snow Crash: "dot pattern from space -> brain-damage
Ah, yes, that wasn't the bit I wanted to create from Snow Crash. Just
the vision of the cyber-world, with user agents and a multi-company
federated but contiguous space.
fair enough.
Linden Labs tried to do similar with Second Life, but it hasn't really
caught on very well in-general.
however, most prior attempts: VRML, Adobe Atmosphere, ... have generally
gone nowhere.
I once considered trying to pursue similar goals, essentially
hybridizing 3D engines and a web-based substrate (HTTP, ...).
I once built something simplistic (3D chat-room style) IIRC built mostly
on top of HTTP and Jabber/XMPP (however, its capabilities were limited
mostly as XMPP uses textual XML and bandwidth throttling). this wasn't
maintained mostly as it was severely uninspiring.
if anything is going to really work, it is probably going to need to be
based on a more game-like technology and architecture (and ideally try
to deliver a more game-like user experience).
an example would be if it delivered the "look and feel" of something
like Quake3 or Doom3 or Unreal Tournament or similar, but was a more
open and extensible architecture (based on HTTP servers and otherwise
open protocols).
potentially, messages could be delivered over an XMPP-like message
protocol, albeit ideally it would need higher bandwidth limits,
server-side scene-management, and probably a compressed and/or binary
transport: XML+Deflate, WBXML+Deflate, SBXE(1)+Deflate, or EXI.
1: SBXE is my own binary XML format, which is a similar format to WBXML,
but is a little more compact and supports a few more standard XML
features (such as namespaces). in my tests, SBXE+Deflate tends to be
about 30%-40% smaller than Textual XML+Deflate (which is in my tests
often 75%-90% smaller than uncompressed XML).
however, given that SBXE does not use schemas, but builds a model from
prior tags, so its relative overhead is higher for small disjoint
messages, as it tends to need to retransmit tag names, ... but tends to
do better in my tests for larger messages (it will partly predict later
tags from prior ones). it is also reasonably Deflate-friendly (Deflate
usually compresses down by around an additional 25%-75% in my tests).
note that XML+Deflate is typically smaller than raw SBXE.
I have not personally tested against EXI (EXI is a bitstream-oriented
schema-based encoding, and is a W3C standards recomendation).
I am personally not so much of a fan of schema-based encodings though,
as then one has additional hassles (such as having to write schemas), as
well as hindering or eliminating the use of free-form messages (IMO, it
is much like the matter of Vector Quantization and codebooks vs Deflate:
VQ+Codebooks can be potentially smaller, but Deflate is far more generic).
Deflate+Textual XML generally works fairly well though in terms of
shaving down size.
note that my current 3D engine is using an S-Expression based protocol
for client/server communication (was less effort and potentially
higher-performance than XML), but a hack could allow shoving basic
S-Exps over an XML-based message transport.
(foo (bar 1 2) 3)
=>
<sexp xmlns="...">
<T n="foo"><T n="bar"><I v="1"/><I v="2"/></T><I v="3"/></T>
</sexp>
I started out with language design and VM implementation some time
around 2000.
at the time I was using Guile, but was frustrated some with it.
for whatever reason,
I skimmed over the source of it and several other Scheme
implementations, and
threw together my own.
My start in language design is similar. Guile was one of the
implementations I studied, though I did not use it.
I used it some, but at the time (late 90s) is wasn't very good (what
really killed it for me though was that it was hard-coded to call
"abort()" at the first sign of trouble, which was hardly a desirable
behavior in my case).
most of my stuff generally handled most error-conditions by returning
"UNDEFINED", which was generally intended as a "catch-all error
condition". ideally, I would go add proper exception handling eventually
(having everything fail silently with UNDEFINED popping up everywhere is
ultimately not very good for debugging things not working, but the issue
is that mixed-language exception-handling is an awkward issue, so it
hasn't really been dealt with effectively).
/Too much power with too little perspective - that is the problem./
I doubt "power" is the cause of this problem.
being simplistic or made from cruft or hacks are far more likely
to be their downfall.
Power, in the absence of perspective, leads humans to develop
cruft. Hacks are how we compose cruft.
Self-discipline and best practices are ways to 'tame' the use of
power, when we have too much of it. Unfortunately, these are not
practices that scale. Self discipline won't help when your libraries
suck because /someone else/ was not disciplined.
We cannot universally fix perspective. Humans /need/ the ability to
specialize - it takes ten years of passionate education to gain a tiny
fraction of useful human knowledge today. Those numbers will only
further diverge over time, and filtering the useful knowledge from
speculation and drivel will only become more difficult. It is too easy
to miss lessons of the past. That problem will only get worse.
Between power and perspective, power is the problem we can most
effectively tame by technological means.
IMO, this still doesn't seem like a "power" issue, but whatever...
I regard this as "ivory tower VM design", the likely biggest
example of this strategy being the JVM, however, it is not unique
to the JVM (most VMs have it to a greater or lesser extend).
The JVM was not designed in an ivory tower, BGB. Further, it was
pressured and pushed by Sun's marketing before its developers thought
it ready. JVM was a classic example of 'worse is better, get on base
first and patch up later'.
I means "ivory tower" in the sense of being isolated from the world and
intending to replace and remake it in some "better" way, with concerns
with playing well with the outside world being largely disregarded.
For ivory tower languages, I suggest you review Haskell,
Oz/Mozart, Maude, Coq, Lustre, Charity, Bloom, Mathematica, and
Scheme. I'm certain you could learn a lot from them.
several of these languages have the same basic issues.
but, I meant more in the isolationist sense, rather than in the sense of
"it was made by academics", however I have ran across a few academics
who have generally held these attitudes as well.
people I call "Q's" because they are sort of like Q from Star Trek,
mostly going around and accusing everyone else of being
stupid/ignorant/... and similar, or being condescending towards any sort
of applied or commercial use of knowledge, ... or go start throwing
insults at anyone with religious beliefs and making comments about
"flying spaghetti monsters" and strawman arguments and so on, ...
mostly I have seen this among math and physics people though.
admittedly, I really don't really like those sort of people.
People who use "ivory tower" as a snub offer the impression of
being close-minded and prejudicial. For impressions sake alone, you
might want to work on that.
seems common enough though, anyways I was not intending it as an attack
on academics in general though, but rather a particular way of
engineering VMs and similar.
one would otherwise need some other term for "closed isolated VM design".
I don't think the problem actually requires anywhere near this
level of resources, rather it more requires FFIs capable of doing
a bit of "heavy lifting" needed to integrate disparate languages
and technologies in a relatively seamless and automatic manner.
a good portion of the problem then boils down to data-mining and
heuristics.
source code/headers/... itself actually provides a good deal of
information for how to interface with it.
I agree that we can automate a lot more of FFI/foreign-service
integration, and that doing so is a worthy goal.
But for integration to be 'seamless' is much more difficult. I don't
expect /seamless/ integration even with a common programming model,
due to variation and mismatch in data models. (Reactive and RESTful
help, but there are fundamental ontology issues.) Foreign service
integration is faced with many extra challenges: different concurrency
and concurrency-control models, different evaluation-properties
(strict, lazy, parallel, beta-reduction), different type
systems, different error-propagation models, different memory
management models, different data and control-flow models, different
temporal and spatial properties, different distribution models, and on
and on and on.
Only when two languages are conceptually 'close' to one another (e.g.
two procedural languages) is integration typically effective. I'll
repeat that: effective integration requires that the languages share a
lot in common - i.e. a common basis, foundation, substrate.
it can glue together dynamically and statically typed languages, and can
go as far as to marshall data structures, ..., which is probably "good
enough".
the one language I can't really readily interop with is Java, but this
is partly because the design of the Java language and tools makes
transparent interfacing with much of anything not-Java fairly awkward.
technically, I can far more easily interface C# (on MS's .NET) and
BGBScript than I can Java and C. I had done a quick test to this effect
before, IIRC using a little bit of C++/CLI code as glue, and then
eval'ing BS fragments and calling into BS functions from within C# code
in Visual Studio.
also, it should theoretically work the other way as well (a little bit
of C++/CLI glue should allow reasonably direct calls from BS -> C#).
however, I have my own issues with .NET though (mostly that I couldn't
get C++/CLI to work with Mono, and I am not as motivated by Windows-only
options).
All that aside, 'the problem' I was discussing seems to be independent
of foreign services integration: today, we cannot effectively compose,
integrate, and scale frameworks, DSLs, and applications developed /in
the same/ programming model or language. Even /without/ the extra
complications of foreign service integration, there is a major problem
that needs solving.
fair enough.
these are more library/API design issues though.
On a side note, I have spent a lot of time developing some excellent
approaches for foreign-service integration. I've rejected use of a
'Foreign /Function/ Interface' in favor of a plug-in extensible
runtime. Each 'plugin' publishes modules and capabilities to the
runtime, along with metadata to distinguish them. These are made
available to application developers as objects or modules within the
language. For the runtime itself, I supported built-ins, dynamic
plugins, and separate executables (via Unix or TCP socket).
Applications are also able to register their own caps as modules, and
thus provide services..
The resulting model is much close in feel to how browsers use plugins,
how web-serves use CGI, or how service-oriented architectures locate
dependencies, but has the advantages of being open, extensible,
flexible, distributed, securable, resilient (i.e. allowing fallback
services), relatively easy to debug, language agnostic, and supporting
a continuous transition strategy for slow absorption of foreign services.
ok.
My original design for this was developed in 2008 and mostly
implemented in C++ in 2009, but I've since transitioned to Haskell and
will need to start from scratch. That's okay, it should be much easier
the second time. Parts of my implementation, especially the 'vat'
model, are already designed with this integration in mind.
So I have not ignored the foreign service integration problem. I just
haven't considered it a difficult or interesting problem for a few
years now.
I don't really bother with the issue much beyond gluing the languages in
a relatively seamless way.
if an API is poorly designed, then this is its own problem.
there is still a lot that can be done without requiring
fundamental changes.
I agree. Problem is, many of the more interesting things I want to do
would be complicated, inefficient, unsafe, or insecure with our
current foundations.
granted, but expecting any sort of widespread fundamental changes is not
likely to happen anytime soon, so probably better is to "go with the flow".
one may parse code, and then compile it to a stack-machine based
IL, as personally I have had the most luck working with
stack-machines. IME, stack machines are fairly easy to reason
about from software, so I have had best luck with things like
temporaries/register-allocation/type-analysis/value-flow/...
working in terms of the stack.
The very idea of 'control-flow' is difficult to reason about -
especially once you add a dash of concurrency, parallelism,
reactivity, inversion of control, or heterogeneous memory (FPGA, CUDA,
distributed, etc.). Stack machines are quite /simplistic/ in the grand
scheme of things.
it doesn't bother with non-local control flow (jumps internal to a given
block).
anything outside of a given block generally was a matter left to the ABI.
in this case, jumps are basically a raw jump, but there is a little bit
of complexity here due to "stack-value phi" issues, namely:
at a label, any values on-stack have to refer to the same physical
storage (register or memory address), meaning that possibly at the jump,
the values need to be shoved over to where they need to go.
in my codegen, this was handled a little more naively:
at any point where virtual the stack layout was the same (same items
with the same types), then the physical stack layout would also match
(mostly because memory-storage for stack elements was always reserved,
and proceeded in a top-down order respecting value alignment and similar);
in this case, when a jump would occur, anything cached in registers was
flushed back to onto the stack or into variables.
with pure temporaries, things get a little more hairy, as then one has
to basically move the values from their current locations into the
target locations prior to the jump.
as for concurrency or parallelism, well, the codegen itself doesn't
care, it just assumes sequential execution.
shared variables would generally be handled by using the "volatile"
modifier.
other features, such as SIMD, async calls, ... can be added on without
otherwise effective the basic execution model (SIMD-based types are
supported, such as vectors and quaternions).
Seriously, we've been building some massive amounts of cruft around
these stack machines for decades now. If you have a language that
implements without complication on a stack machine, I hypothesize that
your language is just as simplistic.
many languages can generally be implemented fine on stack machines.
C and C++ work fine on stack machines.
Java and C# also are typically compiled to stack machines (both the JVM
and CIL are stack machines).
likewise goes for ActionScript (the AVM2 is also a stack machine).
likewise, any other languages which compile for either the JVM or .NET
also work on stack machines (Scala, ... or VB.NET, F#, ...).
all in all, stack machines are fairly well proven.
_______________________________________________
fonc mailing list
fonc@vpri.org
http://vpri.org/mailman/listinfo/fonc
