Stefano Mazzocchi wrote:
> [For those of you who didn't study elettronic, MOSFET transistors have
> sources, drains and gates. Sources generate electrons, drains consume
> them and gates control the flow]
MOSFETs, aahh listen to the music...
For those who do not know, FETs (Field Emission Transistor) transistors
behave more like tubes than their PNP or NPN solid state cousins. As a
result, they tend to have a more pleasing sound when they approach
saturation (they still clip a little hard but you soften the sound
some). MOSFET (Metal Oxide Semiconductor Field Emission Transistor) is
a type of FET with a small insulator on the gate.
Since I studied music recording, and Studio Electronics, I know exactly
what Stefano is talking about.
BTW: there is a slight correction in the definition of Sources and
Drains in regards to FETs. The source is where electrons enter
(generated by an external source like a battery or socket) and the
drain is where electrons leave. The gates control the amount of
resistance that the FET, and consequently the the current flowing
through the FET.
I love FETs. :)
> The original idea of a pipeline was
>
> g -> t* -> s
>
> you all know this very well.
>
> Then I realized that the same pipelines shared the same properties, so I
> came up with the notion of resource views, which required a way to 'get
> out' of the normal pipeline flow. This was implemented with 'views' and
> 'labels'
>
> g -> t* -> s
>
> [label] -> t* -> s
>
> so that we could ask for a specific 'view' and do something like
>
> g -> t -> t* -> s [normal view]
> |
> + -> t* -> s [another view]
>
> Then we wanted to merge different pipelines with 'aggregation', so we
> needed a way to do something like
>
> g -> t* -> A -> t* -> s
> ^
> g -> t* ---+
>
> this created the notion of 'internal pipelines', those pipelines who
> don't need a serializer because their sax events are aggregated
> directly.
Now, I want to interject that it is easier to understand the usefulness
of Aggregations than Views. For that reason, we need another way of
viewing the system.
> Now, Vadim proposed the notion of having serializers that write on
> output streams which are not connected to the requesting agent.
>
> Something like this
>
> request -> g -> t* -> MX -> t* -> s -> response
> |
> t*
> |
> s
> |
> v
> stream
>
> this reminded me a lot of transistor 'drains', the place where the
> electrons are consumed.
I understand the requirement. What we need is the ability to
initiate asynchronous pipelines that consume information from a
request. I say asynchronous because in this case, we are not
merging the information back into the critical stream.
There has been some talk about making SAX streams concurrent (i.e.
an aggregation of several pipelines all running at the same time
to reduce time to the client). So the Multiplexer/Demultiplexer
idea does have some merit.
However, let use take the abstraction up from the FET analogy.
Since I come from a recording background I will use the analogy of
a patch bay and console. The purpose of a patch bay is to route signals
through what is known as outboard equipment (processors outside the
console), and back into the console. The Console takes several
signals and combines them into only a few (stereo, 5.1 format, 7.1
format).
In a typical recording session, we will take the input of several
instruments and route them directly to the media. Because the
musicians all want to hear each other, along with what is already
on tape, the mixer is used to play back what is on tape and provide
a working mix for the musician. Typically each musician wants to
hear themselves the loudest ;).
During this stage, the console has several output options. Depending
on the console model, you can have as many as 8 different mixes for
the musicians (more than enough) and still have yet a different one
for the engineer. There is only a little bit of processing done to
make the artist feel comfortable.
To bring the analogy into the Cocoon world, each "pipeline" represents
a signal. The signal comes in through the preamp to amplify it, through
the recording media to record it, through EQ and compression to
transform it, and finally through the power amplifier so we can hear it.
In Cocon, the Generator acts as the preamp. It takes an arbitrary
source and converts it into a usable format (SAX events). Typically we
skip the "recording" phase for most applications, but I will get back
to this later. The EQ and compression are analogous to different types
of transformers. Lastly, the power amp is equivalent to the serializer.
Stepping back one more abstraction, we have Transducers, Signal Routers,
and Signal Modifiers. Microphones, Tape Heads, Speakers, A/D/A
converters are all types of transducers. They convert one form of
energy into another. Also as you can see by the inclusion of
microphones and speakers in the list of transducers, the concept of a
transducer does not indicate a specific direction of signal flow. In
fact, microphones can be used as an inefficient speaker and speakers
can be used as an inefficient microphone (I know of one engineer who
uses a 6" speaker to record the bass drum).
In Cocoon land we have three transducers: Generator, Serializer, and
Reader. This is ok, because it reflects real life.
The Signal Modifiers are EQs, dynamics processors, digital signal
processors, and other specialty devices. EQs modify the frequency
content of the signal. Dynamics processors like compressors and
expanders modify the amplitude content of a signal. Digital signal
processors typically modify the time domain (phase) of a signal,
although they work equally well for all areas of signal modification.
This is where all the fun happens.
In Cocoon our signal modifier is the Transformer. There are many
types of transformers that are powerful and easy to use.
Moving on to signal routers, we are talking about the combination of the
patch bay and the console. The console has limited routing capabilities
because its main function is to combine a large number of signals into
a small number of signals. It delegates the responsibility of complex
routing to the patch bay. The patch bay is the real signal router in
the system. We can use the patch bay to split any signal to two or
three outputs to allow different outboard devices to modify the signal
concurrently. The inputs and outputs of those devices and all the
channels on the console, and all the tracks on recording media are
connected to the patch bay. The patch bay is usually soft-wired so that
there is a default flow for the signals that can be easily overridden
by inserting patch cables.
The closest thing we have to a signal router in Cocoon is the Sitemap.
The sitemap does not provide a soft-wiring feature that assists in
creating a solution. We only have hard-wiring which means that we have
to explicitly state how we handle each invokation of the sitemap. Many
HTTP servers provide a soft-wiring that matches urls to the file system.
This is an easily understood default that is as fragile as your file
hierarchy. I don't have any thoughts on how to provide a middle of the
road soft-wiring for the sitemap.
One of the areas that we are facing (as the additions of views and
aggregations attest) is that the sitemap semantics do not provide a
balanced and flexible approach to signal (request/response) routing.
I think that the addition of Actions can attest to that. Lets take
Cocoon and instead of a web application platform lets place it in the
light of a web services platform (all XML all the time).
Let's say we have a complex system that takes a new item submission from
a manufacturer and puts it in the queue for a retailer to review.
Typically there are a set of canned responses that the Web Service has
to provide. Those responses would be: "Success", "Internal Error",
"Incorrectly Formatted Request". In the best of worlds, we would
directly process the incoming XML into the database and shove out a
response from a different source. An example flow would be:
"Internal Error"->Response
+
Request->Validate->Database
| +
| "Success"->Response
+
"Incorrectly Formatted Request"->Response
As you can see, the incoming request's destination is the database--not
the message response. In this system we have 3 transducers: Request,
Response, and Database. To ease development, we would want to make
a direct line from the request to the database because the input request
is formatted as XML. Also, we want to kick off a canned response as
soon as it is safe to do so. If there is any validation error, we want
to kick out a response while we are cleaning up resources. If there
is a database error, we want to kick out a response while we are
cleaning up resources. Once the request is properly processed we can
safely return the canned "Success" response.
Currently we have a complex way of accomplishing these goals:
We use Actions to perform validation and database entry. Using clues
from the return values of the actions, we select the response as our
generator and folow it immediately with the XML serializer.
It would be better if we could separate the input and output streams
so that the output stream can be kicked off as soon as we know which
one. With a separate input stream, we can get rid of the need for
Actions altogether and have a collection of kick-but transformers and
transducers that take the request and directly place it in whatever
storage device we need.
This allows simple things to remain simple (i.e. regular web sites
only need output streams), but allows things to be flexible. For
instance, let's take the same example and have it kick off a couple
other pipelines. One is a pipeline that processes payments from the
SOAP header informaiton, one is a pipeline that informs the retailer
that there is a new product in their queue. Now we have three
separate destinations that no longer need to keep track of each other.
With the new CommandManager and event processing tools in Excalibur
scratchpad, it is easy to kick off an asynchronous pipeline without
worrying about a multiplicity of threads (we can have a controlled
number of threads that handle the truly asynchronous events).
Now, when we bring this infrastructure into a portal, we have to
worry about recombining pipelines. For instance, one request will
have several inputs, all needing to be combined into one output.
This is a slightly different problem that is not as friendly to
asynchronous processing. The main reason for this is that SAX events
are inherently ordered. You will have to cache parts of the other
pipelines until they are needed, a requirement wich adds overhead
in the form of RAM or disk IO. I doubt we will better our response
times significantly if we allow asynchronous demultiplexing (or
mixing). I am open to more discussion on the subject though....
Comments?
(Stefano, I know I went a completely different way than you started
out but I think we have something to throw in the mix here that will
get us closer to where we want to be).
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, email: [EMAIL PROTECTED]
