Dear Peter,
I believe we need to clarify some of 'requirements' first. Maybe, there
could be several approaches for EXI1.0 or maybe we need to propose
something to EXI1.x(maybe x=1), upon such requirement discussion.
Topics may involve:
- Minimal client requirement
- Minimal server requirement
- Schema management
- Efficiency requirement
- ...
I'll make another thread on requirement discussion following this mail.
(2013/03/13 12:26), Peter Waher wrote:
§3.3 in the XEP handles this. The server is free to add cache rules
to avoid explosion of number of derived schemas. The server is also
free to reject uploading or downloading requests, for any reason
(§2.4)
Agreed. BTW, is the schema itself encoded in EXI or plaintext? The
example seems to be in plaintext but it could be EXI-encoded binary.
Then upload in xs:base64binary will become straitforward Binary type
upload, far more efficient.
The problem with this approach, is that name seldom change,
especially during development. And a slight change, a new attribute,
a new element, etc., will completely change the compression.
Furthermore, errors produced in this way will be extremely difficult
to debug. An efficient and fool-proof way to communicate using
different schema versions (having the same namespace and schema IDs)
is necessary.
§2.4 also proposes the possibility to install such schema files
manually on the server. The XEP allows for different scenarios.
Agreed. We need different mechanism for development stage and deployment
stage. I believe schemaLocation and hash is sufficient enough.
It is supposed that the EXI compression engine works in XML fragment
mode, where each stanza is compressed separately. I don't see that
self contained elements would not be required in this case.
Could you elaborate this? Are you thinking something like this?
[EXI Header][EXI Option]
SD
SE(stream)
SE(some-stanza) ... EE(some-stanza) // single stanza
EE(stream)
ED
(padding)
SD
SE(stream)
SE(some-stanza) ... EE(some-stanza)
EE(stream)
ED
or
[EXI Header][EXI Option]
SD
SE(some-stanza) ... EE(some-stanza)
ED
(padding)
SD
SE(some-stanza) ... EE(some-stanza)
ED
or something else?
(for those who are not familiar with EXI, SD/ED: start/end document,
SE/EE: start/end element)
To be honest, I have never implemented EXI Fragment Document Grammar so
my understanding may wrong.
Tables
should not be reused between stanzas, since tables can be very
small, but, as you point out in your example, number of possible
strings may be large (for instance many different JIDs). However,
possible strings within a message are much smaller, making also
references to tables shorter within the bit-packed message.
I think this makes far worse compression ratio (according to my experiment).
Finally it's up to the implementor how to setup the EXI compression
engine. Some may feel bit-packed is better, some that byte-packed is
better.
I'll add an implementation note in the XEP regarding this.
In my experience in SEP2[1], we didn't need full option for it. You may
interested in EXI Profile[2], too.
[1] SEP2: smart energy profile 2
http://www.zigbee.org/Standards/ZigBeeSmartEnergy/Version20Documents.aspx
SEP2 editors have had long discussion on how to use EXI in constrained
devices and agreed to use it in schema-informed, non-strict grammar with
schemaID and schema negotiation for linear extensibility.
[2] http://www.w3.org/TR/exi-profile/
This is a vital aspect of this proposal. For sensor networks and
IoT, especially wireless sensor networks, buffer size is first
priority. Therefore, EXI compression should be done with as much
information about schemas as possible.
Agreed.
We also see this as an important aspect of this proposal: Most
sensors will have pre-compiled code, often (semi-)automatically
generated from schema files, for compression and decompression of
EXI content. Therefore, the proposal Includes the possibility for
the client to reject the connection if parameters are not as
expected.
Agreed, but I think there should be some greatest common divisor (or
'please implement at least this functionarity'-set) for better
interoperability.
(from the other mail)
I've thought more about your proposition regarding session-wide
string tables vs. having to clear tables and buffers between stanzas.
Both have merits, depending on use case.
So, I added an option controlling this behavior: sessionWideBuffers,
as well as descriptions describing the differences in how they work.
As well as a note regarding flushing between stanzas.
We need to be careful to add something on existing standards. In other
words, existing EXI processors should be applicable for this proposal.
Best Regards,
// Yusuke DOI <[email protected]>
