Hi,
To be honest i am glad there a feeling of curiosity and interest about
components.
I have seen the light my friends and I am a believer :-)
And there is hope for a "born again 4D component lovers" community.
I find it sad when I hear 4D developers saying that they tried components and
that they gave up.
Maybe I can share my experience with components.
I have been using components since day 1 (early adopter) and I am *very* happy
with them.
It really changed the way we write code.
I did a summit presentation in back 2012 Paris about "4D development
industrialization with components".
And I attended Peter's very good presentation on components last year in
european summit. Peter is also a fan of components.
Interestingly enough we use it in a difference ways (i'll come back to that),
but we have encountered the same problems/pitfalls and found some
solutions/workround.
A bit of context first...
We are a bespoke developer company, mainly 4D, a bit of Oracle and php/web,
etc.. (several dev, many projets, across several 4D versions).
We work in remotely (not coding together on the same 4D Server).
Before components we would have a common source library (a shell if you want).
We would move code with insider. But Insider is gone and maybe it was a good
thing.
Early on, the methods of our library were grouped in modules (which helped us a
lot to move stuff into components).
I looked into components as an alternative to 4D Insider.
I an idea of what I did not like :
- bloated components
- rely on too many human operations (boring, errors)
And what I wanted :
- reduce duplicated code across projets
- re-use technical code
- improve code quality
- fixe code once and share fixes across projects
- versionning
- avoid naming collision (a sort of namespace)
- performance (using compiled components in interpreted host is great
compromise)
- logs to diagnose stuff
- documentation
- unit tested code (in an ideal world)
- move more than just code/methods, sometimes resources, etc...
- share code between versions (I don't enjoy moving code and fixes
across libraries in several 4D versions)
- wanted to simplify updating code when changing 4D version
- modular approach : have access to a collection of component where we
could choose and pick according to the needs of a given project.
I came to the conclusion, early on, that our code was made into two parts :
- business code
- technical code
The business code is rarely re-used. Also, working on tables, etc... is not
simple with components (working with SQL or pointers) make things more
complicated and brittle in my opinion.
And I have never used component "local" data.
So I don't do business code in components. I know Peter has a different
approach on this. It is fine, there is no right or wrong. Juste different
taste, experience, needs and methods.
The technical code is in fact a collection of modules which can be, for the
most part, independant of each other.
So for instance we have written about 40 to 50 components.
Some projects are using 2 or 3 components. Some are using 30+.
Some are published with source code for free (aws_component for S3 for
instance).
Some are commercial (ociLib to help coding with 4D for OCI and provide
retro-comptibilty to the legacy "4D for Oracle" plugin).
The vast majority is private and used internally.
This is for instance, a part of our collection :
- BLB : blob stuff
- CRC : checksum (md5, sha1, sha2, sha384, sha512, crypto)
- CPDF : pdf handling stuff (using "Coherent pdf" binaries)
- DBG : log writing
- ENV : information about system (os, disks, etc...)
- FS : filesystem stuff
- HEX : hexadecimal stuff
- HTTP : http stuff
- HTML : html stuff
- IC : internet commands stuff
- IO : input / output (import and export, cdv, etc...)
- PDF : pdf printing stuff
- PRM : handling "environment" parameters
- PTR : pointer stuff
- MSG : alerts, messages and progress stuff
- OBJ : object stuff (well this one is Cannon's object module converted
into a component)
- SEM : semaphore stuff
- SMTP : smtp stuff
- TAB : array stuff
- TXT : text
- XDOC : auto documentation
- XML : xml stuff
- ZIP : 7zip stuff (compression and AES encryption)
- etc.
You get the idea...
Most of theses components are independent to each other.
Few higher level components have "strong coupling" because they explicitly call
another component. When you use a compiled application, 4D will detect the
missing component an complain when you open you structure.
Most have a "soft coupling" : most components spit out logs. For this, they
call the "DBG_component". But this is done with a call to EXECUTE METHOD (only
if the DBG component is installed). So that the component will work with or
without the "DBG_component". In effect, the "DBG_component" is optional.
But it is also ok to duplicate code in components. Let's have a look in the C
world. Do you see libraries always linking to the md5 system library ? No. md5
is a simple C algorithm therefore is is often duplicated and included and
statically compiled with the source.
So if you need a simple function from another component, don't be dogmatic and
avoid dependency hell. I prefer duplication code from one component to another
than introducing a dependency.
What I discovered also, is that components are a great way to embed binaries.
For instance I embed 7zip binaries (mac and windows for each architectures) in
the zip. It is sometimes "big" it it can avoid dependencies on system,
installing libraries, incompatible versions, etc...
Also you could embed php scripts in a component. Recently I wrote a component
for fellow developers in Czeck republic which did xmldsig with soap (http
client, xml and crypto signature). This was done in a component. To do this I
need crypto function from php to sign and get public from p12 password
protected certificate file. So I juste wrote the php function in a php file,
embedded in the resource dir and wrote the rest of the code in 4D. A bit of
unit test on all this. Et voila. This works great.
All our components and projects are in Unicode mode.
Also I still write most of my components in v12. And they can be most of the
time used in more recent of 4D v13, v14.
For 4D v15 I need recompiling because of Mac 64 bits missing in earlier
compiled components.
Of course it is difficult to benefit from new features (but it is sometimes
possible).
But I will soon move my components to a newer version (4D v15 maybe). And I
won't back port new features bug fix to v12 (it is a pain).
For instance, 4D introduce a nice "4D Progress" component in v13.
We our own component in earlier versions to do that "MSG_component". So I
updated our component in v13 and made it call the "4D Progress" component.
So when converting on of projects from v12 to v13, we just replace/update the
MSG component and *boom* we suddenly have "4D Progress" thermometers without
rewriting a single ligne in the host database.
By the way I check if a component is present only once at init. Then I use an
interprocess variable to see weather or not I should call the other component
with EXECUTE METHOD.
I haven't measured if EXECUTE METHOD has a significant overhead to be honest...
Yes there are things which are a bit tricky, like isolation host/component. It
is not always obvious.
There is an effective (and relatively simple) workaround : use callbacks !
The first one I had was "get pointer" from a variable name. Once you understand
the value of "callbacks", it becomes natural to write callbacks.
Some which I recently stumbled on : list, io filter, etc...
Yes documentation is not very explicit in terms of host/component behaviour. So
it sometimes you'll needs trial and error/proofing/testbed.
Moving further with callback, you can introduce IoC (inversion of control). The
"Hollywood Principle" : "Don’t Call Us, We’ll Call You".
For instance a component can read an Excel file and call the host callback a
method in the host to see what to do with the data found in Excel...
The callback can be called for each cell and the host code can decide what to
do with the data.
NOTE : for callback -> don't forget to use "share method between host and
component".
Most of these things when you think about it may feel a bit like a pain, but
not having this isolation would be WORSE !!!
Without this, components could really alter the behaviour of host and hurt you.
This would be not good.
Hint/tip : callback could be added automatically by component into the host on
init.
It is true sometimes it would make things easier if there was more commands
which could behave in either context of host/component with the magic *.
I use a naming convention. All methods of components are prefixed with a
component code.
For instance, for the blob component => component code BLB.
BLB_blobAppend => one underscore means visible and public (shared between host
and component)
BLB__somePrivateMethod => two underscore means hidden and private (note shared
between host and component)
We use lots of logs (another subject altogether) and more and more ASSERTS to
see if the component received correct parameters (check pointer pointer types,
nil pointer, assert errors, etc...)
Also when you make components, you have to realize that you effectively create
a "public" API and you have to think forward and to think backward
compatibility.
A new version of the component could break an app if you change parameters,
etc...
So it needs to be done in a careful way (and to be documented).
Therefore we are "versioning" the components.
We decided to use compiled components. We use the compiled component even when
we write code in the interpreted host. So there is no surprise.
Because we use compiled code, we cannot see comments. Therefore we needed good
documentation.
We ended up writing our own documentation system (with custom tags in header)
to generate pretty documentation which is inserted in the method "comment"
section and is visible in host.
We also sometimes, can fix an application, by just installing the new version
of the component if the bug is in the component. So we don't have to deploy the
complete application (and we don't therefore we don't need to test it if the
new component has been properly tested).
There are things I want to improve. I would like to make the process to making
a new version of a component more automatic (faster, less error prone, not
really interesting moving files around, etc...), maybe export source into git.
It is definetly possible I just haven't taken the time to so it.
Conclusion :
Components are a great way to re-use utility code.
We have adopted a modular approach and we are very happy with it.
This is just my experience and my view...
HTH
Bruno
**********************************************************************
4D Internet Users Group (4D iNUG)
FAQ: http://lists.4d.com/faqnug.html
Archive: http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub: mailto:[email protected]
**********************************************************************
