I don't follow the list any longer, but I do check in to look for answers.
Today I was looking for the an answer to "why don't my object variables on
fields automatically redraw when changed?" Didn't find it, does anyone have
a workaround? While I was checking on my question, I ran into this massive
thread on objects. As it turns out, I've been working in this area for a
couple of weeks and have a few thoughts to share. I haven't read the whole
thread, so apologies for repetitions. I don't have time for this post...but
here it is anyway. If you care about this issue enough to work on it,
you'll find a bunch of solid ideas in the TL;DR version. I don't have the
time to edit it into something shorter and better, but here is a very short
version:
* 4D's new system leads to very easy to break, brittle, hard to correct
coding problems. By default.
* You can use tables instead of objects. But do you want to?
* 4D dropped the ball on supporting complex types, so we each have to do
all of that work on our own.
* *JSON Validate* can help. Some.
* It takes quite a bit of work to write a custom type definition and
validation system but, if you need quality code, you don't have a better
alternative.
* Use constructors!
TL;DR version:
First off, some background about types. There are only a few very core
issues in language design...you could have spare fingers on one
hand....type is one of those core questions. What we've got in 4D:
* Primitive types.
* Arrays of primitives.
* Dictionaries/associative arrays.
We do *not* have compound types nor do we have objects. *C_OBJECT* is a
dictionary, not an object. We need dictionaries, so that's great, but the
name is misleading to many. Anyway, what difference does it make? Well, a
compound type has many, many advantages. It's a collection of elements into
a distinguishable format with *rules*. 4D's tables are the closest thing to
this we've got. But there are tons of times when you really don't want a
table, you want a pure, in-memory structure. Or at least I do. And every
other programmer in history does. Just saying.
What difference does it make that we don't have compound types (structs in
C, records in Pascal, whatever you prefer to call them?) It makes a huge,
huge difference. Here's some of what compound/complex types are good for:
* They're *types *(formally defined data structures) so you can pass them
around and get an interpreter, linter, or compiler to complain if you pass
the wrong type. You know how 4D uses longints for a zillion things, like
window refs, process ids numbers, etc. With a custom type, you could have
windowRef, or processId as distinguishable types. Pass in some rando number
to a method requiring a window reference and get an error from the
interpreter or, depending on how it's all coded, the compiler. I used a
simple type (longint) here as an example, but the same applies to more
complex structures.
* Type can embed rules right into the structured format. In a sense,
objects are complex types with behavior bound in. In a simpler case, you
can add ranges to elements - like "only allow numbers from 1-3" or "only
allow the strings 'Mon','Tue', or 'Wed'", force reals to round to a certain
number of decimal places, etc., etc. Again, 4D tables are the closest thing
a pure-4D programmer will be familiar with. And the field-level constraints
in 4D are pretty painful to use (required, etc.) because of how error
handling is implemented in 4D...so many people don't use field-level
constraints. I don't.
Okay, so we don't have types...how much difference does it make in V17. It
makes a huge difference, as people have clearly noticed already and are
exploring on this thread. I like the new dot notation, but there are some
massive issues with names. Such as:
* If you add a name to an object, the element is added. What if it's a
typo? You get a new element. No error. This is described as a "feature".
Well, sometimes it is a feature, sometimes it's a bug. There's no "strict
mode" choice. That's a missing feature in 4D. Or a design flaw, call it
what you will.
* Case-sensitive. I just can't say how stupid I find this. Has anyone in
history *ever* found this useful? If so, get them away from the keyboard
and put them into management or something where they can do less harm. Pet
peeve. Tabs not spaces people! Spaces, who does that? But I digress from my
digression. 4D says that this is to be "more compatible with JavaScript."
Why? I can't think of any reason. I've spent enough time with JavaScript to
really enjoy some of its features (closures!, some of the
messaging/delegation, the range of normal operators...man 4D needs ++,
etc.), but I've never heard *anyone* outside of 4D point to JavaScript as
an example of what a language should look like. Quite the opposite.
* So, an "object" can have anything in it. Or not. It's a completely
unfixed format. Sometimes that's what you want. Most of the time? I can
hardly think of *any* time I want a completely random structure or a
totally variable one. It's a minority case to the point of near irrelevance
to me. But this is the *only** mode 4D offers. So, the interpreter doesn't
complain about misspelled element names (there's no such concept), missing
elements (no concept), etc.. The compiler does not complain about any of
this for the same reasons. Oh, and Find in design does not find all
instances of strings embedded in form objects. (Found this the other day in
17.0.)
So what you're left with is a system where you can introduce a whole host
of problems that are super easy (inevitable) to create, quite hard to
prevent, and very hard to detect. Many of the things I would cal errors
happen silently. Things are just borked and you really have no idea
why...or even when...they went pear-shaped. All because of poor type
discipline. I've tried it all out and, honestly, it breaks down super fast.
More than a few element? Need to edit names? Need to update the format of
an object with a few types? Need to deal with a universe where not everyone
types perfectly all of the time? Yeah, no help for any of that.
What you need is the ability to *declare* a type and have the interpreter
and compiler test against this. This is 50+ year old tech, nothing new. I
say interpreter and compiler because there are cases for each. 4D does
neither. Period. This is an unforgivable failure. There is no reasonable
excuse. I've heard one excuse which is that it' all about the "power of
dynamic languages." That's nonsense. Dynamic language designers are very
much involved in figuring out better, faster ways to do type validation at
runtime. (It took me only a couple of minutes with Google to find lot of
contemporary research on this and related subjects.) And 4D is not some
totally type-fluid system, it complains about types all the time. It just
doesn't support complex/compound types. 4D, for whatever reason, doesn't
want to do it so they punted and gave us access to JSON Schema.
For those that have never dealt with schema systems before, they're exactly
the sort of computer design task that people with a particular turn of mind
enjoy. *These are exactly the people that should be let no where near the
design of schemas.* Too complicated, way fast. XML is from those kind of
people, JSON is not, for those that appreciate the difference in those
systems. (The uses for JSON and XML overlap more than they don't, the
complexity of XML is about 10,000 times greater than JSON...for weird cases
most people can't even imagine.) Even Tim Bray, co-author of the XML specs
says that the JSON Schema spec is incomprehensible. I've looked, and it is.
But on the upside, there's been enough work done with it that you can
accomplish a lot without diving into the specs. Thank goodness. JSON Schema
makes some easy things very easy, some hard things pretty easy, some easy
things surprisingly hard, and some hard things crazy hard. But here's a
quick rundown of what I managed without having to get deep into the weeds:
* Make sure that a *C_OBJECT* has only the elements specified as allowed.
This detects typo, capitalization problems and the like.
* Make sure that a *C_OBJECT* has all required elements.
* Make sure that an element in a *C_OBJECT* has a value from a set list of
values.
Yeah, I guess that's it. You can do a lot more, I didn't. Making sure that
a string isn't empty is surprisingly complex. (It requires multiple schemas
and "JSON Pointers", which is just a name for reusable chunks of defined
schema.) Anyway, the hard-to-check-for-empty-string problem lead me to
discover something quite nice: You can extend the JSON Schema JSON and it
works fine. I'll show a few fragments I've turned into examples for this
discussion. For this one, imagine I've got a format named "print_job" with
an element named "job_name". Check out this fragment:
"job_name": {
"type": "string",
"description": "Documentation here!",
"examples": "",
"default": "",
"required": true
}
What you're looking at is a property list that defines the attributes of
the print_job.job_name element. Those are all standard JSON Schema
attributes. But you can inject extra elements into the property list and
they don't hurt the validator. For example:
"job_name": {
"type": "string",
"examples": "",
"default": "",
"description": "Documentation here!",
"required": true,
"meta_type_name": "odt_element",
"meta_object_id": "Generate UUID()",
"element_name": "job_name",
"type_4d_code": 2,
"type_4d_name": "Text",
"examples": "",
"default": "",
"required": true,
"schema_format_rule": "",
"validation_pattern": "",
"not_empty": false,
"string_length_rule": "Must not be empty",
"lookup_type_name": ""
}
The extra stuff passes through JSON Validate without doing anything, but
it's then useful to custom code. For example, for "string_length_rule", I
can just pick that up and generically test an object for conformance. The
"lookup_type_name" is another semantic checker. As an example, one of the
lookup types I've implemented is "isAMethod". So, I can have an object
format with an element that includes an element of type "string" with a
lookup type of "isAMethod" (or table, or field, or type name,
etc....whatever you need.) To implement something of that sort, you have to
go past what any schema system can be expected to do. Fair enough, you're
doing a lookup on a live catalog. In this case method names.
Anyway, with all of that stuff in place, you then can build your own type
declaration and validation system. The validation is done using *JSON
Validate* and then a custom checker that reads any extended attributes
you've designed. The only other alternatives in 4D are:
* Write code perfectly. All the time. Forever. (Naming conventions help. I
suggest either initialLowerCamel like JavaScript or
snake_case_like_postgres.)
* Hope for the best.
* Use tables, the only thing like a struct supported by the environment.
Tables are fine, but they're just not always an ideal match for certain
program situations or deployment contexts.
Oh, a detail point for anyone that looked at the extended attributes
fragment. See "meta_type_name"? That's a big deal. Name it what you want.
It's the custom type identifier. So, you can look at a *C_OBJECT* and read
what type it is. So what? A type signature is super useful whenever you've
got some kind of generic dispatching going on. It lends itself to
message-oriented architectures very nicely. My little module is prefixed
with ODT (Object Data Type), so the signatures let you do this:
*ODT_New* (type name) : New object of that type
*ODT_Validate* (Object) : Validate that type
In my case, I'm only dispatching to internal methods to call something like
*PrintJob_New* () : Object of type "print_job". Like that. Personally, I
prefer to call PrintJob_New explicitly as it makes the 4D code easier to
track down and manage, but *ODT_New*("print_job") also works fine. And now
you see why I've got "lookup_type_name" = "isMethodName." At startup I can
load all of my type definitions once statically and validate them. Someone
renamed a method? The validator will spot the flaw *before the error is
invoked.*
Speaking of constructors, Rob L. some time back mentioned that in C++ it's
a community standard practice (language requirement?) to initialize objects
with all elements in place. Then you never have to worry if they exist,
they're just there. That's how I do things, it's great. The only exceptions
are cases where the existence/presence of an element is itself significant.
I don't do that, but other code does. JSON Schema, for example, complains
about empty "enum" (allowed values) arrays and can be confused by
minLength, maxLength elements that you don't need. So, 98%+ of the time, my
objects create everything when constructed.
*Bonus*: Call a constructor and get a complete object. Change the object
format with some new element, all of your existing code gets those elements
with zero rework. Sweet! Just like with *CREATE RECORD*.
And speaking more of constructors, it's kind of a pain to do
$print_job_object:=*PrintJob_New*// Get a blank object with all of the
defaults.
$print_job_object.job_name:="3 inch label"
...etc.
What I like to do is this:
print_job_object:=*PrintJob_New*(*New object*(\
"job_name";"3 inch label"))
What? The PrintJob_New call creates a default object, like *CREATE RECORD*
does with a table. Then the New object call in $1 passes in an object with
whatever elements (fields) you want to overwrite. I have all of my
constructors take an optional $1 for this very purpose. They, in turn, test
if there is a $1 and, if so, pass it through to a generic routine. Here's
the shell of a constructor:
// New_PrintJob
*C_BOOLEAN*($apply_overrides)
$apply_overrides:=*False*
*If* (*Count parameters*>=1)
$overrides_o:=$1
$apply_overrides:=*True*
*End if *
// The standard object constructor code goes here.
// ------------------------------------------------------------
// Apply overrides, if any
// ------------------------------------------------------------
*If* ($apply_overrides)
*Object_Union* ($new_o;$overrides_o) // Feed any elements passed in $1
into the new base object.
*End if*
The *Object_Union* code does nothing but add the items from the override to
the base object:
*C_OBJECT*($1;$base_object)
*C_OBJECT*($2;$override_object)
$base_object:=$1
$override_object:=$2
*ARRAY TEXT*($element_names_at;0)
*OB GET PROPERTY NAMES*($override_object;$element_names_at)
*C_LONGINT*($elements_count)
*C_LONGINT*($elements_index)
$elements_count:=*Size of array*($element_names_at)
*For* ($elements_index;1;$elements_count)
*C_TEXT*($element_name)
$element_name:=$element_names_at{$elements_index}
*OB SET*($base_object;$element_name;OB Get($override_object;$element_name))
*End for *
Works great!
The question remains, when to run the validation code? It feels like it
might be expensive to do it constantly. I'll figure that out in the next
week or two. But for now, I've got some obvious cases:
* Structures that are loaded once at startup get verified
* Structures that are definitively loadable are loaded and verified before
build time.
* Objects built in big loops get sampled. As in, the first row out of
10,000 gets validated. This was the initial motivation behind all of this,
I'm converting objects to Postgres UPSERT code and if the object format is
broken or modified, my code breaks. So now I'll be able to spot-verify the
system reliably.
In other V17 news, I *really* like the property binding implementation.
It's not 100% of what I want (shocker!) but, dang, it's super easy for the
level of functionality provided. Very, very nicely done. I'd like property
binding:
Form.print_button.enabled :: Form.selected_customer_object # *Null*
...and I'd like some kind of on binding refresh event to attach code to.
But even without those features, what's there now works really nicely.
Question: I've got an Object_Union sort of thing above....bad name. Has
anyone written a suite of tools to combine/difference/merge, etc. objects?
If so, would they be willing to share? Seems useful!
**********************************************************************
4D Internet Users Group (4D iNUG)
Archive: http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub: mailto:[email protected]
**********************************************************************
