Re: External data storage question

[John Baughman via 4D_Tech]

On Aug 4, 2017, at 1:25 PM, Timothy Penner via 4D_Tech <4d_tech@lists.4d.com> 
wrote:

> For this specific issue, I wonder if the behavior is mentioned in the 
> references to "custom mode" in the 'External Data Storage' documentation here;
> http://livedoc.4d.com/4Dv15.4/help/Title/en/page3202.html 
> 
This behavior is not mentioned nor is there any discussion of how the data is 
cached. I read the blurb in the design ref while I was trying to debug the 
problem and again just now.

I stil think this might be a bug or at least unintended behavior (isn’t that 
the definition of a bug?). The documentation for Set External Data Path 
explicitly states that the command does not need to be reissued unless the path 
changes... “4D automatically maintains the link between the field of the record 
and the file on disk.”

I went ahead and added the following comment to livedoc. FYI, When you save the 
comment it says that the comment will be sent to the doc team after validation. 

> 
> While the docs say that once this command is executed you do not need to 
> execute the command again unless you update the path, if you modify the field 
> you must in fact issue this command again. Otherwise accessing the field 
> during  the current session will continue to return the original version of 
> the field, the version when it was first loaded, even though the file on disk 
> has been updated.
> 
> It appears that the field is being cached when first loaded and the cache is 
> not being updated when the field is modified and saved. 
> 
> Set External Data Path appears to be refreshing the cache with the newly 
> saved version of the field on disk even when the path has not changed.

John

John Baughman
Kailua, Hawaii
(808) 262-0328
john...@hawaii.rr.com





**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

RE: External data storage question

[Timothy Penner via 4D_Tech]

Hi,

> Would the livedoc version get the information into the docs for all supported 
> versions of the product?

I believe public comments carry over to all versions that article is used by.
For example, this comment made on August 2015 is visible for v14:
- http://livedoc.4d.com/4Dv14.6/help/Command/en/page717.html
and v16 (even though v16 did not exist when the comment was made):
- http://livedoc.4d.com/4Dv16.1/help/Command/en/page717.html

> Otherwise, what do you have to do - file a doc bug? Is that a thing at 4D?

Using the "send to doc team" option on the livedoc is essentially submitting a 
doc bug, you will be notifying the doc team of your comment without making the 
comment public. This is the process I follow when I encounter doc bugs or wish 
that something be added to the docs. Of course you can also contact 4D 
Technical Support, then the TSE will investigate the issue and help in 
determining if it is actually bug, a doc bug, or something else.

For this specific issue, I wonder if the behavior is mentioned in the 
references to "custom mode" in the 'External Data Storage' documentation here;
http://livedoc.4d.com/4Dv15.4/help/Title/en/page3202.html

-Tim



**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

4D Equivalent from SQL

[Robert McKeever via 4D_Tech]

Working with V15.

The definition script for SQL for a few fields is ‘bigint (18)’.

If I specify a 64 bit integer, I get negative numbers upon import. No 
calculations are done with the numbers, so should I define them as alpha fields?

_
Bob McKeever  http://www.mswl.com 
McKeever's Software Wizardry
Port Coquitlam, B.C.
bobmckee...@mac.com




**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: External data storage question

[David Adams via 4D_Tech]

Would the livedoc version get the information into the docs for all
supported versions of the product? If so, it sounds like John could write
it up. Otherwise, what do you have to do - file a doc bug? Is that a thing
at 4D?
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

RE: External data storage question

[Timothy Penner via 4D_Tech]

Hi David,

> Not sure how that happens, but I know that there's a way if you log in to 
> something, somewhere, somehow...Or 4D could add something to the docs.

Do you mean this?

Tech Tip: The Comments feature for the Documentation Site .
http://kb.4d.com/assetid=77724

-Tim





**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: External data storage question

[David Adams via 4D_Tech]

John,

It's great that you found this and tracked down the solution within a day,
with help from 4D.

Since this sounds like a potentially painful gotcha, it would be great if
the docs were updated with some notes. Not sure how that happens, but I
know that there's a way if you log in to something, somewhere, somehow...Or
4D could add something to the docs.
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: UUID vs Longint primary key

[Nigel Greenlee via 4D_Tech]

+2

Nigel Greenlee


> On 4 Aug 2017, at 12:09, Marcus Straßmann via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> +1
> 
> 
> MacStrass - Marcus Straßmann
> Softwareentwicklung und Beratung
> Auf der Markscheide 35
> D-44807 Bochum
> 
> Mobil: +49 (173) 374 39 92
> eMail: macstr...@macstrass.de
> 
> Am 04.08.2017 um 10:39 schrieb Herr Alexander Heintz via 4D_Tech 
> <4d_tech@lists.4d.com>:
> 
>>> Someone can explain when is better use UUID and when Longint field in 
>>> primary key?
>> 
>> If you can even remotely think of a scenario where you might ever be 
>> challenged to merge databases, go with a UUID!!!
>> It will save you incredible headaches!
>> Switched to UUIDs long ago, never looked back, best decision ever!
>> 
>> Of course if such a scenario is unthinkable, you can use longints
>> 
>> **
>> 4D Internet Users Group (4D iNUG)
>> FAQ:  http://lists.4d.com/faqnug.html
>> Archive:  http://lists.4d.com/archives.html
>> Options: http://lists.4d.com/mailman/options/4d_tech
>> Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
>> **
> 
> **
> 4D Internet Users Group (4D iNUG)
> FAQ:  http://lists.4d.com/faqnug.html
> Archive:  http://lists.4d.com/archives.html
> Options: http://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
> **

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: External data storage question

[John Baughman via 4D_Tech]

Kirk,

Correct me if I am wrong but, there is no choice with external data and 
client server as the external data folder is on the server machine. The 
managing of the external file has to be done on the server and a trigger is the 
logical place to do it as the external file has to be updated every time the 
image is modified or deleted.

John


> On Aug 4, 2017, at 4:32 AM, Kirk Brooks via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> ​Also keep in mind triggers always run on the server. So doing something
> like file manipulation is problematic and you can't rely on IP vars.
> 
> ​I'd move the file manipulation back to the form context. No idea if it
> will matter in this specific instance​ but it will be easier to get the
> user involved if something isn't working as expected.

John Baughman
Kailua, Hawaii
(808) 262-0328
john...@hawaii.rr.com





**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: UUID vs Longint primary key

[Marcus Straßmann via 4D_Tech]

+1


MacStrass - Marcus Straßmann
Softwareentwicklung und Beratung
Auf der Markscheide 35
D-44807 Bochum

Mobil: +49 (173) 374 39 92
eMail: macstr...@macstrass.de

Am 04.08.2017 um 10:39 schrieb Herr Alexander Heintz via 4D_Tech 
<4d_tech@lists.4d.com>:

>> Someone can explain when is better use UUID and when Longint field in 
>> primary key?
> 
> If you can even remotely think of a scenario where you might ever be 
> challenged to merge databases, go with a UUID!!!
> It will save you incredible headaches!
> Switched to UUIDs long ago, never looked back, best decision ever!
> 
> Of course if such a scenario is unthinkable, you can use longints
> 
> **
> 4D Internet Users Group (4D iNUG)
> FAQ:  http://lists.4d.com/faqnug.html
> Archive:  http://lists.4d.com/archives.html
> Options: http://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
> **

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: External data storage question

[Kirk Brooks via 4D_Tech]

John,

On Fri, Aug 4, 2017 at 1:36 AM, John Baughman via 4D_Tech <
4d_tech@lists.4d.com> wrote:

> 3. Click the save button. The table trigger’s on saving new record event
> does the following...
> a. Saves the picture to my documents folder with WRITE PICTURE FILE
> b. Sets the external data path with SET EXTERNAL DATA PATH


​I use triggers ​a lot and I avoid actions this involved in them. I view
triggers as a way to check the internal consistency of the record and
enforce data rules. At most I'll do things like apply timestamps and
similar. Triggers need to be very quick. Remember that a trigger is a
function and returning anything other than zero causes the record not to
save or delete.

​Also keep in mind triggers always run on the server. So doing something
like file manipulation is problematic and you can't rely on IP vars.

​I'd move the file manipulation back to the form context. No idea if it
will matter in this specific instance​ but it will be easier to get the
user involved if something isn't working as expected.


-- 
Kirk Brooks
San Francisco, CA
===

*The only thing necessary for the triumph of evil is for good men to do
nothing.*

*- Edmund Burke*
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: UUID vs Longint primary key

[Kirk Brooks via 4D_Tech]

I'll add my vote to Jody's approach.

I prefer UUIDs for primary keys for all the reasons mentioned.
Having a longint index field is useful as well but for different reasons as
Jody explains. My most common use is to be able to quickly sort a table in
the order records were created. And it is superior for referencing records
- attempting to type a UUID is not something one wants to do.

On Fri, Aug 4, 2017 at 6:51 AM, Jody Bevan via 4D_Tech <4d_tech@lists.4d.com
> wrote:

> On another note, we assign a counter ourselves (not auto generated). There
> are times where we reset counters and being able to do that is great.
>

Resetting the counter can be done with ​SET DATABASE PARAMETER ( [Table] ;
Table sequence number ; value ). I haven't had any issues with bad sequence
numbers since about vers 6.

-- 
Kirk Brooks
San Francisco, CA
===

*The only thing necessary for the triumph of evil is for good men to do
nothing.*

*- Edmund Burke*
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: UUID vs Longint primary key

[Jody Bevan via 4D_Tech]

So what might be not so good with UUIDs? 

After watching how people work with our systems for 26 years (most of it 
without UUID capability) I noticed that they often referred to the record by 
the Unique LongInt we created for each record. For the first year we hid our 
unique ID - why would they want that. Then we exposed it to the admins for a 
year. Then through popular request we exposed it to everyone. Then we not only 
exposed it but changed the name of the display of the field to JonokeMed #. 
Referring to record # 150, or 275,983 is much easier than referring to a long 
UUID.

A UUID just is not a workable solution for that - at least in my mind. 
Therefore our solution today is that we do use UUIDs for all records for the 
reason of any possibility of merging records. We have had clients move across 
the Canada to join up with another client, and they need the records merged. 
Arrh a lot of work to write that the first time, and then continue to support 
it with each new table.

We use the UUID as the Primary Key record, but still have a Key field. This 
number is exposed to the end users to use as they wish. On another note, we 
assign a counter ourselves (not auto generated). There are times where we reset 
counters and being able to do that is great. 

Jody Bevan
ARGUS Productions Inc.
Developer

Argus Productions Inc. 




> On Aug 4, 2017, at 7:40 AM, Keith Culotta via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> The UUID has been the easiest to work with.  When a customer needs a 
> recognizable ID, an auto-incremented Longint can be added to the table.  
> UUIDs are perfect for avoiding conflicts when updating or exchanging records 
> between databases.  They can also be used as Process IDs or SVG IDs when one 
> of those is linked to a record.  All this can be accomplished with a Longint, 
> but it's more straightforward with UUIDs.  
> 
> Keith - CDI
> 
>> On Aug 4, 2017, at 3:27 AM, stardata.info via 4D_Tech <4d_tech@lists.4d.com> 
>> wrote:
>> 
>> Hi All,
>> 
>> Someone can explain when is better use UUID and when Longint field in 
>> primary key?
>> 
>> Thanks
>> Ferdinando

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: UUID vs Longint primary key

[npdennis via 4D_Tech]

> I can’t speak to any performance differences when using them as indexed key 
> fields for searches … they likely have similar performance, but that’s just a 
> guess.

I can speak to the performance, they are not the same. UUID is faster.

4D uses BTrees (binary trees or sometimes called red/black trees) to store 
index structures. A sequential number is the slowest possible way to create or 
add to a BTree. The treee needs to remain balanced, so there is a lot of 
rebalancing the needs to occur when numbers are added sequentially. UUID’s are 
much more random and require less tree balancing and are therefore faster for 
delete, create operations.

I started using UUID since version 11 and love them.

--
Neil Dennis
4D Developer since 1990


GreaText - Designing Software for the Way You Work
716 East 1850 N
North Ogden, UT 84414

mailto:npden...@greatext.com
http://www.greatext.com/


**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: External data storage question

[Keisuke Miyako via 4D_Tech]

what happens if you do

[myTable]myPicture:=[myTable]myPicture*0

before you load the new jpg ?

your trigger is running, so the picture is touched for the database, we can 
assume that.
but the picture on form is not updated, so it sounds like a ref count issue for 
the form.

besides, you describe

> 6. Load a different jpg into the picture field.


the exact way in which you do that (command, drag and drop, etc) might hold the 
key to understanding.

> 2017/08/04 17:36、John Baughman via 4D_Tech <4d_tech@lists.4D.com> のメール：
> 6. Load a different jpg into the picture field.



**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

RE: External data storage question

[Thomas Maul via 4D_Tech]

> OK. I got my hand slapped for posting a bug to the beta forum.

I apologize for my bad English or being unclear in my answer.
I only wanted to help you to speed up the process, as getting solutions for 
issues is - in my experience - faster using the 4D Partner support directly, as 
the team is there to help you to understand the behavior and help to report the 
bug.
I wanted to help, that's only.
In no way the intention was to say don't do that.
Of course you can post directly, if you prefer that.

I'm sorry
Thomas Maul



**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: UUID vs Longint primary key

[Ronald Rosell via 4D_Tech]

I can’t speak to any performance differences when using them as indexed key 
fields for searches … they likely have similar performance, but that’s just a 
guess.

But here’s one situation where UUID has an advantage.  We historically used the 
Sequence Number (a longint) as the primary key, until we discovered a 
shortcoming:  Merging data from multiple data files leads to duplicated primary 
keys.  With a sequence number, unless you somehow modify it (modifying the 
automatically-generated Sequence Number with some sort of instance prefix for 
the datafile) you will run into situations where different datafiles will have 
the same sequence numbers for various records.  That’s not a problem if those 
files will forever remain separate, but it requires a lot of re-sequencing of 
those tables and related tables if the files are ever merged or need to 
communicate with one another while referencing primary keys.

With UUID, it is highly unlikely (nearly impossible) that two data files will 
generate the same UUID for any records;  the chances are nearly 100% that all 
of the UUID’s will be unique, across every instance of your database.  So, if 
you later set up some kind of inter-database communication referencing records 
by primary key, or have to merge data files, you’re much less likely to run 
into issues.
__

Ron Rosell
President
StreamLMS


> On Aug 4, 2017, at 1:27 AM, stardata.info via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> Hi All,
> 
> Someone can explain when is better use UUID and when Longint field in primary 
> key?
> 
> Thanks
> Ferdinando
> **
> 4D Internet Users Group (4D iNUG)
> FAQ:  http://lists.4d.com/faqnug.html
> Archive:  http://lists.4d.com/archives.html
> Options: http://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
> **

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: External data storage question

[John Baughman via 4D_Tech]

OK. I got my hand slapped for posting a bug to the beta forum. Somewhat miffed, 
I created a test db in v16.1 so I can either prove it to be a beta bug, or be 
able to talk to 4D TS in the morning with a version other than the beta. Well I 
am having the same problem in v16.1. So, yes I erred by posting a bug to the 
beta forum.

Anyway…. Anyone have any idea what I might be doing wrong? I do not see 
anything in the docs that says I need to do anything when opening the record in 
an input form. Outside of loading the record, I should not have to do anything 
to get the picture from the file on disk. Is not that the whole purpose of SET 
EXTERNAL DATA? If I procedurally load the record and check the picture size, 4D 
automatically loads the picture and I in fact get it’s size.

I have posted the steps I am taking to make this work. Maybe someone can point 
out what I am doing wrong before I make a fool of myself a second time when I 
call 4D TS in the morning. ;-)

Thanks,

John



Here is what I am doing in my test DB. I have a table with the picture field 
set to store the picture outside the data file. I have an output and input form 
created automatically by 4D in the user environment. The input form has the 
Image field and a button to load a picture into the picture field.

1. Create a new record, using New Record menu item in the user environment.

2. Load a jpg into the picture field.

3. Click the save button. The table trigger’s on saving new record event does 
the following...

a. Saves the picture to my documents folder with WRITE PICTURE FILE
b. Sets the external data path with SET EXTERNAL DATA PATH

4. Create a second record as above with a different jpg.

5. Return to the first record input form and verify the the original picture is 
still being displayed properly.

6. Load a different jpg into the picture field. Ok, the new picture is 
displayed properly. So far so good.

7. Click the save button. The table trigger's saving existing record event does 
the following…

a. Saves the new picture to my documents folder with WRITE PICTURE 
FILE. Same file name and path so the existing file on disk is updated.
b. Using Get External Data Path, check if the data path is still set 
properly if not SET EXTERNAL DATA PATH again… it is in fact always properly set.

8. Open the second record just to make sure that every thing from the first is 
cleared and a different record is loaded. Second looks good displaying it’s 
picture as expected.

9. Open the first record… The original picture is displayed!!!

10. Restart the database. Now the new picture displays as it should in the 
first record.








> On Aug 3, 2017, at 1:21 PM, John Baughman via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> I recently changed storing a picture field outside the database from 
> automatic location to custom path name to file. Still working on getting it 
> ready for deployment.
> 
> I just noticed that if I open a form with the picture field displayed, then 
> change the picture to a different image, save it, then reopen the record, the 
> picture field continues to display the old picture. The picture file in the 
> storage folder reflects the new picture. I can actually watch it change as 
> the record is saved. The bad thing is if I save the record again the old 
> picture gets saved to the storage folder and the new picture is lost. If I 
> restart the database, without having reentered and saved again, the new 
> picture displays properly.
> 
> I know that I have to manage the creation and modification of the picture on 
> disk which I am doing, but I do not see anything that says I have to load the 
> picture from the storage folder. Am I missing something?
> 
> I also recently started working this db in v16 R4 beta so this may be a bug 
> in the beta? I do not really have an easy way to test this is an earlier 
> version of 4D.
> 
> John
> 
> 
> John Baughman
> Kailua, Hawaii
> (808) 262-0328
> john...@hawaii.rr.com
> 
> 
> 
> 
> 
> **
> 4D Internet Users Group (4D iNUG)
> FAQ:  http://lists.4d.com/faqnug.html
> Archive:  http://lists.4d.com/archives.html
> Options: http://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
> **

John Baughman
Kailua, Hawaii
(808) 262-0328
john...@hawaii.rr.com





**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

UUID vs Longint primary key

[stardata.info via 4D_Tech]

Hi All,

Someone can explain when is better use UUID and when Longint field in 
primary key?


Thanks
Ferdinando
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**