Re: 4D Server v18.1LTS - Startup Takes 10-15 Minutes

[Tom Benedict via 4D_Tech]

There’s a good article on the 4D Blog describing pros and cons of binary and 
Project Mode:

https://blog.4d.com/binary-database-vs-project-database/ 
 

At the bottom of the post in the “Con” section it mentions:

Development with distributed source code
Each developer codes alone on their copy of the code. Need for organization and 
rules to facilitate the sharing of work.

Access to the code from a client is read-only
Able to test and debug in Client/Server, but it is not [possible to] modify the 
deployed code on the server. You must reopen the database with 4D Developer, 
make the modification, and restart the server.


Tom Benedict

 

> On Apr 10, 2020, at 13:26, Kirk Brooks via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> Tim,
> This is great info. Thanks for testing it out.
> 
> On Fri, Apr 10, 2020 at 10:57 AM Tim Nevels via 4D_Tech <
> 4d_tech@lists.4d.com> wrote:
> 
>> I also tested editing the text of a method with TextEdit while the Project
>> was open on 4D Server. The method was not open with 4D Remote. In fact no
>> 4D Remote was even connected to 4D Server. I could change the method text
>> and save it. No errors from TextEdit. Opened the method file again and the
>> changes I made were there.
>> 
>> But when I started 4D Remote, connected to 4D Server, went to Design and
>> opened that method, my change was not there. It was still in the method
>> text file, but the changes didn’t show up in the 4D Method Editor.
>> 
> Take a look in the local resources folder for the client. This sounds like
> the client gets its own cache of files from the server and 'runs' those.
> 
> I would like for the to support 4D Team Developer using Project mode. There
>> are times when this is a good solution. Example, doing 4D Server testing of
>> parts of an application. Be nice to be able to test it running on 4D Server
>> and fix problems. And be nice to continue to have all the advantages of 4D
>> Team Developer in addition to version control advantages that Project mode
>> provides.
>> 
> I agree. I would like for support to be more familiar with the new stuff in
> general.
> 
> -- 
> Kirk Brooks
> San Francisco, CA
**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Server v18.1LTS - Startup Takes 10-15 Minutes

[Kirk Brooks via 4D_Tech]

Tim,
This is great info. Thanks for testing it out.

On Fri, Apr 10, 2020 at 10:57 AM Tim Nevels via 4D_Tech <
4d_tech@lists.4d.com> wrote:

> I also tested editing the text of a method with TextEdit while the Project
> was open on 4D Server. The method was not open with 4D Remote. In fact no
> 4D Remote was even connected to 4D Server. I could change the method text
> and save it. No errors from TextEdit. Opened the method file again and the
> changes I made were there.
>
> But when I started 4D Remote, connected to 4D Server, went to Design and
> opened that method, my change was not there. It was still in the method
> text file, but the changes didn’t show up in the 4D Method Editor.
>
Take a look in the local resources folder for the client. This sounds like
the client gets its own cache of files from the server and 'runs' those.

I would like for the to support 4D Team Developer using Project mode. There
> are times when this is a good solution. Example, doing 4D Server testing of
> parts of an application. Be nice to be able to test it running on 4D Server
> and fix problems. And be nice to continue to have all the advantages of 4D
> Team Developer in addition to version control advantages that Project mode
> provides.
>
I agree. I would like for support to be more familiar with the new stuff in
general.

-- 
Kirk Brooks
San Francisco, CA
==
**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Server v18.1LTS - Startup Takes 10-15 Minutes

[Tim Nevels via 4D_Tech]

On Apr 10, 2020, at 12:09 PM, Kirk Brooks wrote:

> This is what I found also. Thinking about it you can see how it makes sense
> - a Project is simply a collection of text files. They can be edited by 4D
> Method Editor or a text editor at the same time. On your local machine
> that's manageable, desirable, but on the server it would be a nightmare. I
> haven't tested this but I suspect you can edit files directly on the server
> with a text editor. And you could also simply add or remove files. So you
> could add or remove large amounts of functionality on the fly if you have
> access to the physical disk. That could be great or could be terrible. The
> important part is those changes are all taking place outside of 4D itself
> and 4D Server has no control over them.
> 
> In a development setting this could be pretty dynamic and useful. Fun, I
> dare say. But I would be hesitant to deploy such a setup to a
> production setting.

I tested opening a method on 4D Server v18.1 uncompiled in Project mode with 1 
4D Remote, then try to open that same method with another 4D Remote. You get 
the exact same dialog box you get in binary mode saying the method is already 
being edited by another user. So 4D Server is managing locked status of methods 
— and probably forms — in Project mode just like it does in binary mode. 

I also tested editing the text of a method with TextEdit while the Project was 
open on 4D Server. The method was not open with 4D Remote. In fact no 4D Remote 
was even connected to 4D Server. I could change the method text and save it. No 
errors from TextEdit. Opened the method file again and the changes I made were 
there.

But when I started 4D Remote, connected to 4D Server, went to Design and opened 
that method, my change was not there. It was still in the method text file, but 
the changes didn’t show up in the 4D Method Editor. 

Who knows what is planned for Project mode running interpreted and 4D Server. 
I’m guessing Project mode is still not fully implemented and complete. In fact 
4D has actually made comments to indicate that. Example is method comments. You 
lose those when you convert to Project mode, and method comments has NOT been 
listed as a deprecated feature. It’s just not implemented yet.

I would like for the to support 4D Team Developer using Project mode. There are 
times when this is a good solution. Example, doing 4D Server testing of parts 
of an application. Be nice to be able to test it running on 4D Server and fix 
problems. And be nice to continue to have all the advantages of 4D Team 
Developer in addition to version control advantages that Project mode provides. 

Remember when 4D v11 was first released. There was no 4D Server available for 
months. (And v11.0 was a real horror show if you thought it was a valid, stable 
development environment. It was not!) Project mode may be be in the same boat. 
They wanted to get it out on a schedule and just couldn’t fully implement 
everything by the release date. 

And if they intended to NEVER support uncompiled Project mode on 4D Server, 
they could have just put up an error message upon launch and tell you it’s not 
allowed. But they didn’t do that. Makes me think there is a plan to support it. 
And I couldn’t find any reference to this not being supported in v18 docs. 

If we had been able to go to 4D Summit I’m sure these limitations and 
deficiencies of v18 would have been addressed. But we had to all stay home 
because of corona. 

I read today that they have stopped all production of Corona beer in Mexico. 
Shut the plant down and sent everyone home. Why… the virus of course. Beer is 
“non-essential” according to the Mexican government. 

Tim

*
Tim Nevels
Innovative Solutions
785-749-3444
timnev...@mac.com
*

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Server v18.1LTS - Startup Takes 10-15 Minutes

[Kirk Brooks via 4D_Tech]

This is what I found also. Thinking about it you can see how it makes sense
- a Project is simply a collection of text files. They can be edited by 4D
Method Editor or a text editor at the same time. On your local machine
that's manageable, desirable, but on the server it would be a nightmare. I
haven't tested this but I suspect you can edit files directly on the server
with a text editor. And you could also simply add or remove files. So you
could add or remove large amounts of functionality on the fly if you have
access to the physical disk. That could be great or could be terrible. The
important part is those changes are all taking place outside of 4D itself
and 4D Server has no control over them.

In a development setting this could be pretty dynamic and useful. Fun, I
dare say. But I would be hesitant to deploy such a setup to a
production setting.

On Fri, Apr 10, 2020 at 8:50 AM Tim Nevels via 4D_Tech <4d_tech@lists.4d.com>
wrote:

> But it appears that right now an uncompiled Project database is basically
> in read only mode in terms of the Design environment.
>

-- 
Kirk Brooks
San Francisco, CA
==
**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Server v18.1LTS - Startup Takes 10-15 Minutes

[4dinug via 4D_Tech]

On 4/10/2020 8:50 AM, Tim Nevels via 4D_Tech wrote:


I can confirm that v18.1 4D Server can open an uncompiled Project database and 
users can connect to it and it appears to work. You can get into Design mode 
and open the structure, methods and forms but you can't make changes to them.

You also cannot create new methods or forms. The “+” button in the Explorer is 
disabled and if you use a keyboard shortcut to create a new method it give you 
an error “There is no method for this table/form. You can’t create one because 
the disk is locked.”

I’ve checked the v18 documentation and I can’t find an explicit statement about 
using an uncompiled project database with 4D Server. So I’m not sure if this is 
something that will be fully implemented in a later version. But it appears 
that right now an uncompiled Project database is basically in read only mode in 
terms of the Design environment.


Ah, Tim. Thank you for your gentle application of the clue stick. I now 
see that I had a fundamentally flawed understanding of the proper usage 
of project databases. I obviously missed the explanation of the workflow 
for deployment and that project databases were intended for off-line 
development with a build/compilation step prior to deployment in production.


Fair enough. Will re-jigger my expectations and look forward to the day 
when I can develop in real-time against a server-deployed application in 
project format.


Thanks again.

-M

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Server v18.1LTS - Startup Takes 10-15 Minutes

[Tim Nevels via 4D_Tech]

On Apr 10, 2020, at 9:38 AM, M wrote:

> Well, your question promoted me to re-convert the database on the server 
> host. Previously I had converted to v18.1 on my desktop development 
> machine and also converted from binary to project database. So, I have 
> now opened a fresh copy of the v17R6 database with v18.1 LTS on the 
> server itself. It opened pretty much instantly after the conversion 
> warning dialog. I then closed and re-opened the database several times 
> in succession, each time with no delay.
> 
> So, my conclusion is that converting the database to a v18 project 
> database is the performance killer. Which is mildly annoying, because 
> the project database format was a most significant reason why I wanted 
> to upgrade, since it is a really appealing feature. But apparently it 
> still has some major flaws.
> 
> So, now knowing that the project database aspect is the problem, is it 
> worth reporting a bug? Or would it be best to just wait for a few more 
> iterations of v18 for this feature to mature?

I can confirm that v18.1 4D Server can open an uncompiled Project database and 
users can connect to it and it appears to work. You can get into Design mode 
and open the structure, methods and forms but you can't make changes to them. 

You also cannot create new methods or forms. The “+” button in the Explorer is 
disabled and if you use a keyboard shortcut to create a new method it give you 
an error “There is no method for this table/form. You can’t create one because 
the disk is locked.”

I’ve checked the v18 documentation and I can’t find an explicit statement about 
using an uncompiled project database with 4D Server. So I’m not sure if this is 
something that will be fully implemented in a later version. But it appears 
that right now an uncompiled Project database is basically in read only mode in 
terms of the Design environment.  

Tim

*
Tim Nevels
Innovative Solutions
785-749-3444
timnev...@mac.com
*

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Server v18.1LTS - Startup Takes 10-15 Minutes

[Tom Benedict via 4D_Tech]

Whoops. I missed the part about it running in interpreted mode. So please 
disregard my comments about .4DZ. 

Tom Benedict

> On Apr 10, 2020, at 08:02, Tom Benedict via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> On Apr 10, 2020, at 07:28, 4dinug via 4D_Tech <4d_tech@lists.4d.com> wrote:
> 
>> 
>> So, my conclusion is that converting the database to a v18 project database 
>> is the performance killer. Which is mildly annoying, because the project 
>> database >format was a most significant reason why I wanted to upgrade, 
>> since it is a really appealing feature. But apparently it still has some 
>> major flaws.
>> 
> 
> One thing that Is different between a built Project database and a built 
> Binary database is that the Project Build generates a .4DZ package, rather 
> than a .4DC. I’m wondering if that zipped package is the source of the 
> problem.
> 
> BTW, is performance good once the 10-15 minutes of startup finishes?
> 
>> So, now knowing that the project database aspect is the problem, is it worth 
>> reporting a bug? Or would it be best to just wait for a few more iterations 
>> of v18 for >this feature to mature?
> 
> Please report it. This would be worthwhile to sort out sooner rather than 
> later.
> 
> Tom Benedict  

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Server v18.1LTS - Startup Takes 10-15 Minutes

[John DeSoi via 4D_Tech]

Sorry if I missed it, but I still did not see in your messages that you 
reproduced the slow startup with conversion on a separate non-VM machine. Also 
wondering if there is any difference between 4D single user and 4D Server. I 
have not used 4D Server to convert structures to project mode.

I regularly convert a 17.4 binary structure to 18 project mode for version 
control. It has over 5000 methods and many hundreds of forms. It only takes a 
few seconds to export to project mode (using 4D single user).

A work-around might be to covert to project mode before moving the structure to 
the VM server.

John DeSoi, Ph.D.


> On Apr 10, 2020, at 9:33 AM, 4dinug via 4D_Tech <4d_tech@lists.4d.com> wrote:
> 
> FWIW, this a regular server application running in interpreted mode. Nothing 
> terribly unusual about it. Methods and forms number probably in the 1,000 - 
> 1,500 range, with a 1.5 GB data file. It runs absolutely fine in single-user 
> on my desktop PC.
> 
> But, in reply to Keisuke's earlier response, I did as you suggested. And it 
> is is indeed the project database format conversion that is the root of the 
> problem. The same database converted to v18.1 and left in binary format has 
> no performance issues at all.

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Server v18.1LTS - Startup Takes 10-15 Minutes

[Tom Benedict via 4D_Tech]

On Apr 10, 2020, at 07:28, 4dinug via 4D_Tech <4d_tech@lists.4d.com> wrote:

>
>So, my conclusion is that converting the database to a v18 project database is 
>the performance killer. Which is mildly annoying, because the project database 
>>format was a most significant reason why I wanted to upgrade, since it is a 
>really appealing feature. But apparently it still has some major flaws.
>

One thing that Is different between a built Project database and a built Binary 
database is that the Project Build generates a .4DZ package, rather than a 
.4DC. I’m wondering if that zipped package is the source of the problem.

BTW, is performance good once the 10-15 minutes of startup finishes?

>So, now knowing that the project database aspect is the problem, is it worth 
>reporting a bug? Or would it be best to just wait for a few more iterations of 
>v18 for >this feature to mature?

Please report it. This would be worthwhile to sort out sooner rather than later.

Tom Benedict
**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Server v18.1LTS - Startup Takes 10-15 Minutes

[4dinug via 4D_Tech]

On 4/10/2020 6:18 AM, John DeSoi via 4D_Tech wrote:

Have you tried it using binary mode to see if there is any difference? Is there 
anything unusual about your project, e.g. a really large number of methods or 
forms? You get the same results on different computers? Is it a regular server 
or a merged server application? I assume your application is compiled; if so, 
have you tried to open it with 4D server interpreted?


Thank you for your reply, John.

FWIW, this a regular server application running in interpreted mode. 
Nothing terribly unusual about it. Methods and forms number probably in 
the 1,000 - 1,500 range, with a 1.5 GB data file. It runs absolutely 
fine in single-user on my desktop PC.


But, in reply to Keisuke's earlier response, I did as you suggested. And 
it is is indeed the project database format conversion that is the root 
of the problem. The same database converted to v18.1 and left in binary 
format has no performance issues at all.


Many thanks again.

-M
**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Server v18.1LTS - Startup Takes 10-15 Minutes

[4dinug via 4D_Tech]

On 4/9/2020 10:24 PM, Keisuke Miyako via 4D_Tech wrote:

Thank you for your reply.


I have an application I'm trying to convert from 4D v17R6 to v18.1LTS. It is 
hosted on Windows Server 2016. It was a well-behaved application when running 
v17R6 with no notable issues. However, now that it has been upgraded and 
running on v18.1 LTS, it takes 10-15 minutes for the application to start-up in 
4D Server. Specifically, from the moment that the database is selected from the 
Open... menu item until the Server Administration Window appears is between 10 
and 15 minutes.


is it possible that the application, for whatever reason, needs to copy all 
your database files to a virtual store?


Hmm. I don't know what a virtual store is, so probably not.


---

also, windows server by default has a conservative power plan, which prevents 
4D Server from drawing on its full potential.

https://support.microsoft.com/en-us/help/2207548/slow-performance-on-windows-server-when-using-the-balanced-power-plan
https://docs.microsoft.com/en-us/windows-server/administration/performance-tuning/hardware/power/power-performance-tuning

of course, that does not explain why 17 R6 is doing just fine.


The server host is a VM running in a large data center. Interrogating 
the Power settings shows nothing that appear would be throttling power, 
i.e., all options set to "never sleep", with Power Place set to "High 
performance".



---


The same application opens pretty much instantly when running in single-user 
mode on my development machine (Windows 10 Pro). So the only difference is host 
application (4D Server) and OS.


what about the hardware?
more specifically, do you know if the storage is HD or SSD?


Running on VMWare, as above. I don't know exactly how the VM has been 
provisioned, but we're hosted in a university data center with some 
resources. Disk Management reveals nothing about the hardware 
configuration except to say "VMWare Virtual Disk" for each volume.



I would rule out the OS being an issue since the same application ran fine on 
the same host machine and opened with no delay just prior to the upgrade from 
v17R6 to v18.1LTS.


have you tried to launch 4D Server on your dev. machine, Windows 10 Pro,
to see if the correlation is really linked to the application type?

and just to rule out that the issue is not application specific,
you might want to compare the time with a fresh 4DB.


Well, your question promoted me to re-convert the database on the server 
host. Previously I had converted to v18.1 on my desktop development 
machine and also converted from binary to project database. So, I have 
now opened a fresh copy of the v17R6 database with v18.1 LTS on the 
server itself. It opened pretty much instantly after the conversion 
warning dialog. I then closed and re-opened the database several times 
in succession, each time with no delay.


So, my conclusion is that converting the database to a v18 project 
database is the performance killer. Which is mildly annoying, because 
the project database format was a most significant reason why I wanted 
to upgrade, since it is a really appealing feature. But apparently it 
still has some major flaws.


So, now knowing that the project database aspect is the problem, is it 
worth reporting a bug? Or would it be best to just wait for a few more 
iterations of v18 for this feature to mature?


Many thanks again for your assistance.
**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Server v18.1LTS - Startup Takes 10-15 Minutes

[John DeSoi via 4D_Tech]

Have you tried it using binary mode to see if there is any difference? Is there 
anything unusual about your project, e.g. a really large number of methods or 
forms? You get the same results on different computers? Is it a regular server 
or a merged server application? I assume your application is compiled; if so, 
have you tried to open it with 4D server interpreted?

John DeSoi, Ph.D.


> On Apr 9, 2020, at 11:42 PM, 4dinug via 4D_Tech <4d_tech@lists.4d.com> wrote:
> 
> FWIW, this is a project (i.e., not binary) database. I'm not sure whether 
> that is pertinent to this issue since, as mentioned above, it runs with no 
> delay on 4D single-user.

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Server v18.1LTS - Startup Takes 10-15 Minutes

[Keisuke Miyako via 4D_Tech]

first off, I don't have any answers to your questions.

but if I may throw some ideas:

> I have an application I'm trying to convert from 4D v17R6 to v18.1LTS. It is 
> hosted on Windows Server 2016. It was a well-behaved application when running 
> v17R6 with no notable issues. However, now that it has been upgraded and 
> running on v18.1 LTS, it takes 10-15 minutes for the application to start-up 
> in 4D Server. Specifically, from the moment that the database is selected 
> from the Open... menu item until the Server Administration Window appears is 
> between 10 and 15 minutes.

is it possible that the application, for whatever reason, needs to copy all 
your database files to a virtual store?

---

also, windows server by default has a conservative power plan, which prevents 
4D Server from drawing on its full potential.

https://support.microsoft.com/en-us/help/2207548/slow-performance-on-windows-server-when-using-the-balanced-power-plan
https://docs.microsoft.com/en-us/windows-server/administration/performance-tuning/hardware/power/power-performance-tuning

of course, that does not explain why 17 R6 is doing just fine.

---

> The same application opens pretty much instantly when running in single-user 
> mode on my development machine (Windows 10 Pro). So the only difference is 
> host application (4D Server) and OS.

what about the hardware?
more specifically, do you know if the storage is HD or SSD?

> I would rule out the OS being an issue since the same application ran fine on 
> the same host machine and opened with no delay just prior to the upgrade from 
> v17R6 to v18.1LTS.

have you tried to launch 4D Server on your dev. machine, Windows 10 Pro,
to see if the correlation is really linked to the application type?

and just to rule out that the issue is not application specific,
you might want to compare the time with a fresh 4DB.
**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**