Re: 4D Structure Export...

[Tom Benedict via 4D_Tech]

Thanks for the response Lutz. 

This is very helpful and increases my confidence that we will be able to 
provide a solution.

Tom Benedict

> On Jan 11, 2019, at 01:14, Epperlein, Lutz (agendo) via 4D_Tech 
> <4d_tech@lists.4d.com> wrote:
> 
> Hi Tom,
> 
> The use of the VC_Framework component is completely independent of the 
> version control system. I don't know much about the Team Foundation Server, 
> but in AFAIK it has means to manage the source code. 
> 
> But again, the approach works with 4D and the import functionality of the 
> VC_Framework component, but has two limitations:
> 
> - Performing an additional step after checking out the code from the source 
> code administration. The steps are: 
> 1. you need to open a method that you are sure is not included in the current 
> changeset. This is because you cannot import methods that are already open in 
> 4D.
> 2. Then you need to select the import macro. This will trigger the import 
> method within the component. The rest runs automatically. It is always a good 
> idea to perform a syntax check afterwards.
> 
> - The other thing is that methods cannot be deleted during the import. The 
> automatic export by the component deletes previously exported methods, but 
> the re-import is not able to do this. So you have to do this manually and you 
> have to communicate with your team about deleted methods.
> 
> I would recommend from time to time to use a new central copy of the 4D 
> structure file and to do a re-import. There are so many things besides forms 
> that are not handled by the component, e.g. changes in the database structure.
> We copy the current structure file to certain archive folders on our file 
> server during the build process so that we can use that copy.
> 
> But besides all the problems, we have been using the component very 
> successfully for years. We are used to doing the additional steps now, that's 
> not a real problem, you have to keep it in mind.
> 
> Regards
> Lutz Epperlein
> 
> 
>> -Original Message-
>> From: 4D_Tech [mailto:4d_tech-boun...@lists.4d.com] On Behalf Of Tom 
>> Benedict via
>> 4D_Tech
>> 
>> Hi Lutz,
>> 
>> Sorry to bug you again, but we’re moving forward on a project where we are 
>> consulting
>> with a client who is setting up an agile development team to use 4D. They 
>> will be
>> managing this team with Microsoft Team Foundation Server. I’m thinking that 
>> TFS
>> should be able to serve as a repository for the 4D code (methods and forms) 
>> using the
>> VC_Framework component that you have updated.
>> 
>>> We have been doing something similar for some time with the VC_Framework
>> component. The original version of the component doesn't support the 
>> re-import of
>> code so we extended it with an import function for methods. The procedure is 
>> a bit
>> inconvenient, you have to do an additional step (the import) manually after 
>> checking
>> out the code from the versioning system (Mercurial in our case) but it works 
>> flawlessly.
>> We use this workflow and the vc_framework component since 4d V13. Look here 
>> if you
>> are interested: https://github.com/elutz/vc-framework-v17
>> 
>> The part that isn’t completely clear to me is the Build/Release Management 
>> part. I’m
>> trying to figure now is how to link code in the repository to TFS work items 
>> so that the
>> release manifest that TFS produces includes all the methods and forms which 
>> are code
>> complete and validated by QA. In your process, how do you know which methods 
>> to
>> import?
>> 
>> Tom Benedict
>> *
>> *
>> 4D Internet Users Group (4D iNUG)
>> Archive:  http://lists.4d.com/archives.html
>> Options: https://lists.4d.com/mailman/options/4d_tech
>> Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
>> *
>> *
> **
> 4D Internet Users Group (4D iNUG)
> Archive:  http://lists.4d.com/archives.html
> Options: https://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
> **

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

RE: 4D Structure Export...

[Epperlein, Lutz (agendo) via 4D_Tech]

Hi Tom,

The use of the VC_Framework component is completely independent of the version 
control system. I don't know much about the Team Foundation Server, but in 
AFAIK it has means to manage the source code. 

But again, the approach works with 4D and the import functionality of the 
VC_Framework component, but has two limitations:

- Performing an additional step after checking out the code from the source 
code administration. The steps are: 
1. you need to open a method that you are sure is not included in the current 
changeset. This is because you cannot import methods that are already open in 
4D.
2. Then you need to select the import macro. This will trigger the import 
method within the component. The rest runs automatically. It is always a good 
idea to perform a syntax check afterwards.

- The other thing is that methods cannot be deleted during the import. The 
automatic export by the component deletes previously exported methods, but the 
re-import is not able to do this. So you have to do this manually and you have 
to communicate with your team about deleted methods.

I would recommend from time to time to use a new central copy of the 4D 
structure file and to do a re-import. There are so many things besides forms 
that are not handled by the component, e.g. changes in the database structure.
We copy the current structure file to certain archive folders on our file 
server during the build process so that we can use that copy.

But besides all the problems, we have been using the component very 
successfully for years. We are used to doing the additional steps now, that's 
not a real problem, you have to keep it in mind.

Regards
Lutz Epperlein


> -Original Message-
> From: 4D_Tech [mailto:4d_tech-boun...@lists.4d.com] On Behalf Of Tom Benedict 
> via
> 4D_Tech
> 
> Hi Lutz,
> 
> Sorry to bug you again, but we’re moving forward on a project where we are 
> consulting
> with a client who is setting up an agile development team to use 4D. They 
> will be
> managing this team with Microsoft Team Foundation Server. I’m thinking that 
> TFS
> should be able to serve as a repository for the 4D code (methods and forms) 
> using the
> VC_Framework component that you have updated.
> 
> > We have been doing something similar for some time with the VC_Framework
> component. The original version of the component doesn't support the 
> re-import of
> code so we extended it with an import function for methods. The procedure is 
> a bit
> inconvenient, you have to do an additional step (the import) manually after 
> checking
> out the code from the versioning system (Mercurial in our case) but it works 
> flawlessly.
> We use this workflow and the vc_framework component since 4d V13. Look here 
> if you
> are interested: https://github.com/elutz/vc-framework-v17
> 
> The part that isn’t completely clear to me is the Build/Release Management 
> part. I’m
> trying to figure now is how to link code in the repository to TFS work items 
> so that the
> release manifest that TFS produces includes all the methods and forms which 
> are code
> complete and validated by QA. In your process, how do you know which methods 
> to
> import?
> 
> Tom Benedict
> *
> *
> 4D Internet Users Group (4D iNUG)
> Archive:  http://lists.4d.com/archives.html
> Options: https://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
> *
> *
**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Structure Export...

[Tom Benedict via 4D_Tech]

Hi Lutz,

Sorry to bug you again, but we’re moving forward on a project where we are 
consulting with a client who is setting up an agile development team to use 4D. 
They will be managing this team with Microsoft Team Foundation Server. I’m 
thinking that TFS should be able to serve as a repository for the 4D code 
(methods and forms) using the VC_Framework component that you have updated. 

> We have been doing something similar for some time with the VC_Framework 
> component. The original version of the component doesn't support the 
> re-import of code so we extended it with an import function for methods. The 
> procedure is a bit inconvenient, you have to do an additional step (the 
> import) manually after checking out the code from the versioning system 
> (Mercurial in our case) but it works flawlessly. We use this workflow and the 
> vc_framework component since 4d V13. Look here if you are interested: 
> https://github.com/elutz/vc-framework-v17 
> 
The part that isn’t completely clear to me is the Build/Release Management 
part. I’m trying to figure now is how to link code in the repository to TFS 
work items so that the release manifest that TFS produces includes all the 
methods and forms which are code complete and validated by QA. In your process, 
how do you know which methods to import?

Tom Benedict
**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Structure Export...

[Tom Benedict via 4D_Tech]

Hi Lutz,

I also need support for Release Management, so I need a way to track version 
numbers. 

In the past I have initially written a major version number to a List object in 
the structure file, then incremented a minor version number every time I run 
Compile & Build, creating a file copy of the .4DB and .4DC file with the 
version number appended to the filename. Then I can deploy those structures to 
QA, UAT, Production environments and know positively what code is where. That 
allowed Method_History to track all edits by developer, version number and date 
time while it also created an archive of all versions of the structure. 

Does the VC software (Git, SVN, Mercurial) support that? I’ve just started to 
work with Mercurial and the terminology and process is a little overwhelming, 
so I can’t tell how much I will need to change my workflow to move to an 
external VC tool.

Thanks,

Tom

> On Dec 10, 2018, at 01:26, Epperlein, Lutz (agendo) via 4D_Tech 
> <4d_tech@lists.4d.com> wrote:
> 
>> Maybe we need to consider moving team development from the centralized 4D 
>> Team
>> Developer model to a distributed model  using individual copies of 4D. That 
>> would
>> match the classic development model and practice which non-4D developers have
>> enjoyed for decades Is anyone doing
>> this yet? I would try it myself, but I’m not a Partner at this time.
> 
> We have been doing something similar for some time with the VC_Framework 
> component. The original version of the component doesn't support the 
> re-import of code so we extended it with an import function for methods. The 
> procedure is a bit inconvenient, you have to do an additional step (the 
> import) manually after checking out the code from the versioning system 
> (Mercurial in our case) but it works flawlessly. We use this workflow and the 
> vc_framework component since 4d V13. Look here if you are interested: 
> https://github.com/elutz/vc-framework-v17
> 
> A remark: we don't use it for forms. An export of forms is possible using the 
> VC_framework. But we don’t re-import  forms, since this is not possible 
> before 17R3. (And we don't do much GUI development in 4D, it is web 
> development mostly). It might be possible with 17R3 but we have to extend the 
> component again. If we start with 17R3 (or later V18) I think we will adapt 
> the component using the new features.
> 
> Regarding this new features in 4D V17R3 I think they are based mainly on the 
> code of this component, the exported results look very similar. But a more in 
> depth test with the 17R3 we didn't do yet. 
> 
> Regards 
> Lutz
> 
> 
> --  
> Lutz Epperlein  
> --
> Agendo Gesellschaft für politische Planung mbH
> Köpenicker Str. 9
> 10997 Berlin
> http://www.agendo.de/
> --
> 
> 
> **
> 4D Internet Users Group (4D iNUG)
> Archive:  http://lists.4d.com/archives.html
> Options: https://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
> **

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Structure Export...

[Tom Benedict via 4D_Tech]

Thanks for posting the link Lutz. I will dig into it deeper. It looks 
encouraging.

Tom Benedict  

> On Dec 10, 2018, at 01:26, Epperlein, Lutz (agendo) via 4D_Tech 
> <4d_tech@lists.4d.com> wrote:
> 
>> Maybe we need to consider moving team development from the centralized 4D 
>> Team
>> Developer model to a distributed model  using individual copies of 4D. That 
>> would
>> match the classic development model and practice which non-4D developers have
>> enjoyed for decades Is anyone doing
>> this yet? I would try it myself, but I’m not a Partner at this time.
> 
> We have been doing something similar for some time with the VC_Framework 
> component. The original version of the component doesn't support the 
> re-import of code so we extended it with an import function for methods. The 
> procedure is a bit inconvenient, you have to do an additional step (the 
> import) manually after checking out the code from the versioning system 
> (Mercurial in our case) but it works flawlessly. We use this workflow and the 
> vc_framework component since 4d V13. Look here if you are interested: 
> https://github.com/elutz/vc-framework-v17
> 
> A remark: we don't use it for forms. An export of forms is possible using the 
> VC_framework. But we don’t re-import  forms, since this is not possible 
> before 17R3. (And we don't do much GUI development in 4D, it is web 
> development mostly). It might be possible with 17R3 but we have to extend the 
> component again. If we start with 17R3 (or later V18) I think we will adapt 
> the component using the new features.
> 
> Regarding this new features in 4D V17R3 I think they are based mainly on the 
> code of this component, the exported results look very similar. But a more in 
> depth test with the 17R3 we didn't do yet. 
> 
> Regards 
> Lutz
> 

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

RE: 4D Structure Export...

[Epperlein, Lutz (agendo) via 4D_Tech]

> Maybe we need to consider moving team development from the centralized 4D Team
> Developer model to a distributed model  using individual copies of 4D. That 
> would
> match the classic development model and practice which non-4D developers have
> enjoyed for decades Is anyone doing
> this yet? I would try it myself, but I’m not a Partner at this time.

We have been doing something similar for some time with the VC_Framework 
component. The original version of the component doesn't support the re-import 
of code so we extended it with an import function for methods. The procedure is 
a bit inconvenient, you have to do an additional step (the import) manually 
after checking out the code from the versioning system (Mercurial in our case) 
but it works flawlessly. We use this workflow and the vc_framework component 
since 4d V13. Look here if you are interested: 
https://github.com/elutz/vc-framework-v17

A remark: we don't use it for forms. An export of forms is possible using the 
VC_framework. But we don’t re-import  forms, since this is not possible before 
17R3. (And we don't do much GUI development in 4D, it is web development 
mostly). It might be possible with 17R3 but we have to extend the component 
again. If we start with 17R3 (or later V18) I think we will adapt the component 
using the new features.

Regarding this new features in 4D V17R3 I think they are based mainly on the 
code of this component, the exported results look very similar. But a more in 
depth test with the 17R3 we didn't do yet. 

Regards 
Lutz


--  
Lutz Epperlein  
--
Agendo Gesellschaft für politische Planung mbH
Köpenicker Str. 9
10997 Berlin
http://www.agendo.de/
--


**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Structure Export...

[Arnaud de Montard via 4D_Tech]

> Le 7 déc. 2018 à 23:11, Tom Benedict via 4D_Tech <4d_tech@lists.4d.com> a 
> écrit :
> 
>> [...]
> I see how one would get the modified date for the method using METHOD GET 
> MODIFICATION DATE, but I don’t see a METHOD GET MODIFIED BY. 

Thinking that Current user works already on server side… If we had events 
triggered (add, modify, delete) by structure objects (methods, forms) on server 
too, life would be simple. 

-- 
Arnaud de Montard 




**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Structure Export...

[Tom Benedict via 4D_Tech]

Thanks for your comments Kirk.

> On Dec 7, 2018, at 11:07, Kirk Brooks via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> I'm not familiar with VC_Framework but the big improvement in R3 is the
> addition of Form convert to dynamic. This allows you to export all the
> forms  - not just the object and form methods. The objects created are
> fully functional dynamic form objects. This is the last piece we needed to
> be able to fully build a database from exported files.
> 
Have you (or anyone) built a database from exported files? I’m thinking this 
would a great way to support branching and team development.

> If it doesn't natively it's not difficult to set the created/modified dates
> of the new files. I do this with method exports because the exported files
> are dated with the current date/time.
> 
I see how one would get the modified date for the method using METHOD GET 
MODIFICATION DATE, but I don’t see a METHOD GET MODIFIED BY. Running on server 
in a ‘batch’ export, which, if you are using 4D Team Developer, is the only way 
that Export structure file works, I don’t think there’s a way to know what user 
modified the method/form/etc. 

Maybe we need to consider moving team development from the centralized 4D Team 
Developer model to a distributed model  using individual copies of 4D. That 
would match the classic development model and practice which non-4D developers 
have enjoyed for decades. If 4D v17R3 can actually build a structure from text 
files generated by Export structure file, I’m thinking it would be possible. Is 
anyone doing this yet? I would try it myself, but I’m not a Partner at this 
time.

Tom Benedict



**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Structure Export...

[John DeSoi via 4D_Tech]

> 
> On Dec 7, 2018, at 1:07 PM, Kirk Brooks via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> I'm not familiar with VC_Framework but the big improvement in R3 is the
> addition of Form convert to dynamic. This allows you to export all the
> forms  - not just the object and form methods. The objects created are
> fully functional dynamic form objects. This is the last piece we needed to
> be able to fully build a database from exported files.


17 R3 is not quite there for exporting forms in a way that can easily be moved 
to another structure. Form convert to dynamic does not include the form or 
object method paths. I'm hoping that will change, see the recent discussion on 
the forum.

If you export the forms from the structure (not using Form convert to dynamic), 
the form/object method references are there. But I don't think you can use that 
to reload the form in another database because the method property can only be 
a project method. So to use the form in another database you would have to 
translate the form/object methods into a project methods somehow.


> It's easy to diff
> folders with methods or form definitions. I've done this with folders
> containing thousands of methods in a couple of minutes.

The other way to do it is to just export all the files to the same version 
controlled directory. The commit for each release. The version control tool 
will show exactly which files are changed since the last release and tracks all 
changes over time. For 4D, you'll just need to add something to remove methods 
that have been deleted or renamed. Git automatically figures out when a file 
has been renamed instead of deleted.


John DeSoi, Ph.D.
**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Structure Export...

[Kirk Brooks via 4D_Tech]

Tom,

On Fri, Dec 7, 2018 at 10:14 AM Tom Benedict via 4D_Tech <
4d_tech@lists.4d.com> wrote:

>  if this feature does more than the export we’ve been able to do since
> v12/14 via various versions of VC_Framework. Is this simply a higher level
> of integration with the app or does it actually offer more functionality?
>
I'm not familiar with VC_Framework but the big improvement in R3 is the
addition of Form convert to dynamic. This allows you to export all the
forms  - not just the object and form methods. The objects created are
fully functional dynamic form objects. This is the last piece we needed to
be able to fully build a database from exported files.


> I’m trying to understand how it supports team development on 4D Server.
> Among other things, I want to track all changes to all objects so I know
> who changed what and when they made the change. Reviewing the Export
> structure file command documentation
> http://doc.4d.com/4Dv17R3/4D/17-R3/Export-structure-file.301-3965587.en.html
> <
> http://doc.4d.com/4Dv17R3/4D/17-R3/Export-structure-file.301-3965587.en.html>
> and the table which describes the exported files, it’s not clear whether
> the files are tagged with 'mod date' and 'mod by' values.
>
If it doesn't natively it's not difficult to set the created/modified dates
of the new files. I do this with method exports because the exported files
are dated with the current date/time.

For me the advantage here is being able to export all the database elements
into text files. Pretty much all the version control solutions around are
based on managing text files so this opens up a lot of options. when it
comes to identifying changes it's possible to use any number of diffing
tools. Like BBEdit or DeltaWalker, to name a couple. It's easy to diff
folders with methods or form definitions. I've done this with folders
containing thousands of methods in a couple of minutes.

I think dynamic forms are truly a game changer in the 4D world. Now that we
can export existing forms into the format required there's backward
compatibility.


-- 
Kirk Brooks
San Francisco, CA
===

*We go vote - they go home*
**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**