Re: 4D Debug Log Reader - Plugin Commands

[Justin Carr via 4D_Tech]

On 1 Dec 2017, at 11:28 am, Allan Udy via 4D_Tech <4d_tech@lists.4d.com> wrote:
> 
>> Justin previously determined that this is the case, and in his Log Parsing 
>> database that we're using, the plugin command ID's are assumed to start with 
>>  3;YYY.
> 
> Through a bit of trial and error I can now say with reasonable certainty that 
> in v16 at least, the plugin Commands now appear to start with 1, rather than 
> 3 as was necessary in previous versions of your Debug Log Reader.
> 
> A few of the Plugin commands showing up in my Debug Log files take 
> parameters, and so it's relatively straight forward to match them up with the 
> actual commands used in the source.  It was just a matter of re-parsing the 
> source Plug-in folder with a different starting ID no each time until the 
> database then indicates that the Plugin calls are from 4D Internet Commands.

Hi Allan

I wish it were always so. I've just generated some logs using v16R5 and the 
plugins are starting from 3 in them. Maybe it's application-specific. Or maybe 
it's different for server/client/solo (these were client logs).

Anyway, the update I posted to github the other day makes it a little less 
painful to change the plugin numbers - a right-click lets you renumber them all 
in one go and you don't need to reparse the logs files to see the changes - 
just rebuild the hiererchical command list (by holding down the Shift key when 
selecing them) for any processes you've already examined.

cheers
J
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Debug Log Reader - Plugin Commands

[Allan Udy via 4D_Tech]

Hi Justin,

Justin previously determined that this is the case, and in his Log 
Parsing database that we're using, the plugin command ID's are assumed 
to start with  3;YYY.


Through a bit of trial and error I can now say with reasonable certainty 
that in v16 at least, the plugin Commands now appear to start with 1, 
rather than 3 as was necessary in previous versions of your Debug Log 
Reader.


A few of the Plugin commands showing up in my Debug Log files take 
parameters, and so it's relatively straight forward to match them up 
with the actual commands used in the source.  It was just a matter of 
re-parsing the source Plug-in folder with a different starting ID no 
each time until the database then indicates that the Plugin calls are 
from 4D Internet Commands.


Cheers,
Allan Udy

Golden Micro Solutions Ltd, Blenheim, New Zealand
http://www.golden.co.nz


**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Debug Log Reader - Plugin Commands

[Justin Carr via 4D_Tech]

On 30 Nov 2017, at 10:41 am, Keisuke Miyako via 4D_Tech <4d_tech@lists.4d.com> 
wrote:
> 
> to clarify:
> 
> - command number
> 
> the command number explained in this example corresponds to the right side of 
> e.g. "3;1"
> and it can be derived from manifest.json inside the plugin bundle (v14 SDK),
> or a specific resource rsrc inside resources (mac) or RSR adjacent to the 4DX 
> (windows) in legacy architecture.
> 
> - plugin number
> 
> the left side, which was the focus of this thread,
> evidently corresponds to their position in PLUGIN LIST (bumped by the number 
> of preinstalled legacy plugins)
> 
> - plugin ID
> 
> plugins are identified by their name.
> they also have an internal integer ID,
> but they are evidently not used for identification unless the value is lower 
> than 15000.
> 
> - event ID
> 
> sometimes in the log file you see entry point numbers, normally negative,
> which tells which event was process by the plugin.
> they way plugins work is that the scheduler yields to every plugin for every 
> defined event,
> and the plugin may or may not take the opportunity to do something.
> (like form objects, except the plugin does not assert in which set of events 
> it is interested,
> so it is called every time even though it may pass on the majority of events)
> 
> http://sources.4d.com/trac/4d_4dpluginapi/wiki/EntryPoints.h.htm

This is exactly how the Debug Log Reader works, assuming the PLUGIN LIST 
returns the plugins in alphabetical order.

What isn't clear (and I don't expect you to be able to answer) is under what 
circumstances the preinstalled "legacy" plugins come into play. We receive huge 
numbers of log files from our customers and most but not all of the time the 
plugin numbering starts from 3. What the two hidden plugins are and why they 
are loaded in some situtaions and not others is a mystery.

What would be enormously beneficial would be to have the plugin IDs and names 
logged in a separate file wherever the logging is being run, ie. server, client 
or single user.

It would also be extremely helpful if the process info log that gets created on 
the server was also created for clients and single users, given that we don't 
always have logging on the server if we're investigating an issue that 
manifests itself only at the front end.

cheers
J

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Debug Log Reader - Plugin Commands

[Keisuke Miyako via 4D_Tech]

to clarify:

- command number

the command number explained in this example corresponds to the right side of 
e.g. "3;1"
and it can be derived from manifest.json inside the plugin bundle (v14 SDK),
or a specific resource rsrc inside resources (mac) or RSR adjacent to the 4DX 
(windows) in legacy architecture.

- plugin number

the left side, which was the focus of this thread,
evidently corresponds to their position in PLUGIN LIST (bumped by the number of 
preinstalled legacy plugins)

- plugin ID

plugins are identified by their name.
they also have an internal integer ID,
but they are evidently not used for identification unless the value is lower 
than 15000.

- event ID

sometimes in the log file you see entry point numbers, normally negative,
which tells which event was process by the plugin.
they way plugins work is that the scheduler yields to every plugin for every 
defined event,
and the plugin may or may not take the opportunity to do something.
(like form objects, except the plugin does not assert in which set of events it 
is interested,
so it is called every time even though it may pass on the majority of events)

http://sources.4d.com/trac/4d_4dpluginapi/wiki/EntryPoints.h.htm




**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Debug Log Reader - Plugin Commands

[Justin Carr via 4D_Tech]

On 30 Nov 2017, at 2:41 am, Keisuke Miyako via 4D_Tech <4d_tech@lists.4d.com> 
wrote:
> 
> the relation between plugin command names and numbers can be studied from 
> manifest.json (v14 sdk) or *.rsrc (earlier)
> 
> https://github.com/4D-JP/plugin-command-numbers

The number used to identify the plugin in the debug log bears no relation to 
the plugin's id.
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Debug Log Reader - Plugin Commands

[Allan Udy via 4D_Tech]

Hi Keisuke,


From: Keisuke Miyako

the relation between plugin command names and numbers can be studied from 
manifest.json (v14 sdk) or *.rsrc (earlier)

https://github.com/4D-JP/plugin-command-numbers


Greatly appreciate you taking the time trying to help, but I really 
don't understand your message


Where/what is '...manifest.json (v14 sdk) or *.rsrc (earlier)...'  ?

The issue that Justin Carr and I are discussing is in determining from 
the 4D DebugLogs, what plugins are being referred to by the  X;YYY 
references in the Log file.


In your 'plugin-command-numbers' example database from GitHub, in Method 
2 there is the comment:


  //in the v14 format (4), the plugin command is printed as 3;10,
  //where the first figure is the plugin number
  //apparently the index in PLUGIN LIST +2,
  //and the second figure is the command number.

Justin previously determined that this is the case, and in his Log 
Parsing database that we're using, the plugin command ID's are assumed 
to start with  3;YYY.


I'm now trying to examine a Debug Log file generated from a v16R4 
database, and there are Plugin References of the form:2;414


Thus, in v16 (at least), the statement from v14 that '... the index in 
PLUGIN LIST +2...'  is now INCORRECT.


I'm trying to determine if for v16 it should be that '...the index in 
PLUGIN LIST +1...', or indeed if it is something else.


At this stage I'm unsure if the (many many) Plugin calls I'm seeing in 
the log (2;414) are being generated by 4D Internet Commands 
EX_IS_PROCESS_DYING or not... and whether this is having an effect on 
the 4D Remotes being disconnected or not...


When 4D produce a tool as useful as the DebugLog for helping developers 
solve very difficult issues, it would be really great if 4D would also 
publish all the information required by developers to read these logs 
(or better yet provide a good tool to do so).   ;-)  I would have 
thought that a 4D produced (and continually maintained and updated) 
DebugLog Parser and Analyser application would not only be of benefit to 
developers, but also 4D support staff


Cheers,
Allan Udy

Golden Micro Solutions Ltd, Blenheim, New Zealand
http://www.golden.co.nz

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Debug Log Reader - Plugin Commands

[Keisuke Miyako via 4D_Tech]

the relation between plugin command names and numbers can be studied from 
manifest.json (v14 sdk) or *.rsrc (earlier)

https://github.com/4D-JP/plugin-command-numbers



**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Debug Log Reader - Plugin Commands

[Add Komoncharoensiri via 4D_Tech]

Ah, my mistake. You are right, it doesn’t.

Add




On 11/28/17, 8:04 PM, "4D_Tech on behalf of Justin Carr via 4D_Tech" 
<4d_tech-boun...@lists.4d.com on behalf of 4d_tech@lists.4d.com> wrote:

On 29 Nov 2017, at 8:44 am, Add Komoncharoensiri via 4D_Tech 
<4d_tech@lists.4d.com> wrote:
>
> Hi Guys,
>
> Have you looked at New Log Parser technote 
(http://kb.4d.com/assetid=77880). I think you’ll find it to be very useful.
>

Hi Add

Yep. I've given it a look. It's a nice tool but doesn't handle converting 
plugin numbers/function numbers to names, which was the subject of this thread.

cheers
J
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Debug Log Reader - Plugin Commands

[Justin Carr via 4D_Tech]

Hi Allan

For what it's worth I've uploaded a new version to github. This version allows 
renumbering of the plugins using a right-click. It also now determines the 
plugin and function names dynamically when building the hierarchical list (or 
list box if there's a lot of commands) so you don't need to reparse the log 
files from scratch. You just need to Shift-click on any processes which have 
already had the hierarchical list built to force rebuilding of it. Lastly it 
will automatically insert the process name if it can be determined from the 
logs.

cheers
J
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Debug Log Reader - Plugin Commands

[Justin Carr via 4D_Tech]

On 29 Nov 2017, at 8:44 am, Add Komoncharoensiri via 4D_Tech 
<4d_tech@lists.4d.com> wrote:
> 
> Hi Guys,
> 
> Have you looked at New Log Parser technote (http://kb.4d.com/assetid=77880). 
> I think you’ll find it to be very useful.
> 

Hi Add

Yep. I've given it a look. It's a nice tool but doesn't handle converting 
plugin numbers/function numbers to names, which was the subject of this thread.

cheers
J
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Debug Log Reader - Plugin Commands

[Add Komoncharoensiri via 4D_Tech]

Hi Guys,

Have you looked at New Log Parser technote (http://kb.4d.com/assetid=77880). I 
think you’ll find it to be very useful.

Regards,
Add




On 11/28/17, 11:40 AM, "4D_Tech on behalf of Allan Udy via 4D_Tech" 
<4d_tech-boun...@lists.4d.com on behalf of 4d_tech@lists.4d.com> wrote:

Hi Justin,

Thanks for that extra info. I'll have to have a play around some more
and try and work out what 4D is actually doing.

ATTENTION 4D:   It would be really useful if you would actually publish
the details (somewhere easy to find) of how the DebugLogs determine the
Plugin Command IDs

Allan


> From: Justin Carr
>
>> >
>> >  HI,
>> >
>> >  A while back I got hold of the v15 '4D  Debug Log Reader' database in 
order to view some of4D's Low Level debugging Log files.  It helped me a lot at 
the time.Does anyone (the author?) know WHY  $vL_PluginID is initialised to 3 
at the start of the loop through the external plugins?
>>
> That would be me. This was always a tricky piece of the puzzle. 
Unfortunately there was no way (that I could find) for determining which plugin 
was allocated which number in the debug logs. The best I could determine was 
that it seemed to load them in alphabetical order and it usually seemed to 
start from 3, although not always. I think there used to be two "plugins" that 
were built-in to4D  - I want to say one of them was the OLE Tools - not sure 
what the other one was. I deduced rightly or wrongly that4D  was allocating 
these numbers first. It's entirely possible that this has changed in more 
recent versions.
>
> If it seems to be starting from one now, it would certainly be safe to 
change it in the code. You can also change the numbers in the front-end by 
double-clicking on them, but the log files would ned to be re-parsed after 
you've changed any numbers.
>


Cheers,
Allan Udy

Golden Micro Solutions Ltd, Blenheim, New Zealand
http://www.golden.co.nz

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Debug Log Reader - Plugin Commands

[Allan Udy via 4D_Tech]

Hi Justin,

Thanks for that extra info. I'll have to have a play around some more 
and try and work out what 4D is actually doing.


ATTENTION 4D:   It would be really useful if you would actually publish 
the details (somewhere easy to find) of how the DebugLogs determine the 
Plugin Command IDs


Allan



From: Justin Carr

>  
>  HI,
>  
>  A while back I got hold of the v15 '4D  Debug Log Reader' database in order to view some of4D's Low Level debugging Log files.  It helped me a lot at the time.Does anyone (the author?) know WHY  $vL_PluginID is initialised to 3 at the start of the loop through the external plugins?



That would be me. This was always a tricky piece of the puzzle. Unfortunately there was 
no way (that I could find) for determining which plugin was allocated which number in the 
debug logs. The best I could determine was that it seemed to load them in alphabetical 
order and it usually seemed to start from 3, although not always. I think there used to 
be two "plugins" that were built-in to4D  - I want to say one of them was the 
OLE Tools - not sure what the other one was. I deduced rightly or wrongly that4D  was 
allocating these numbers first. It's entirely possible that this has changed in more 
recent versions.

If it seems to be starting from one now, it would certainly be safe to change 
it in the code. You can also change the numbers in the front-end by 
double-clicking on them, but the log files would ned to be re-parsed after 
you've changed any numbers.




Cheers,
Allan Udy

Golden Micro Solutions Ltd, Blenheim, New Zealand
http://www.golden.co.nz

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: 4D Debug Log Reader - Plugin Commands

[Justin Carr via 4D_Tech]

On 28 Nov 2017, at 1:52 pm, Allan Udy via 4D_Tech <4d_tech@lists.4d.com> wrote:
> 
> HI,
> 
> A while back I got hold of the v15 '4D Debug Log Reader' database in order to 
> view some of 4D's Low Level debugging Log files.  It helped me a lot at the 
> time.
> 
> I've still got the Read Me pdf but it doesn't say where this came from or who 
> wrote it, and I've now forgotten.
> 
> The application has to parse the Plugins folder of the source database that 
> you a're debugging, in order to correctly identify the actual command names 
> used by the Plugins.
> 
> Having upgraded this to v16R4, I'm now having problems looking at the log 
> files from the v16 application that we're debugging.
> 
> There's a method in the code called 'RSRC_Select' which basically goes 
> through the plugin folder and parses all the commands:
> 
>$vL_NumPlugins:=Size of array($aT_Bundle)` How many .bundle Plugins are 
> there
> 
>If ($vL_NumPlugins>0)
> 
>$vL_PluginID:=3 <-  HERE'S THE QUESTION -- WHY IS THIS 3
> 
>For ($vL_PluginIndex;1;$vL_NumPlugins)
>If (RSRC_Parse 
> ($vT_Folder+$aT_Bundle{$vL_PluginIndex};$vL_PluginID))
>$vL_PluginID:=$vL_PluginID+1
>End if
>End for
>End if
> 
> 
> Does anyone (the author?) know WHY  $vL_PluginID is initialised to 3 at the 
> start of the loop through the external plugins?

That would be me. This was always a tricky piece of the puzzle. Unfortunately 
there was no way (that I could find) for determining which plugin was allocated 
which number in the debug logs. The best I could determine was that it seemed 
to load them in alphabetical order and it usually seemed to start from 3, 
although not always. I think there used to be two "plugins" that were built-in 
to 4D - I want to say one of them was the OLE Tools - not sure what the other 
one was. I deduced rightly or wrongly that 4D was allocating these numbers 
first. It's entirely possible that this has changed in more recent versions.

If it seems to be starting from one now, it would certainly be safe to change 
it in the code. You can also change the numbers in the front-end by 
double-clicking on them, but the log files would ned to be re-parsed after 
you've changed any numbers.

cheers
Justin Carr
Genie Solutions
**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

4D Debug Log Reader - Plugin Commands

[Allan Udy via 4D_Tech]

HI,

A while back I got hold of the v15 '4D Debug Log Reader' database in 
order to view some of 4D's Low Level debugging Log files.  It helped me 
a lot at the time.


I've still got the Read Me pdf but it doesn't say where this came from 
or who wrote it, and I've now forgotten.


The application has to parse the Plugins folder of the source database 
that you a're debugging, in order to correctly identify the actual 
command names used by the Plugins.


Having upgraded this to v16R4, I'm now having problems looking at the 
log files from the v16 application that we're debugging.


There's a method in the code called 'RSRC_Select' which basically goes 
through the plugin folder and parses all the commands:


$vL_NumPlugins:=Size of array($aT_Bundle)` How many .bundle Plugins 
are there


If ($vL_NumPlugins>0)

$vL_PluginID:=3 <-  HERE'S THE QUESTION -- WHY IS THIS 3

For ($vL_PluginIndex;1;$vL_NumPlugins)
If (RSRC_Parse 
($vT_Folder+$aT_Bundle{$vL_PluginIndex};$vL_PluginID))

$vL_PluginID:=$vL_PluginID+1
End if
End for
End if


Does anyone (the author?) know WHY  $vL_PluginID is initialised to 3 at 
the start of the loop through the external plugins?


This suggests to me that Plug 1 and 2 might be internal to 4D?

In the v16 Log files I'm trying to read, there are calls to plugin 
2;414  --- that is Plugin ID 2, but I have no idea what that plugid 
refers to.


Any ideas?


Cheers,
Allan Udy

Golden Micro Solutions Ltd, Blenheim, New Zealand
http://www.golden.co.nz

**
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**