Hi Brian,
It's what in Sydney ... Saturday early morning before 8am? And you just
wrote this detailed post? You must eat technical manuals for breakfast
and
string codes for dessert. I tip my hat to you sir.
Thank you very much for the depth of this explanation. This certainly
saved
me a ton of time and work in trial/error, and misdirection. Now I have a
roadmap to follow.
I'll let you know if I succeed and hopefully even come up with my own
twists.
I wonder if Lew, Bruce and Thom were able to finish that new DP version
with
STDOUT printing you mention.
Sincere gratitude for all your time and kind guidance.
Ed.
-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Brian Hancock
Sent: Friday, August 10, 2007 2:42 PM
To: Dataperfect Users Discussion Group
Subject: Re: [Dataperf] Linking DataPerfect data to the Website
Hi Ed,
You should be able to still use the application despite it having a user
login add /U-username:password to the DataPerfect command line eg
DP.EXE wargames.str /U-gwb:afghanistan /EI-tjn74owg.log /SID-tjn74owg /W
In my application I have a panel that accepts all the CGI data pairs, one
record per pair. That panel has 4 fields,
SessionID
FieldName
FieldData
ReportNumber
The webpage has two fields, username and target. (this username has
nothing
to do with the DP login) . This gets sent to my server script, where I
create the sythetic TLOG which looks like:
#C:
=2=*1"tjn74owg"*2"username"*3"George"*4"14"**
#C:
=2=*1"tjn74owg"*2"target"*3"Iraq"*4"14"**
#R:
=14=
The Report Number that I embed in each line is not really necessary,
however
because its easy to add, or delete reports in DP, and since the /#R:=
command uses the ordinal position of the report on the report list, I use
it
for control, so my report checks that the correct data for that report is
being sent. In the above example Report 14 might have been a program to
send aid to the target, and report 15 might have been to nuke it, and so
these do not get mixed up the report validates that it is getting data for
report 14 and exists gracefully if you are not running the correct report.
I use the PERL module
use File::Temp qw/ tempfile tempdir
together with the function
my ($fh, $filename) = tempfile("TXXXXXXX", DIR => $dir,SUFFIX => ".LOG",
UNLINK => 1) ;
to create a unique filename, which is automatically deleted (unlinked)
when
the script finishes. When I am debugging I leave the unlink bit out so
that
the log files are left on the server. I put all the LOG files and other
temporary files in a TEMP subfolder so my DP startup command is more like:
DP.EXE wargames.str /D-/U-gwb:afghanistan /EI-temp/tjn74owg.log
/SID-tjn74owg /W /D-temp
Each of my DP reports is based on a dummy panel, which has one record
only.
Actually it is not just a dummy record, I record all my system variables,
eg
licence dates, limitations on number of records that can be created, date
formats, server timezones, folders, URL's etc. etc...
the Reports all write their data to a file with the filename
/temp/user.field[0].XML (ie based on the SID). My script deletes this file
after it has finished using it. (Note that DP always writes it files named
with uppercase letters even if you specify then as lowercase. This is
important to note if you are running on a Linux/Unix server.
In the body of my report, I firstly create a SubReport with a virtual link
to the WebParams panel where my CGI data was dumped from the TLOG, linked
on
SessionID which I retrieve with user.field[0] and the Report Number. I
then
loop through each record looking for the specific field names and data for
the report, and pull the data out into RV's. If incorrect data is
supplied,
then I can exit the report.
Immediately after this SubReport, I create a subreport back to this same
panel, linked the same way, but this time I delete each record, so that my
webparams panel is not growing uncontrollably, and also so that in the
extremely unlikely even that a SessionID (which I create from the base of
the Temporary filename I use for the TLOG) is reused. I think the odds of
that are about 1 in 79 billion. But it is good housekeeping to delete
these
except when you are debugging.
After that sub report finishes, I then put SubReports handling the logic
of
the application. I use lots of SubReports, mainly to provide looping
structures, and data checking. My most complex report has 39 sub reports,
but most of my reports have about about 5 or so...
In the DP application you have to design panels with the thought of
quickly
processing reports, making sure you have all the necessary indexes, so you
can either create Virtual SubReports or using SKIP to RV so that you are
not
iterating across data which is not needed. You only have about 2 minutes
before the webserver will force a shutdown on DP, 60 seconds before a web
browser will time out, or about 20 seconds before a user will time out, or
5
milliseconds for an impatient user.
I use XML as the output from DP most of the time. Because the logic in
reports can be very complex, you do not want to be generating HTML
presentation stuff in their too, and you have only about 64k for the
definition of a DP report. With XML I can use comments <!-- comment -->
to
document the report which helps in debugging. Because I often have lots of
SubReports, I number the sections and document the entry and exit from
them
with comments such as
<!-- 15. Begin Products -->
and on exit
<!-- 15 Exit Products -->
They make it easier to navigate when you are writing the reports. They add
a
little to the size of the XML output, but I strip out the comments with
XSLT
before delivering it back to the client application, but it makes
debugging
the reports a whole lot easier...
For most of the time my PERL scripts are very straightforward, so I can
use
Library functions. All they really need to do is to capture the web data,
write it to the TLOG, invoke DOSEMU (in my case) and the DP application,
wait for DP to finish, either reading in the data and do whatever ever is
necessary with it, eg "print" it back to the user, run an XSLT processor,
or
send it to mail or text messaging gateway. Generally a new simple script
might only take about 10 lines of code. The libary functions naturally
take up a bit of coding...
There was some discussions a while back that Lew, Bruce and Thom were
working on a new version of DP that could print directly to STDOUT which
could mean good benefits for some DP web applications. Saving some of the
mucking around with temporary files.
Regards
Brian
----- Original Message -----
From: "E. Marfil, MAST UNITED" <[EMAIL PROTECTED]>
To: "'Dataperfect Users Discussion Group'" <[email protected]>
Sent: Friday, August 10, 2007 9:46 AM
Subject: RE: [Dataperf] Linking DataPerfect data to the Website
Hi Brian,
Okay, I can invoke DP and run the report using a synthetic transaction
log.
However, it stops at the user login prompt. Can I add something to the
script or transaction log to make it skip the login? Or perhaps I should
create a non-secured .str db (let's call it weblink.str) with only one
panel
Param.DAT (if I may borrow yours) accessing a specific datafile of my
real
.str db. Is this what you mean in your notes regarding using one panel?
I understand that I can use the web user provided query i.e. order number
"12345678" as the session id or transaction log filename created in the
Param.DAT panel. Am I correct to assume the reason the record needs to
be
created is because this is the only way to relay this data to the report,
since we can't have the DP report function prompting the web user for the
report parameters? If this is so, then I should have the record deleted
from the panel at the end of the report, thus keeping the panel empty.
My
only worry here is if additional users query the database before a record
has been deleted.
As for the transaction log file and the text file resulted from the
report,
can these files be automatically deleted after the report content has
been
retrieved by PERL to display on the web user's screen?
Thank you for your help.
Ed.
-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Brian Hancock
Sent: Tuesday, August 07, 2007 8:48 PM
To: Dataperfect Users Discussion Group
Subject: Re: [Dataperf] Linking DataPerfect data to the Website
Hi Ed,
In DP version 2.x there is the ability to start DP from the command line,
and have it process a transaction log.
eg
dp.exe app.str /w /ei=tlog.log /sid=uniqueid
/w= Latin ISO char set
/ei=reference to a synthetic transaction log file
/sid=unique identifier, which can be retrieved from reports using the
user.field[0] variable
The Transaction log is one which I create with the Perl scripts... it
could
be created in VB, VB.NET, C+, C#, or any other language.. Transaction
Logs
are created by DP when you turn the feature on, in this case the
Transaction
is not created natively by DP so it is referred to as a "synthetic"
transaction log. In DP v 2.x an additional command, the =R: so that a DP
report can be run...
Although you could use the TLog to import data directly into the target
panels, I prefer to import it to a parameters panel where the report can
loop through each of the values.
Because web is multiuser you need a way to bind the data in the TLog
(which
needs to be uniquely named) to that particular execution instance of DP.
By
including the value used in the /sid parameter as a field in each import
record in the TLog, you can retrieve the execution "session identifier"
with
the user.field[0] and bind it to the data in the paramaters import table.
Although you can create HTML and use CSS to style it directly from DP, I
prefer to deliver XML and then use XSLT and CSS to present it. This means
I
can deliver the data to a variety of client programs, and not just a web
browser.
I run it on a hosting company's Linux server using DOSEMU... It runs on
Windows, using the Apache web server for Windows, but I had problems
running
it on Windows Internet Information Server IIS... (although it has been a
while since I tried it, and perhaps newer versions make it possible).
I never have to touch the system, it just keeps running... I have served
many pages, and I find it completely reliable. I cannot rememeber exactly
but I have run at least 10 concurrent sessions (each session only lasts a
few seconds), and so it can handle quite a large loading...
Good luck, I really hope someone else starts running with this, as I am
sure others will come up with good ideas...
Regards
Brian
----- Original Message -----
From: "E. Marfil, MAST UNITED" <[EMAIL PROTECTED]>
To: "'Dataperfect Users Discussion Group'" <[email protected]>
Sent: Wednesday, August 08, 2007 12:26 PM
Subject: RE: [Dataperf] Linking DataPerfect data to the Website
Hi Brian,
I read the reference in the DP Manual regarding the Transaction Log but
it
doesn't explain much of anything regarding it's use in a web environ.
If I understood your article right ...
The PERL script (which I assume is a type of CGI script) invoked from
the
website, triggers DP in the server to run a specific panel's transaction
log
report, then shuts down DP again, reads the log file searching for a
match
to the parameters selected by the web user, retrieving said data and
displaying to the web user's screen.
Unfortunately, I didn't understand where (or how) you triggered DP to
start
and run the specific report.
Please let me know if my very limited understanding is at least in the
ballpark or if I'm not even in the same zip code that maybe I should get
professional help.
Thanks.
Ed.
-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of E. Marfil, MAST
UNITED
Sent: Monday, August 06, 2007 1:25 PM
To: 'Dataperfect Users Discussion Group'
Subject: RE: [Dataperf] Linking DataPerfect data to the Website
Hi Brian,
Thank you for the link and for posting your process. There is a lot of
new
information to digest and more research on my part to understand the
info.
Even understanding only a small portion of it I can see that you must
have
spent a lot of effort in keeping the construction as simple as possible
but
still derive the results you aimed for.
Regards,
Ed.
-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Brian Hancock
Sent: Sunday, August 05, 2007 6:59 PM
To: Dataperfect Users Discussion Group
Subject: Re: [Dataperf] Linking DataPerfect data to the Website
Hi Edward,
I have put some notes together on my website about the web enabling of
an
application..
http://brileigh.net/bws/story/
I was hoping to release a commerical application based on it for this
year,
but I have been snowed under with other work, so I have stilled used the
pilot of it again this year... It is working well, and is very
reliable.
I
have served up hundreds of thousands of session on it now...
Regards
Brian
----- Original Message -----
From: "E. Marfil, MAST UNITED" <[EMAIL PROTECTED]>
To: "'Dataperfect Users Discussion Group'" <[email protected]>
Sent: Monday, August 06, 2007 5:28 AM
Subject: [Dataperf] Linking DataPerfect data to the Website
Hello all.
I'm seeking advice on how to provide data from a DP database to a
website
inquiry prompt.
I'm trying to build a website where users can query an order number and
the
result will show the historical and current status of that order. The
status data is stored in a DataPerfect database on my server. All I
need
is
to have the webpage extract it from DataPerfect. I'm not HTML savvy so
I'm
not sure if this is simple to do or not.
Right now, my option is to have DP run a report creating a file for
each
order number (using the number as the filename) with its status history
and
when queried, the website opens the filename and displays the data.
Unfortunately, this means the data is only as fresh as when the DP
report
was most recently run, as well as having to manually run the report
frequently. I'm wondering if there is a more "real-time" method to
this
and
hopefully automatic or less human involvement to keep the data current.
Thanks for any opinion.
Ed.
_______________________________________________
Dataperf mailing list
[email protected]
http://lists.dataperfect.nl/mailman/listinfo/dataperf
_______________________________________________
Dataperf mailing list
[email protected]
http://lists.dataperfect.nl/mailman/listinfo/dataperf
_______________________________________________
Dataperf mailing list
[email protected]
http://lists.dataperfect.nl/mailman/listinfo/dataperf
_______________________________________________
Dataperf mailing list
[email protected]
http://lists.dataperfect.nl/mailman/listinfo/dataperf
_______________________________________________
Dataperf mailing list
[email protected]
http://lists.dataperfect.nl/mailman/listinfo/dataperf
_______________________________________________
Dataperf mailing list
[email protected]
http://lists.dataperfect.nl/mailman/listinfo/dataperf
_______________________________________________
Dataperf mailing list
[email protected]
http://lists.dataperfect.nl/mailman/listinfo/dataperf
_______________________________________________
Dataperf mailing list
[email protected]
http://lists.dataperfect.nl/mailman/listinfo/dataperf
