Hi Brian, It's what in Sydney ... Saturday early morning before 8am? And you just wrote this detailed post? You must eat technical manuals for breakfast and string codes for dessert. I tip my hat to you sir.
Thank you very much for the depth of this explanation. This certainly saved me a ton of time and work in trial/error, and misdirection. Now I have a roadmap to follow. I'll let you know if I succeed and hopefully even come up with my own twists. I wonder if Lew, Bruce and Thom were able to finish that new DP version with STDOUT printing you mention. Sincere gratitude for all your time and kind guidance. Ed. -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Brian Hancock Sent: Friday, August 10, 2007 2:42 PM To: Dataperfect Users Discussion Group Subject: Re: [Dataperf] Linking DataPerfect data to the Website Hi Ed, You should be able to still use the application despite it having a user login add /U-username:password to the DataPerfect command line eg DP.EXE wargames.str /U-gwb:afghanistan /EI-tjn74owg.log /SID-tjn74owg /W In my application I have a panel that accepts all the CGI data pairs, one record per pair. That panel has 4 fields, SessionID FieldName FieldData ReportNumber The webpage has two fields, username and target. (this username has nothing to do with the DP login) . This gets sent to my server script, where I create the sythetic TLOG which looks like: #C: =2=*1"tjn74owg"*2"username"*3"George"*4"14"** #C: =2=*1"tjn74owg"*2"target"*3"Iraq"*4"14"** #R: =14= The Report Number that I embed in each line is not really necessary, however because its easy to add, or delete reports in DP, and since the /#R:= command uses the ordinal position of the report on the report list, I use it for control, so my report checks that the correct data for that report is being sent. In the above example Report 14 might have been a program to send aid to the target, and report 15 might have been to nuke it, and so these do not get mixed up the report validates that it is getting data for report 14 and exists gracefully if you are not running the correct report. I use the PERL module use File::Temp qw/ tempfile tempdir together with the function my ($fh, $filename) = tempfile("TXXXXXXX", DIR => $dir,SUFFIX => ".LOG", UNLINK => 1) ; to create a unique filename, which is automatically deleted (unlinked) when the script finishes. When I am debugging I leave the unlink bit out so that the log files are left on the server. I put all the LOG files and other temporary files in a TEMP subfolder so my DP startup command is more like: DP.EXE wargames.str /D-/U-gwb:afghanistan /EI-temp/tjn74owg.log /SID-tjn74owg /W /D-temp Each of my DP reports is based on a dummy panel, which has one record only. Actually it is not just a dummy record, I record all my system variables, eg licence dates, limitations on number of records that can be created, date formats, server timezones, folders, URL's etc. etc... the Reports all write their data to a file with the filename /temp/user.field[0].XML (ie based on the SID). My script deletes this file after it has finished using it. (Note that DP always writes it files named with uppercase letters even if you specify then as lowercase. This is important to note if you are running on a Linux/Unix server. In the body of my report, I firstly create a SubReport with a virtual link to the WebParams panel where my CGI data was dumped from the TLOG, linked on SessionID which I retrieve with user.field[0] and the Report Number. I then loop through each record looking for the specific field names and data for the report, and pull the data out into RV's. If incorrect data is supplied, then I can exit the report. Immediately after this SubReport, I create a subreport back to this same panel, linked the same way, but this time I delete each record, so that my webparams panel is not growing uncontrollably, and also so that in the extremely unlikely even that a SessionID (which I create from the base of the Temporary filename I use for the TLOG) is reused. I think the odds of that are about 1 in 79 billion. But it is good housekeeping to delete these except when you are debugging. After that sub report finishes, I then put SubReports handling the logic of the application. I use lots of SubReports, mainly to provide looping structures, and data checking. My most complex report has 39 sub reports, but most of my reports have about about 5 or so... In the DP application you have to design panels with the thought of quickly processing reports, making sure you have all the necessary indexes, so you can either create Virtual SubReports or using SKIP to RV so that you are not iterating across data which is not needed. You only have about 2 minutes before the webserver will force a shutdown on DP, 60 seconds before a web browser will time out, or about 20 seconds before a user will time out, or 5 milliseconds for an impatient user. I use XML as the output from DP most of the time. Because the logic in reports can be very complex, you do not want to be generating HTML presentation stuff in their too, and you have only about 64k for the definition of a DP report. With XML I can use comments <!-- comment --> to document the report which helps in debugging. Because I often have lots of SubReports, I number the sections and document the entry and exit from them with comments such as <!-- 15. Begin Products --> and on exit <!-- 15 Exit Products --> They make it easier to navigate when you are writing the reports. They add a little to the size of the XML output, but I strip out the comments with XSLT before delivering it back to the client application, but it makes debugging the reports a whole lot easier... For most of the time my PERL scripts are very straightforward, so I can use Library functions. All they really need to do is to capture the web data, write it to the TLOG, invoke DOSEMU (in my case) and the DP application, wait for DP to finish, either reading in the data and do whatever ever is necessary with it, eg "print" it back to the user, run an XSLT processor, or send it to mail or text messaging gateway. Generally a new simple script might only take about 10 lines of code. The libary functions naturally take up a bit of coding... There was some discussions a while back that Lew, Bruce and Thom were working on a new version of DP that could print directly to STDOUT which could mean good benefits for some DP web applications. Saving some of the mucking around with temporary files. Regards Brian ----- Original Message ----- From: "E. Marfil, MAST UNITED" <[EMAIL PROTECTED]> To: "'Dataperfect Users Discussion Group'" <[email protected]> Sent: Friday, August 10, 2007 9:46 AM Subject: RE: [Dataperf] Linking DataPerfect data to the Website > Hi Brian, > > Okay, I can invoke DP and run the report using a synthetic transaction > log. > However, it stops at the user login prompt. Can I add something to the > script or transaction log to make it skip the login? Or perhaps I should > create a non-secured .str db (let's call it weblink.str) with only one > panel > Param.DAT (if I may borrow yours) accessing a specific datafile of my real > .str db. Is this what you mean in your notes regarding using one panel? > > I understand that I can use the web user provided query i.e. order number > "12345678" as the session id or transaction log filename created in the > Param.DAT panel. Am I correct to assume the reason the record needs to be > created is because this is the only way to relay this data to the report, > since we can't have the DP report function prompting the web user for the > report parameters? If this is so, then I should have the record deleted > from the panel at the end of the report, thus keeping the panel empty. My > only worry here is if additional users query the database before a record > has been deleted. > > As for the transaction log file and the text file resulted from the > report, > can these files be automatically deleted after the report content has been > retrieved by PERL to display on the web user's screen? > > Thank you for your help. > > Ed. > > > > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Brian Hancock > Sent: Tuesday, August 07, 2007 8:48 PM > To: Dataperfect Users Discussion Group > Subject: Re: [Dataperf] Linking DataPerfect data to the Website > > Hi Ed, > > In DP version 2.x there is the ability to start DP from the command line, > and have it process a transaction log. > > eg > > dp.exe app.str /w /ei=tlog.log /sid=uniqueid > > /w= Latin ISO char set > /ei=reference to a synthetic transaction log file > /sid=unique identifier, which can be retrieved from reports using the > user.field[0] variable > > The Transaction log is one which I create with the Perl scripts... it > could > > be created in VB, VB.NET, C+, C#, or any other language.. Transaction > Logs > are created by DP when you turn the feature on, in this case the > Transaction > > is not created natively by DP so it is referred to as a "synthetic" > transaction log. In DP v 2.x an additional command, the =R: so that a DP > report can be run... > > Although you could use the TLog to import data directly into the target > panels, I prefer to import it to a parameters panel where the report can > loop through each of the values. > > Because web is multiuser you need a way to bind the data in the TLog > (which > needs to be uniquely named) to that particular execution instance of DP. > By > > including the value used in the /sid parameter as a field in each import > record in the TLog, you can retrieve the execution "session identifier" > with > > the user.field[0] and bind it to the data in the paramaters import table. > > Although you can create HTML and use CSS to style it directly from DP, I > prefer to deliver XML and then use XSLT and CSS to present it. This means > I > can deliver the data to a variety of client programs, and not just a web > browser. > > I run it on a hosting company's Linux server using DOSEMU... It runs on > Windows, using the Apache web server for Windows, but I had problems > running > > it on Windows Internet Information Server IIS... (although it has been a > while since I tried it, and perhaps newer versions make it possible). > > I never have to touch the system, it just keeps running... I have served > many pages, and I find it completely reliable. I cannot rememeber exactly > but I have run at least 10 concurrent sessions (each session only lasts a > few seconds), and so it can handle quite a large loading... > > Good luck, I really hope someone else starts running with this, as I am > sure others will come up with good ideas... > > Regards > Brian > > > > ----- Original Message ----- > From: "E. Marfil, MAST UNITED" <[EMAIL PROTECTED]> > To: "'Dataperfect Users Discussion Group'" <[email protected]> > Sent: Wednesday, August 08, 2007 12:26 PM > Subject: RE: [Dataperf] Linking DataPerfect data to the Website > > >> Hi Brian, >> >> I read the reference in the DP Manual regarding the Transaction Log but >> it >> doesn't explain much of anything regarding it's use in a web environ. >> If I understood your article right ... >> The PERL script (which I assume is a type of CGI script) invoked from the >> website, triggers DP in the server to run a specific panel's transaction >> log >> report, then shuts down DP again, reads the log file searching for a >> match >> to the parameters selected by the web user, retrieving said data and >> displaying to the web user's screen. >> Unfortunately, I didn't understand where (or how) you triggered DP to >> start >> and run the specific report. >> >> Please let me know if my very limited understanding is at least in the >> ballpark or if I'm not even in the same zip code that maybe I should get >> professional help. >> >> Thanks. >> >> Ed. >> >> >> >> >> -----Original Message----- >> From: [EMAIL PROTECTED] >> [mailto:[EMAIL PROTECTED] On Behalf Of E. Marfil, MAST >> UNITED >> Sent: Monday, August 06, 2007 1:25 PM >> To: 'Dataperfect Users Discussion Group' >> Subject: RE: [Dataperf] Linking DataPerfect data to the Website >> >> Hi Brian, >> >> Thank you for the link and for posting your process. There is a lot of >> new >> information to digest and more research on my part to understand the >> info. >> Even understanding only a small portion of it I can see that you must >> have >> spent a lot of effort in keeping the construction as simple as possible >> but >> still derive the results you aimed for. >> >> Regards, >> >> Ed. >> >> >> -----Original Message----- >> From: [EMAIL PROTECTED] >> [mailto:[EMAIL PROTECTED] On Behalf Of Brian Hancock >> Sent: Sunday, August 05, 2007 6:59 PM >> To: Dataperfect Users Discussion Group >> Subject: Re: [Dataperf] Linking DataPerfect data to the Website >> >> Hi Edward, >> >> I have put some notes together on my website about the web enabling of an >> application.. >> http://brileigh.net/bws/story/ >> >> I was hoping to release a commerical application based on it for this >> year, >> but I have been snowed under with other work, so I have stilled used the >> pilot of it again this year... It is working well, and is very reliable. >> I >> >> have served up hundreds of thousands of session on it now... >> >> Regards >> Brian >> >> >> >> ----- Original Message ----- >> From: "E. Marfil, MAST UNITED" <[EMAIL PROTECTED]> >> To: "'Dataperfect Users Discussion Group'" <[email protected]> >> Sent: Monday, August 06, 2007 5:28 AM >> Subject: [Dataperf] Linking DataPerfect data to the Website >> >> >>> Hello all. >>> >>> I'm seeking advice on how to provide data from a DP database to a >>> website >>> inquiry prompt. >>> >>> I'm trying to build a website where users can query an order number and >>> the >>> result will show the historical and current status of that order. The >>> status data is stored in a DataPerfect database on my server. All I >>> need >>> is >>> to have the webpage extract it from DataPerfect. I'm not HTML savvy so >>> I'm >>> not sure if this is simple to do or not. >>> >>> Right now, my option is to have DP run a report creating a file for each >>> order number (using the number as the filename) with its status history >>> and >>> when queried, the website opens the filename and displays the data. >>> Unfortunately, this means the data is only as fresh as when the DP >>> report >>> was most recently run, as well as having to manually run the report >>> frequently. I'm wondering if there is a more "real-time" method to this >>> and >>> hopefully automatic or less human involvement to keep the data current. >>> >>> Thanks for any opinion. >>> >>> Ed. >>> >>> _______________________________________________ >>> Dataperf mailing list >>> [email protected] >>> http://lists.dataperfect.nl/mailman/listinfo/dataperf >>> >> >> _______________________________________________ >> Dataperf mailing list >> [email protected] >> http://lists.dataperfect.nl/mailman/listinfo/dataperf >> >> _______________________________________________ >> Dataperf mailing list >> [email protected] >> http://lists.dataperfect.nl/mailman/listinfo/dataperf >> >> _______________________________________________ >> Dataperf mailing list >> [email protected] >> http://lists.dataperfect.nl/mailman/listinfo/dataperf >> > > _______________________________________________ > Dataperf mailing list > [email protected] > http://lists.dataperfect.nl/mailman/listinfo/dataperf > > _______________________________________________ > Dataperf mailing list > [email protected] > http://lists.dataperfect.nl/mailman/listinfo/dataperf > _______________________________________________ Dataperf mailing list [email protected] http://lists.dataperfect.nl/mailman/listinfo/dataperf _______________________________________________ Dataperf mailing list [email protected] http://lists.dataperfect.nl/mailman/listinfo/dataperf
