Hi
Eric,
Not
sure if your feed has duplicate records, but if it does, the following
strategy may help.
We
previously had an escalation which kicked off the import. The data being
imported was employee names, locations, etc... Remedy was taking a long
time to process this, due to the sheer volume of
records.
The
thing that seemed silly about this was: Most of the records were static
from one day to the next. We were basically importing tens of
thousands of records, and letting Remedy update any existing records with the
information which had already been there, in order to allow it to create or
update a few dozen records with new data.
We wrote a shell script to do the
following:
- Filter out any lines in the import
file which were identical to lines from the previous import file we
were sent.
- Dump the remaining records to a
staging file.
- Overwrite yesterday's import file with today's
import file, in preparation for tomorrow's run.
- Kick off the import using the staging
file, rather than the complete file.
We
then pointed the existing escalation at that script.
This
helped tremendously. It cut the import time from
several hours to under a minute.
Eric
Cleereman
__20060125_______________________This posting was submitted with HTML in it___-----Original Message-----**
From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED]On Behalf Of Rick Cook
Sent: Friday, May 05, 2006 1:31 PM
To: [email protected]
Subject: Re: Importing DataDon't forget one other standard rule of data import into Remedy - drop the indexes on the import form, and add them back in afterward. Will speed things up to not have the indexes constantly creating after every record is submitting.Rick
**
From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Luke, Brian
Sent: Friday, May 05, 2006 10:19 AM
To: [email protected]
Subject: Re: Importing DataNot positive if it would be faster, but you can eliminate some scheduled steps by using an import mapping and arimportcmd.exe to import the CSV directly into a Remedy form. So long as arimportcmd.exe is executed on the ARS server and the csv file is local, it normally goes quickly.The import tool saves the mapping as an .arm file, and you give it a name similar to a macro.Also like a macro, you don't reference the macro filename directly, but add its directory to the search path, and it finds the name.example:"C:\Program Files\AR System\Admin\arimportcmd.exe" -x %COMPUTERNAME% -u Demo -p password -m SHR-DS-POGOData -d %CD% -f SHR-DS-POGOPeople -l %LogFileName% -o pogo_domain_people_data.csv-m is the mapping name-d is the directory to look for the .arm file inThe rest should be self explanitory.HTH,-Brian Luke
**
From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Dunn, Eric
Sent: Thursday, May 04, 2006 7:35 AM
To: [email protected]
Subject: Importing DataMorning List,Need some advice. First I'll give you my specs:5 year old server running Windows 2000 with SQL Server 2000. ARSystem 5.1.2.Process I am trying to complete with my plans:I have a feed coming from an access database in the form of a csv file. It contains roughly 13,000 records. This data will be imported into a schema in Remedy every month.My plan: the csv file is delivered, a scheduled DTS package truncates the temp table in the database, imports the data and deletes the csv file. The temp table is built as a View Form in Remedy and an escalation is run every night to deliver the data to the schema that is used by the customer. This works. The first import wasn't that bad, the second import took a couple of hours to complete and the third took more than 5 hours to complete. Any ideas on how I can make this automation faster or better?ThanksEric W. DunnProgrammer Analyst III - Remedy/CollaborationMcLeod USA, Inc. - Network Engineering8665 New Trails Dr.The Woodlands, TX 77381(281)465-1265(281)465-1951 fax
NOTICE: This electronic mail transmission may contain confidential information and is intended only for the person(s) named. Any use, copying or disclosure by any other person is strictly prohibited. If you have received this transmission in error, please notify the sender via e-mail.
__20060125_______________________This posting was submitted with HTML in it___ __20060125_______________________This posting was submitted with HTML in it___ __20060125_______________________This posting was submitted with HTML in it___
