I have found and corrected the problem. There were 5 records that had a stray double-quote in them. I will upload the corrected file tomorrow morning.
Regards, - Joe From: Dan Connolly [mailto:[email protected]] Sent: Wednesday, January 29, 2014 1:18 PM To: Finamore, Joe D Cc: [email protected] Subject: loading I2B2 concepts: KUMC, MCW, UMN OK; struggles with MCRF, UTSW, WISC Progress on loading stuff into babel<http://babel.gpcnetwork.org/> continues... 3 down, but... Joe, I'm having trouble with wierd quotes in the c_basecode field the Marshfield CSV data: /usr/bin/psql --host localhost --dbname i2b2 --user i2b2metadata --no-password -c copy mcrf_terms(C_HLEVEL, C_FULLNAME, C_NAME, C_SYNONYM_CD, C_VISUALATTRIBUTES, C_TOTALNUM, C_BASECODE, C_METADATAXML, C_FACTTABLECOLUMN, C_TABLENAME, C_COLUMNNAME, C_COLUMNDATATYPE, C_OPERATOR, C_DIMCODE, C_COMMENT, C_TOOLTIP, UPDATE_DATE, DOWNLOAD_DATE, IMPORT_DATE, SOURCESYSTEM_CD, VALUETYPE_CD) from stdin with (format csv, header true, null 'NULL') ERROR: value too long for type character varying(50) CONTEXT: COPY mcrf_terms, line 42163, column c_basecode: ") Miscell. (Med.Supl.;Non-Drugs),,concept_cd,concept_dimension,concept_path,T,=,\i2b2\Medications\*N..." The offending line is: 5,\i2b2\Medications\*Not Available\DEVICES\Syringe with Needle (Disp)\Syringe with Needle (Disp) (Syringe 3cc/21Gx1-1/2\,Med:29443,N,LA ,0,) Miscell. (Med.Supl.;Non-Drugs)",,concept_cd,concept_dimension,concept_path,T,=,\i2b2\Medications\*Not Available\DEVICES\Syringe with Needle (Disp)\Syringe with Needle (Disp) (Syringe 3cc/21Gx1-1/2\,,Med:29443,2013-10-16 00:00:00.000,2013-10-16 00:00:00.000,2013-10-16 00:00:00.000,RDW, I'm working on encoding issues in the UTSW data. And it looks like the WISC data is tab-separated, not CSV... -- Dan ______________________________________________________________________ The contents of this message may contain private, protected and/or privileged information. If you received this message in error, you should destroy the e-mail message and any attachments or copies, and you are prohibited from retaining, distributing, disclosing or using any information contained within. Please contact the sender and advise of the erroneous delivery by return e-mail or telephone. Thank you for your cooperation.
