Grant,
The most efficient way that I can think about of the top of my head, is to
write a .Net / C++ / Java application that will run and buffer read the
file. I came across this once before and the only solution is to write a
parser in a language that is able to file seek through the file.
Perhaps you could import it into MS Access (by hand) then into Oracle. I am a
MS SQL user myself so I do not know what oracle tools there are available.
grant [EMAIL PROTECTED] 14/06/2007 9:53 am
Hi y'All
Here's the scenario: we have a csv file that contains 1.5 million rows and
is
Thanks Andrew. That's actually the path we're heading down (a custom java
app); I thought I'd post here to ensure we're not re-inventing the wheel...
Thanks!
On 14/06/07, Scott Thornton [EMAIL PROTECTED] wrote:
Perhaps you could import it into MS Access (by hand) then into Oracle. I
am a MS
Hi Grant,
we have used
http://www.emerle.net/programming/display.cfm/t/cfx_text2query with success
Cheers
Rob
_
From: cfaussie@googlegroups.com [mailto:[EMAIL PROTECTED] On Behalf
Of grant
Sent: Thursday, 14 June 2007 9:54 AM
To: cfaussie@googlegroups.com
Subject: [cfaussie] Large
On Thu, Jun 14, 2007 at 10:04:13AM +1000, Andrew Scott wrote:
Grant,
The most efficient way that I can think about of the top of my head, is to
write a .Net / C++ / Java application that will run and buffer read the
file. I came across this once before and the only solution is to write a
] Re: Large CSV's and server timeouts
Thanks Andrew. That's actually the path we're heading down (a custom java
app); I thought I'd post here to ensure we're not re-inventing the wheel...
Thanks!
On 14/06/07, Scott Thornton [EMAIL PROTECTED] wrote:
Perhaps you could import it into MS Access
@googlegroups.com [mailto:[EMAIL PROTECTED] On Behalf
Of Rob Wilson
Sent: Thursday, 14 June 2007 10:16 AM
To: cfaussie@googlegroups.com
Subject: [cfaussie] Re: Large CSV's and server timeouts
Hi Grant,
we have used
http://www.emerle.net/programming/display.cfm/t/cfx_text2query with success
can you use cfsetting and increase the timeout massively for that
request to stop the timeout?
If you are reading in a huge file, cffile is not good, as it can will
read in the entire file to memory (I understand this has been imporved
in CF8)
The is an example of using Java to read in a file
.
www.aegeon.com.au
Phone: +613 8676 4223
Mobile: 0404 998 273
-Original Message-
From: cfaussie@googlegroups.com [mailto:[EMAIL PROTECTED] On Behalf
Of [EMAIL PROTECTED]
Sent: Thursday, 14 June 2007 10:21 AM
To: cfaussie@googlegroups.com
Subject: [cfaussie] Re: Large CSV's and server timeouts
On Thu
June 2007 10:27 AM
To: cfaussie@googlegroups.com
Subject: [cfaussie] Re: Large CSV's and server timeouts
Grant,
That's why I thought I would add that there might be an application already
out there, I can't be 100% sure but I have seen apps that have been able to
manipulate CSV files (IIS
@googlegroups.com
Subject: [cfaussie] Re: Large CSV's and server timeouts
Not sure if this is an option.
But CF8 has new file functions that do incremental reads also.
Regards
Dale Fraser
http://dalefraser.blogspot.com
From: cfaussie@googlegroups.com [mailto:[EMAIL PROTECTED
: 0404 998 273
From: cfaussie@googlegroups.com [mailto:[EMAIL PROTECTED] On Behalf
Of Rob Wilson
Sent: Thursday, 14 June 2007 10:16 AM
To: cfaussie@googlegroups.com
Subject: [cfaussie] Re: Large CSV's and server timeouts
Hi Grant,
we have used
http://www.emerle.net/programming
that can work?
Steve
_
From: cfaussie@googlegroups.com [mailto:[EMAIL PROTECTED] On Behalf
Of Andrew Scott
Sent: Thursday, 14 June 2007 10:40 AM
To: cfaussie@googlegroups.com
Subject: [cfaussie] Re: Large CSV's and server timeouts
Actually wouldn't say they are new functions... Unless I
June 2007 10:27 AM
To: cfaussie@googlegroups.com
Subject: [cfaussie] Re: Large CSV's and server timeouts
Grant,
That's why I thought I would add that there might be an application already
out there, I can't be 100% sure but I have seen apps that have been able to
manipulate CSV files (IIS
] On Behalf
Of Rob Wilson
Sent: Thursday, 14 June 2007 10:51 AM
To: cfaussie@googlegroups.com
Subject: [cfaussie] Re: Large CSV's and server timeouts
Hi Andrew,
some of the smaller files we handle are in the order of 5 meg and larger
files have been in the order of 12 meg
...we have had major issues
[mailto:[EMAIL PROTECTED] On Behalf
Of David Harris
Sent: Thursday, 14 June 2007 10:54 AM
To: cfaussie
Subject: [cfaussie] Re: Large CSV's and server timeouts
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
cfaussie group
Mobile: 0404 998 273
-Original Message-
From: cfaussie@googlegroups.com [mailto:[EMAIL PROTECTED] On Behalf
Of David Harris
Sent: Thursday, 14 June 2007 10:54 AM
To: cfaussie
Subject: [cfaussie] Re: Large CSV's and server timeouts
.
www.aegeon.com.au
Phone: +613 8676 4223
Mobile: 0404 998 273
-Original Message-
From: cfaussie@googlegroups.com [mailto:[EMAIL PROTECTED] On Behalf
Of David Harris
Sent: Thursday, 14 June 2007 11:11 AM
To: cfaussie
Subject: [cfaussie] Re: Large CSV's and server timeouts
After a quick check, I am
Phone: +613 8676 4223
Mobile: 0404 998 273
-Original Message-
From: cfaussie@googlegroups.com [mailto:[EMAIL PROTECTED] On Behalf
Of David Harris
Sent: Thursday, 14 June 2007 11:11 AM
To: cfaussie
Subject: [cfaussie] Re: Large CSV's and server timeouts
After a quick check, I am
NDA peoples NDA...
-Original Message-
From: cfaussie@googlegroups.com [mailto:[EMAIL PROTECTED] On Behalf
Of Andrew Scott
Sent: Thursday, 14 June 2007 11:17 AM
To: cfaussie@googlegroups.com
Subject: [cfaussie] Re: Large CSV's and server timeouts
Doh,
I
11:17 AM
To: cfaussie@googlegroups.com
Subject: [cfaussie] Re: Large CSV's and server timeouts
Doh,
I thought you were talking about cfloop:-) Anyway, I am assuming that it
loads it into memory to process. But really thinking about it, that would be
a very hungry way of doing that new
@googlegroups.com [mailto:[EMAIL PROTECTED] On Behalf
Of David Harris
Sent: Thursday, 14 June 2007 11:24 AM
To: cfaussie
Subject: [cfaussie] Re: Large CSV's and server timeouts
hmmm
I guess I was reading that it doesn't load it in to memory, as the
last line in the docs say ColdFusion closes
11:40 AM
To: cfaussie@googlegroups.com
Subject: [cfaussie] Re: Large CSV's and server timeouts
NDA peoples NDA...
-Original Message-
From: cfaussie@googlegroups.com [mailto:[EMAIL PROTECTED] On Behalf
Of Andrew Scott
Sent: Thursday, 14 June 2007 11:17 AM
Hello,
grant wrote:
Here's the scenario: we have a csv file that contains 1.5 million rows
and is provided by an external vendor. we need to either (a) parse the
csv and insert into a DB, or (b) do lookups directly on the csv.
Coming in a bit late on this. The fact that there are CFXs that
.
www.aegeon.com.au
Phone: +613 8676 4223
Mobile: 0404 998 273
-Original Message-
From: cfaussie@googlegroups.com [mailto:[EMAIL PROTECTED] On Behalf
Of David Harris
Sent: Thursday, 14 June 2007 11:11 AM
To: cfaussie
Subject: [cfaussie] Re: Large CSV's and server timeouts
After a quick check
Subject: [cfaussie] Re: Large CSV's and server timeouts
Doh,
I thought you were talking about cfloop:-) Anyway, I am assuming that it
loads it into memory to process. But really thinking about it, that would be
a very hungry way of doing that new feature. So I hope I am wrong..
Andrew Scott
Subject: [cfaussie] Re: Large CSV's and server timeouts
NDA peoples NDA...
-Original Message-
From: cfaussie@googlegroups.com [mailto:[EMAIL PROTECTED] On Behalf
Of Andrew Scott
Sent: Thursday, 14 June 2007 11:17 AM
To: cfaussie@googlegroups.com
Subject
[mailto:[EMAIL PROTECTED] On Behalf
Of Andrew Scott
Sent: Thursday, 14 June 2007 11:17 AM
To: cfaussie@googlegroups.com
Subject: [cfaussie] Re: Large CSV's and server timeouts
Doh,
I thought you were talking about cfloop:-) Anyway, I am assuming that it
loads it into memory to process
28 matches
Mail list logo
