Re: JDBC ORACLE implementation !
keep in mind it's a matter of scale. If the application will take several years to grow beyond the current requirements, then you're fine. If the requirements are going to grow rapidly, then you may still want to use Oracle for your mined data. If you're dealing with only a couple thousand files and the growth rate is managable, then file system should last for a while. On the otherhand, if you know within the first two years the number of files will double each year, you may want to create an abstraction layer for yourself. This way you can store the summaries locally for now. then when the dataset size grows past a certain point, you can implement oracle and move the storage of the mined data in oracle. That should give you enough flexbility to grow without rewriting huge percentage of the app. peter Swapneel Dange [EMAIL PROTECTED] wrote:yeah i know what ur saying PENG , but i am still at a very basic stage in the FILESYSTEM implementation , thats why before i move any further, i was making sure as to whether considering the ROBUST structure and all that stuff into account will it be better if i implement the SQLPLUS or stick to the FILESYSTEM! comments awaited ! Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange - Do you Yahoo!? Yahoo! Tax Center - forms, calculators, tips, and more
RE: JDBC ORACLE implementation !
I never had problems with 8.1.6, 8.1.7 or 9.x. At least not with JDBC, I had problems with other things, but the JDBC was the easy part. --mikej -=- mike jackson [EMAIL PROTECTED] -Original Message- From: Chong Yu Meng [mailto:[EMAIL PROTECTED] Sent: Monday, February 24, 2003 3:07 PM To: Tomcat Users List Subject: Re: JDBC ORACLE implementation ! I think if you take Oracle installation, configuration and maintenance out of the picture, you definitely have a much more workable plan. I agree with Peter, in that designing the tables and application logic are going to be tough. I once wrote a servlet that processed CDR data from a Cisco switch, and I spent a lot of time getting the logic just right. If I understand you correctly, Swapneel, you need the database for storage only, correct ? Are you planning to use the Oracle text indexer, or are you implementing that yourself? One last thing : JDBC may take you a shorter time to learn than the 2 weeks I put down in an earlier email. On reflection, that is probably padding too much, but I recommend that you do not underestimate the time taken to get the JDBC connection going, especially for Oracle. I've had problems with 8.1.5 before and had to resort to DataDirect drivers. 8.1.7 seems to be ok, though. Regards, chong Peter Lin wrote: overall, using JDBC with Tomcat is the easy part. Deciding how to implement your tables and application logic will be the hard part. If your data is not normalized and doesn't need to be, then the first thing you should look into is statistical analysis of text. there are several well tested algo's for doing this type of processing. Unfortunately I don't know the name of the algo's. I worked on integrating personalization applications a couple years back relating to filtering news. If your needs aren't too complex, it shouldn't take too long to implement some form of statistical analysis. Using file system to store the entire text doesn't necessarily mean you can't store the text summaries in Oracle. Google for related topics and you should be to find some examples. If you're needs are more complex, you'll need to look into grammar based parsing, which is a slow process. Most of the comparison between grammar based and statistical parsing has shown that statistical is more effective. hope that helps. peter Swapneel Dange [EMAIL PROTECTED] wrote:hey peter, mike chong ! so if i stay with a small thing like SQLPLUS, the JDBC connectivity wont be a tough thing to do as compared to the ORACLE implementation. right ? because in last few days after consulting with some people in-house here, i am thinking over the OPTION of SQLPLUS. do commment on this ! Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: JDBC ORACLE implementation !
SQLPlus is part of oracle. You can't do sqlplus without oracle. --mikej -=- mike jackson [EMAIL PROTECTED] -Original Message- From: Swapneel Dange [mailto:[EMAIL PROTECTED] Sent: Monday, February 24, 2003 11:20 PM To: [EMAIL PROTECTED] Subject: Re: JDBC ORACLE implementation ! hey CHONG , PENG ! i think i have really given up the idea of putting up ORACKLE for my support. after all this discussion, i just think that there i sno urgent need for me to take up a HUMONGOUS taks of using ORACLE and i guess i will IMPLEMENT the SQLPLUS or the FILESYSTEM as my alternatives to the DATABASE application. but in the end i would really like to know as to between SQLPLUS or FILESYSTEM, which will be convinient for me to HANDLE string stripping , string comparison and all that stuff ! ( BTW, i have really started implementing the FILESYSTEM to a good level ) do commment about this ! Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange From: Chong Yu Meng [EMAIL PROTECTED] Reply-To: Tomcat Users List [EMAIL PROTECTED] To: Tomcat Users List [EMAIL PROTECTED] Subject: Re: JDBC ORACLE implementation ! Date: Tue, 25 Feb 2003 07:06:31 +0800 I think if you take Oracle installation, configuration and maintenance out of the picture, you definitely have a much more workable plan. I agree with Peter, in that designing the tables and application logic are going to be tough. I once wrote a servlet that processed CDR data from a Cisco switch, and I spent a lot of time getting the logic just right. If I understand you correctly, Swapneel, you need the database for storage only, correct ? Are you planning to use the Oracle text indexer, or are you implementing that yourself? One last thing : JDBC may take you a shorter time to learn than the 2 weeks I put down in an earlier email. On reflection, that is probably padding too much, but I recommend that you do not underestimate the time taken to get the JDBC connection going, especially for Oracle. I've had problems with 8.1.5 before and had to resort to DataDirect drivers. 8.1.7 seems to be ok, though. Regards, chong Peter Lin wrote: overall, using JDBC with Tomcat is the easy part. Deciding how to implement your tables and application logic will be the hard part. If your data is not normalized and doesn't need to be, then the first thing you should look into is statistical analysis of text. there are several well tested algo's for doing this type of processing. Unfortunately I don't know the name of the algo's. I worked on integrating personalization applications a couple years back relating to filtering news. If your needs aren't too complex, it shouldn't take too long to implement some form of statistical analysis. Using file system to store the entire text doesn't necessarily mean you can't store the text summaries in Oracle. Google for related topics and you should be to find some examples. If you're needs are more complex, you'll need to look into grammar based parsing, which is a slow process. Most of the comparison between grammar based and statistical parsing has shown that statistical is more effective. hope that helps. peter Swapneel Dange [EMAIL PROTECTED] wrote:hey peter, mike chong ! so if i stay with a small thing like SQLPLUS, the JDBC connectivity wont be a tough thing to do as compared to the ORACLE implementation. right ? because in last few days after consulting with some people in-house here, i am thinking over the OPTION of SQLPLUS. do commment on this ! Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] _ Add photos to your e-mail with MSN 8. Get 2 months FREE*. http://join.msn.com/?page=features/featuredemail - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: JDBC ORACLE implementation !
hey peter, mike chong ! so if i stay with a small thing like SQLPLUS, the JDBC connectivity wont be a tough thing to do as compared to the ORACLE implementation. right ? because in last few days after consulting with some people in-house here, i am thinking over the OPTION of SQLPLUS. do commment on this ! Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange From: Mike Jackson [EMAIL PROTECTED] Reply-To: Tomcat Users List [EMAIL PROTECTED] To: Tomcat Users List [EMAIL PROTECTED] Subject: RE: JDBC ORACLE implementation ! Date: Thu, 20 Feb 2003 09:14:52 -0800 clip senior DBA's help. when ever I need to tune oracle, I ask my Oracle DBA friends who have 8 yrs of exp for help. peter Me too, oracle can be a pain. If you stay small however all the tuning defaults should work for you. --mikej -=- mike jackson [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] _ Tired of spam? Get advanced junk mail protection with MSN 8. http://join.msn.com/?page=features/junkmail - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: JDBC ORACLE implementation !
overall, using JDBC with Tomcat is the easy part. Deciding how to implement your tables and application logic will be the hard part. If your data is not normalized and doesn't need to be, then the first thing you should look into is statistical analysis of text. there are several well tested algo's for doing this type of processing. Unfortunately I don't know the name of the algo's. I worked on integrating personalization applications a couple years back relating to filtering news. If your needs aren't too complex, it shouldn't take too long to implement some form of statistical analysis. Using file system to store the entire text doesn't necessarily mean you can't store the text summaries in Oracle. Google for related topics and you should be to find some examples. If you're needs are more complex, you'll need to look into grammar based parsing, which is a slow process. Most of the comparison between grammar based and statistical parsing has shown that statistical is more effective. hope that helps. peter Swapneel Dange [EMAIL PROTECTED] wrote:hey peter, mike chong ! so if i stay with a small thing like SQLPLUS, the JDBC connectivity wont be a tough thing to do as compared to the ORACLE implementation. right ? because in last few days after consulting with some people in-house here, i am thinking over the OPTION of SQLPLUS. do commment on this ! Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange From: Mike Jackson Reply-To: Tomcat Users List To: Tomcat Users List Subject: RE: JDBC ORACLE implementation ! Date: Thu, 20 Feb 2003 09:14:52 -0800 senior DBA's help. when ever I need to tune oracle, I ask my Oracle DBA friends who have 8 yrs of exp for help. peter Me too, oracle can be a pain. If you stay small however all the tuning defaults should work for you. --mikej -=- mike jackson [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] _ Tired of spam? Get advanced junk mail protection with MSN 8. http://join.msn.com/?page=features/junkmail - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Do you Yahoo!? Yahoo! Tax Center - forms, calculators, tips, and more
Re: JDBC ORACLE implementation !
I think if you take Oracle installation, configuration and maintenance out of the picture, you definitely have a much more workable plan. I agree with Peter, in that designing the tables and application logic are going to be tough. I once wrote a servlet that processed CDR data from a Cisco switch, and I spent a lot of time getting the logic just right. If I understand you correctly, Swapneel, you need the database for storage only, correct ? Are you planning to use the Oracle text indexer, or are you implementing that yourself? One last thing : JDBC may take you a shorter time to learn than the 2 weeks I put down in an earlier email. On reflection, that is probably padding too much, but I recommend that you do not underestimate the time taken to get the JDBC connection going, especially for Oracle. I've had problems with 8.1.5 before and had to resort to DataDirect drivers. 8.1.7 seems to be ok, though. Regards, chong Peter Lin wrote: overall, using JDBC with Tomcat is the easy part. Deciding how to implement your tables and application logic will be the hard part. If your data is not normalized and doesn't need to be, then the first thing you should look into is statistical analysis of text. there are several well tested algo's for doing this type of processing. Unfortunately I don't know the name of the algo's. I worked on integrating personalization applications a couple years back relating to filtering news. If your needs aren't too complex, it shouldn't take too long to implement some form of statistical analysis. Using file system to store the entire text doesn't necessarily mean you can't store the text summaries in Oracle. Google for related topics and you should be to find some examples. If you're needs are more complex, you'll need to look into grammar based parsing, which is a slow process. Most of the comparison between grammar based and statistical parsing has shown that statistical is more effective. hope that helps. peter Swapneel Dange [EMAIL PROTECTED] wrote:hey peter, mike chong ! so if i stay with a small thing like SQLPLUS, the JDBC connectivity wont be a tough thing to do as compared to the ORACLE implementation. right ? because in last few days after consulting with some people in-house here, i am thinking over the OPTION of SQLPLUS. do commment on this ! Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: JDBC ORACLE implementation !
hey CHONG , PENG ! i think i have really given up the idea of putting up ORACKLE for my support. after all this discussion, i just think that there i sno urgent need for me to take up a HUMONGOUS taks of using ORACLE and i guess i will IMPLEMENT the SQLPLUS or the FILESYSTEM as my alternatives to the DATABASE application. but in the end i would really like to know as to between SQLPLUS or FILESYSTEM, which will be convinient for me to HANDLE string stripping , string comparison and all that stuff ! ( BTW, i have really started implementing the FILESYSTEM to a good level ) do commment about this ! Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange From: Chong Yu Meng [EMAIL PROTECTED] Reply-To: Tomcat Users List [EMAIL PROTECTED] To: Tomcat Users List [EMAIL PROTECTED] Subject: Re: JDBC ORACLE implementation ! Date: Tue, 25 Feb 2003 07:06:31 +0800 I think if you take Oracle installation, configuration and maintenance out of the picture, you definitely have a much more workable plan. I agree with Peter, in that designing the tables and application logic are going to be tough. I once wrote a servlet that processed CDR data from a Cisco switch, and I spent a lot of time getting the logic just right. If I understand you correctly, Swapneel, you need the database for storage only, correct ? Are you planning to use the Oracle text indexer, or are you implementing that yourself? One last thing : JDBC may take you a shorter time to learn than the 2 weeks I put down in an earlier email. On reflection, that is probably padding too much, but I recommend that you do not underestimate the time taken to get the JDBC connection going, especially for Oracle. I've had problems with 8.1.5 before and had to resort to DataDirect drivers. 8.1.7 seems to be ok, though. Regards, chong Peter Lin wrote: overall, using JDBC with Tomcat is the easy part. Deciding how to implement your tables and application logic will be the hard part. If your data is not normalized and doesn't need to be, then the first thing you should look into is statistical analysis of text. there are several well tested algo's for doing this type of processing. Unfortunately I don't know the name of the algo's. I worked on integrating personalization applications a couple years back relating to filtering news. If your needs aren't too complex, it shouldn't take too long to implement some form of statistical analysis. Using file system to store the entire text doesn't necessarily mean you can't store the text summaries in Oracle. Google for related topics and you should be to find some examples. If you're needs are more complex, you'll need to look into grammar based parsing, which is a slow process. Most of the comparison between grammar based and statistical parsing has shown that statistical is more effective. hope that helps. peter Swapneel Dange [EMAIL PROTECTED] wrote:hey peter, mike chong ! so if i stay with a small thing like SQLPLUS, the JDBC connectivity wont be a tough thing to do as compared to the ORACLE implementation. right ? because in last few days after consulting with some people in-house here, i am thinking over the OPTION of SQLPLUS. do commment on this ! Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] _ Add photos to your e-mail with MSN 8. Get 2 months FREE*. http://join.msn.com/?page=features/featuredemail - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: JDBC ORACLE implementation !
If you have already started on a filesystem and if it does what you want it to do at a reasonable speed then stick with it :) . Swapneel Dange wrote: hey CHONG , PENG ! i think i have really given up the idea of putting up ORACKLE for my support. after all this discussion, i just think that there i sno urgent need for me to take up a HUMONGOUS taks of using ORACLE and i guess i will IMPLEMENT the SQLPLUS or the FILESYSTEM as my alternatives to the DATABASE application. but in the end i would really like to know as to between SQLPLUS or FILESYSTEM, which will be convinient for me to HANDLE string stripping , string comparison and all that stuff ! ( BTW, i have really started implementing the FILESYSTEM to a good level ) do commment about this ! Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange From: Chong Yu Meng [EMAIL PROTECTED] Reply-To: Tomcat Users List [EMAIL PROTECTED] To: Tomcat Users List [EMAIL PROTECTED] Subject: Re: JDBC ORACLE implementation ! Date: Tue, 25 Feb 2003 07:06:31 +0800 I think if you take Oracle installation, configuration and maintenance out of the picture, you definitely have a much more workable plan. I agree with Peter, in that designing the tables and application logic are going to be tough. I once wrote a servlet that processed CDR data from a Cisco switch, and I spent a lot of time getting the logic just right. If I understand you correctly, Swapneel, you need the database for storage only, correct ? Are you planning to use the Oracle text indexer, or are you implementing that yourself? One last thing : JDBC may take you a shorter time to learn than the 2 weeks I put down in an earlier email. On reflection, that is probably padding too much, but I recommend that you do not underestimate the time taken to get the JDBC connection going, especially for Oracle. I've had problems with 8.1.5 before and had to resort to DataDirect drivers. 8.1.7 seems to be ok, though. Regards, chong Peter Lin wrote: overall, using JDBC with Tomcat is the easy part. Deciding how to implement your tables and application logic will be the hard part. If your data is not normalized and doesn't need to be, then the first thing you should look into is statistical analysis of text. there are several well tested algo's for doing this type of processing. Unfortunately I don't know the name of the algo's. I worked on integrating personalization applications a couple years back relating to filtering news. If your needs aren't too complex, it shouldn't take too long to implement some form of statistical analysis. Using file system to store the entire text doesn't necessarily mean you can't store the text summaries in Oracle. Google for related topics and you should be to find some examples. If you're needs are more complex, you'll need to look into grammar based parsing, which is a slow process. Most of the comparison between grammar based and statistical parsing has shown that statistical is more effective. hope that helps. peter Swapneel Dange [EMAIL PROTECTED] wrote:hey peter, mike chong ! so if i stay with a small thing like SQLPLUS, the JDBC connectivity wont be a tough thing to do as compared to the ORACLE implementation. right ? because in last few days after consulting with some people in-house here, i am thinking over the OPTION of SQLPLUS. do commment on this ! Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] _ Add photos to your e-mail with MSN 8. Get 2 months FREE*. http://join.msn.com/?page=features/featuredemail - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: JDBC ORACLE implementation !
yeah i know what ur saying PENG , but i am still at a very basic stage in the FILESYSTEM implementation , thats why before i move any further, i was making sure as to whether considering the ROBUST structure and all that stuff into account will it be better if i implement the SQLPLUS or stick to the FILESYSTEM! comments awaited ! Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange From: Peng Tuck Kwok [EMAIL PROTECTED] Reply-To: Tomcat Users List [EMAIL PROTECTED] To: Tomcat Users List [EMAIL PROTECTED] Subject: Re: JDBC ORACLE implementation ! Date: Tue, 25 Feb 2003 15:31:22 +0800 If you have already started on a filesystem and if it does what you want it to do at a reasonable speed then stick with it :) . Swapneel Dange wrote: hey CHONG , PENG ! i think i have really given up the idea of putting up ORACKLE for my support. after all this discussion, i just think that there i sno urgent need for me to take up a HUMONGOUS taks of using ORACLE and i guess i will IMPLEMENT the SQLPLUS or the FILESYSTEM as my alternatives to the DATABASE application. but in the end i would really like to know as to between SQLPLUS or FILESYSTEM, which will be convinient for me to HANDLE string stripping , string comparison and all that stuff ! ( BTW, i have really started implementing the FILESYSTEM to a good level ) do commment about this ! Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange From: Chong Yu Meng [EMAIL PROTECTED] Reply-To: Tomcat Users List [EMAIL PROTECTED] To: Tomcat Users List [EMAIL PROTECTED] Subject: Re: JDBC ORACLE implementation ! Date: Tue, 25 Feb 2003 07:06:31 +0800 I think if you take Oracle installation, configuration and maintenance out of the picture, you definitely have a much more workable plan. I agree with Peter, in that designing the tables and application logic are going to be tough. I once wrote a servlet that processed CDR data from a Cisco switch, and I spent a lot of time getting the logic just right. If I understand you correctly, Swapneel, you need the database for storage only, correct ? Are you planning to use the Oracle text indexer, or are you implementing that yourself? One last thing : JDBC may take you a shorter time to learn than the 2 weeks I put down in an earlier email. On reflection, that is probably padding too much, but I recommend that you do not underestimate the time taken to get the JDBC connection going, especially for Oracle. I've had problems with 8.1.5 before and had to resort to DataDirect drivers. 8.1.7 seems to be ok, though. Regards, chong Peter Lin wrote: overall, using JDBC with Tomcat is the easy part. Deciding how to implement your tables and application logic will be the hard part. If your data is not normalized and doesn't need to be, then the first thing you should look into is statistical analysis of text. there are several well tested algo's for doing this type of processing. Unfortunately I don't know the name of the algo's. I worked on integrating personalization applications a couple years back relating to filtering news. If your needs aren't too complex, it shouldn't take too long to implement some form of statistical analysis. Using file system to store the entire text doesn't necessarily mean you can't store the text summaries in Oracle. Google for related topics and you should be to find some examples. If you're needs are more complex, you'll need to look into grammar based parsing, which is a slow process. Most of the comparison between grammar based and statistical parsing has shown that statistical is more effective. hope that helps. peter Swapneel Dange [EMAIL PROTECTED] wrote:hey peter, mike chong ! so if i stay with a small thing like SQLPLUS, the JDBC connectivity wont be a tough thing to do as compared to the ORACLE implementation. right ? because in last few days after consulting with some people in-house here, i am thinking over the OPTION of SQLPLUS. do commment on this ! Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] _ Add photos to your e-mail with MSN 8. Get 2 months FREE*. http://join.msn.com/?page=features/featuredemail - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] _ STOP MORE SPAM with the new MSN 8 and get 2 months FREE* http://join.msn.com/?page=features/junkmail - To unsubscribe, e
Re: JDBC ORACLE implementation !
hey CHONG ~ i think that i have got my all the answers here in the mail u send me. gosh , i never knew all these details are lying behind my implementation thought of a good database. i really went through all the details u gave me here. anyways i am thinking now more seriously about the FILE SYSTEM and its implementation. anyway, if i get more doubts on this topic, i will put those questions here, thanx all ! Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange From: Chong Yu Meng [EMAIL PROTECTED] Reply-To: Tomcat Users List [EMAIL PROTECTED] To: Tomcat Users List [EMAIL PROTECTED] Subject: Re: JDBC ORACLE implementation ! Date: Thu, 20 Feb 2003 17:11:35 +0800 Hi Swapneel ! I need you to clarify something : are you processing the files first, then putting them into the database, or are you putting all the raw data inside the database and doing SQL operations to extract the data you want ? I am assuming that you want to do the second option, am I right ? I need to make a few other things clear : 1. If you are committing 1MB of data into the database per operation, that would take a looong time, and I'm not sure if that is the most efficient, or even the easiest way to do what you are trying to do. Why I say that is, Oracle is fine when you break up the data and commit, perhaps line by line, to the database. Not very fast or efficient, but still manageable. My experience of Oracle, though limited, has not exactly given me much confidence in its ability to handle huge data in individual fields. Stuff the fields with too much data, and try to work with them, and you will realize the necessity to tune Oracle. I'm not an expert on Oracle, nor do I care to be, so I won't try this. 2. String manipulation with Java 1.4 is possible and quite easy, now that it supports regular expressions. If, like me, you'd like to do things the hard way, you can use the string handling functions in Java to do the processing for you. It's actually a lot easier than it sounds. I refer to O'Reilly's Java Cookbook for examples on this. 3. The least risky way (and some would say, the lazy way) is to just improve on the existing code -- use the FILE SYSTEM solution that already exists and extend the functionality, fix bugs, or improve the GUI. It's not because you may be re-inventing the wheel, as some may say, if you choose to implement it in a database. I am assuming that you don't have a lot of time, and there is no in-house expertise you can refer to (i.e. programmer friends where you live, that you can ask), and, as you say, you are coming into this for the first time. As a rough indicator (and I invite others to correct me), it takes : - 1 month to understand Oracle - 2 weeks to get JDBC working the way you want - 2 weeks or more to work out the business logic - 1 week to figure out how to draw graphs - 2 weeks to fix bugs - 1 week to understand the deployment environment - 1-2 weeks to fix problems related to the deployment environment - Total : 14 weeks That's more than 3 months, and assumes that you devote yourself 100% to the project. A filesystem solution however takes : - 1 week to learn about file i/o - 1 week to learn how to do string manipulation - 1 week to learn graphs - 2 weeks to fix bugs - 2-3 weeks to deploy - Total : 8 weeks You might want to just go with a simple file system solution for now, and slowly work out how to do the rest later, when you are not hurried. It really is better. Regards, pascal chong Swapneel Dange wrote: hey CHONG ! yepp, i think i am hurrying too fast, and i have the FILE SYSTEM with me but the thing is that i need a sytem is which i can atlesat shoud be able to do something like this -- 1) strip of the files of the un-necessary content. 2) then use the data for searching patterns. 3) apply the patterns to the unsual causes. 4) then use the data to relate to GRAPHS right now i am using a file system in which i cant perform the string matching and other operatoins. moreover all thsi has to be implemented on teh WINDOWS NT or XP. so i am sure as to how u implement somethign like shell scripting in windows. anyways thats more or less knowledge i ahev about al lthis. will be great to hear from u about all this ! thanx ! Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] _ Add photos to your e-mail with MSN 8. Get 2 months FREE*. http://join.msn.com/?page=features/featuredemail - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: JDBC ORACLE implementation !
Hi Swapneel ! I need you to clarify something : are you processing the files first, then putting them into the database, or are you putting all the raw data inside the database and doing SQL operations to extract the data you want ? I am assuming that you want to do the second option, am I right ? I need to make a few other things clear : 1. If you are committing 1MB of data into the database per operation, that would take a looong time, and I'm not sure if that is the most efficient, or even the easiest way to do what you are trying to do. Why I say that is, Oracle is fine when you break up the data and commit, perhaps line by line, to the database. Not very fast or efficient, but still manageable. My experience of Oracle, though limited, has not exactly given me much confidence in its ability to handle huge data in individual fields. Stuff the fields with too much data, and try to work with them, and you will realize the necessity to tune Oracle. I'm not an expert on Oracle, nor do I care to be, so I won't try this. 2. String manipulation with Java 1.4 is possible and quite easy, now that it supports regular expressions. If, like me, you'd like to do things the hard way, you can use the string handling functions in Java to do the processing for you. It's actually a lot easier than it sounds. I refer to O'Reilly's Java Cookbook for examples on this. 3. The least risky way (and some would say, the lazy way) is to just improve on the existing code -- use the FILE SYSTEM solution that already exists and extend the functionality, fix bugs, or improve the GUI. It's not because you may be re-inventing the wheel, as some may say, if you choose to implement it in a database. I am assuming that you don't have a lot of time, and there is no in-house expertise you can refer to (i.e. programmer friends where you live, that you can ask), and, as you say, you are coming into this for the first time. As a rough indicator (and I invite others to correct me), it takes : - 1 month to understand Oracle - 2 weeks to get JDBC working the way you want - 2 weeks or more to work out the business logic - 1 week to figure out how to draw graphs - 2 weeks to fix bugs - 1 week to understand the deployment environment - 1-2 weeks to fix problems related to the deployment environment - Total : 14 weeks That's more than 3 months, and assumes that you devote yourself 100% to the project. A filesystem solution however takes : - 1 week to learn about file i/o - 1 week to learn how to do string manipulation - 1 week to learn graphs - 2 weeks to fix bugs - 2-3 weeks to deploy - Total : 8 weeks You might want to just go with a simple file system solution for now, and slowly work out how to do the rest later, when you are not hurried. It really is better. Regards, pascal chong Swapneel Dange wrote: hey CHONG ! yepp, i think i am hurrying too fast, and i have the FILE SYSTEM with me but the thing is that i need a sytem is which i can atlesat shoud be able to do something like this -- 1) strip of the files of the un-necessary content. 2) then use the data for searching patterns. 3) apply the patterns to the unsual causes. 4) then use the data to relate to GRAPHS right now i am using a file system in which i cant perform the string matching and other operatoins. moreover all thsi has to be implemented on teh WINDOWS NT or XP. so i am sure as to how u implement somethign like shell scripting in windows. anyways thats more or less knowledge i ahev about al lthis. will be great to hear from u about all this ! thanx ! Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: JDBC ORACLE implementation !
I think I have a better idea of what you are trying to do. If you goal is to data mine, your best option may not be Oracle. That might sound counter-intuitive, but here are some reasons. Oracle will handle large multi-meg files just fine. In fact GIS (geographical information systems) use oracle. though databases that have extreme storage needs like GIS tend to use specialized databases and implement specialized search algo's. A simple example of this might be, find all routes from point A to B, which use side streets that are not one way. If you had to do this kind of query in oracle it would mean doing some very complex joins and would most likely require additional processing outside of oracle in an app server. when you talk about data mining, there's a wide variety. Not everyone uses data mining to mean the same thing. If you're talking about KDD (knowledge discover in databases), then there's two routes: buy a data mining package, or write your own. Again, if your needs are fairly simple like statistical analysis of the text, then you're better off storing those files locally and have some other deamon process mine the data in the back ground. For example, you wouldn't want to write a stored procedure to mind the data. You could, but that might cause you to pull your hair out. Again, if you're dealing with essays or papers, you're better off processing those in the background and storing those summaries in Oracle. Storing the entire text in oracle won't make your life easier. A common practice in AI for text handling is statistical analysis. The basic idea is, filter out all the words that aren't important, like verbs, adverbs and so on. Count the frequency of the nouns and store those summaries in the database. I hope that helps. you're going to have to do more research on this to get a good understanding of mining techniques. peter Swapneel Dange [EMAIL PROTECTED] wrote:hey peter ! ur right, that there is no transaction involved in this process here. only thing i will be doing is receiving files on the server using the servlets. now may be it was too much thinking on my part to say that i will use ORAVCLE. what do u say that for atleast 7200 files a day of size max 1MB, shouldnt i use ORACLE ? should i try some other options and if YES then what kind of database can i implement. right now i have the FILE SYSTEM implemented here. but i think it has limited my ability to do pattern searching and data mining, thats why i was trying to move to something more stable and robust such as a database which can support TOUGHER queries. awaiting reaply ! Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange From: Peter Lin Reply-To: Tomcat Users List To: Tomcat Users List Subject: Re: JDBC ORACLE implementation ! Date: Wed, 19 Feb 2003 18:09:36 -0800 (PST) First off, you probably should be use Oracle enterprise edition, unless you're on a box with less than 128meg of memory. Oracle personal edition for 8i and 9i is really designed for simple uses. The scenario you've described will probably mean storing the text as a clob or in multiple columns. keep in mind if you store it as a clob, it limits your ability to search performance. breaking the text into columns will allow you to index the content easier. If query time is important, you may want to generate summaries of the text and use that for your indexes instead. as far as connecting to oracle, it's fairly straight forward. databases are handy, but take care with how you implement the application. If you don't need to index the content, or do not need transaction capabilities, you're better off using file system to store the text. RDBMS are designed specifically to handle relational data. If your data isn't relational, using Oracle is a bit over kill. Using the right tool will make your life easier in the long run. peter _ Help STOP SPAM with the new MSN 8 and get 2 months FREE* http://join.msn.com/?page=features/junkmail - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Do you Yahoo!? Yahoo! Tax Center - forms, calculators, tips, and more
Re: JDBC ORACLE implementation !
Chong Yu Meng wrote: As a rough indicator (and I invite others to correct me), it takes : - 1 month to understand Oracle - 2 weeks to get JDBC working the way you want Is this indicator specific to Oracle? I set up the MySQL JDBC driver and had working queries in a matter of hours. (I am not using complex database abstractions, simply submitting SQL queries with JDBC classes.) - 1 week to learn about file i/o I'm not sure about the Java tutorial on this topic, but if you read the File I/O chapter of Bruce Eckel's Thinking in Java (which you can download at http://mindview.net/ ), you can have File I/O basics down in a couple of hours. (My thoughts) Erik - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: JDBC ORACLE implementation !
Database are a lot of fun (well to me they are), but they do a lot more than just queries. Learning every aspect of Oracle is fairly complex and takes years of experience. there's a reason why Oracle DBA's make big bucks. Most people will never use the high end features of Oracle. Things like realtime backup using multiple databases, shared memory, shared storage and various Oracle plug-ins. anyone that has worked with large complex schema's that are high normalized knows how hard it is to manage hundreds of gigs of data. Say you maintain a database for a large retail company and there's terabytes of data about everything from inventory to customer purchasing habits. MySql is great for light weight web applications, but it doesn't begin to touch on hardcore database driven applications. Another example, say you run a Tv listing site, and you get daily updates to 50 tables. The data for every market in the US per week is 500megs of data. Now obviously there are some things you want to update and others you don't. Managing large regular updates like these can be simple or very complicated. Take it to the extreme where you have realtime data. How do you handle a costant stream of data that's several megs per second? tuning JDBC in Oracle is considerably harder than mysql because oracle is designed to run on small and large systems. If you have a E1 with 24 CPU's and 1 terabyte of memory, you have to tune the number of pool per process and so on. I've only had a little hardcore Oracle experience, so I only know how hard it is. I couldn't actually manage large Oracle installations without a senior DBA's help. when ever I need to tune oracle, I ask my Oracle DBA friends who have 8 yrs of exp for help. peter Erik Price [EMAIL PROTECTED] wrote: Chong Yu Meng wrote: As a rough indicator (and I invite others to correct me), it takes : - 1 month to understand Oracle - 2 weeks to get JDBC working the way you want Is this indicator specific to Oracle? I set up the MySQL JDBC driver and had working queries in a matter of hours. (I am not using complex database abstractions, simply submitting SQL queries with JDBC classes.) - 1 week to learn about file i/o I'm not sure about the Java tutorial on this topic, but if you read the File I/O chapter of Bruce Eckel's Thinking in Java (which you can download at http://mindview.net/ ), you can have File I/O basics down in a couple of hours. (My thoughts) Erik - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Do you Yahoo!? Yahoo! Tax Center - forms, calculators, tips, and more
RE: JDBC ORACLE implementation !
1. If you are committing 1MB of data into the database per operation, that would take a looong time, and I'm not sure if that is the most efficient, or even the easiest way to do what you are trying to do. Why I say that is, Oracle is fine when you break up the data and commit, perhaps line by line, to the database. Not very fast or efficient, but still manageable. My experience of Oracle, though limited, has not exactly given me much confidence in its ability to handle huge data in individual fields. Stuff the fields with too much data, and try to work with them, and you will realize the necessity to tune Oracle. I'm not an expert on Oracle, nor do I care to be, so I won't try this. I put huge amounts of data into oracle using transactions, it is fast and quite efficent. The largest lob that I currently have tried is about 43 megs. Breaking the text up by line would make the sql quite a bit more complicated so I'd stay away from that. Also, if you setup the intermedia text index you can do keyword searches from sql against the text data (assuming it's in a clob). Setting up the index isn't too hard, but if you want it in a particular tablespace (other than the user's default tablespace) you'll have to change your default tablespace to the one you want the index in prior to creating the index. After the index is created you can change back to the old one. Also the text index doesn't auto-update, so you'll have to issue a dml command to trigger the index update. --mikej -=- mike jackson [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: JDBC ORACLE implementation !
1 month to understand oracle? I'd say 1 day to get it installed (don't let it create the database using the copy a database method), and another day or two to get used to the oracle oddities. The graphical tools will let you do most things without to much problem, if you were using sqlplus to to everything then maybe a month might make sense. And JDBC 2 weeks? Not even, perhaps a day, but that's on the outside (I'm guessing you just mean connecting to the database). --mikej -=- mike jackson [EMAIL PROTECTED] -Original Message- From: Erik Price [mailto:[EMAIL PROTECTED]] Sent: Thursday, February 20, 2003 8:01 AM To: Tomcat Users List Subject: Re: JDBC ORACLE implementation ! Chong Yu Meng wrote: As a rough indicator (and I invite others to correct me), it takes : - 1 month to understand Oracle - 2 weeks to get JDBC working the way you want Is this indicator specific to Oracle? I set up the MySQL JDBC driver and had working queries in a matter of hours. (I am not using complex database abstractions, simply submitting SQL queries with JDBC classes.) - 1 week to learn about file i/o I'm not sure about the Java tutorial on this topic, but if you read the File I/O chapter of Bruce Eckel's Thinking in Java (which you can download at http://mindview.net/ ), you can have File I/O basics down in a couple of hours. (My thoughts) Erik - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: JDBC ORACLE implementation !
clip senior DBA's help. when ever I need to tune oracle, I ask my Oracle DBA friends who have 8 yrs of exp for help. peter Me too, oracle can be a pain. If you stay small however all the tuning defaults should work for you. --mikej -=- mike jackson [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: JDBC ORACLE implementation !
The implementation is based on which driver you use, which mostly depends on which version of Oracle you are running. What version numbers are you using for Oracle Tomcat - Original Message - From: Swapneel Dange [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Wednesday, February 19, 2003 9:47 AM Subject: JDBC ORACLE implementation ! hi there ~ i am curious to know as to how can u implement ORACLE database in the TOMCAT. and can somebody tell me as to where i can read the DOCUMENTATION for the implementation of the JDBC connectivity under TOMCAT. Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange _ MSN 8 with e-mail virus protection service: 2 months FREE* http://join.msn.com/?page=features/virus - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: JDBC ORACLE implementation !
Hi Swapneel ! If you absolutely MUST use Oracle with Tomcat, use the DataDirect JDBC drivers, not the JDBC drivers bundled with Oracle. I am assuming (since you did not mention the version number of Oracle) that you are pretty new to the Oracle database product. You should then go for the Sequelink JDBC product (http://www.datadirect-technologies.com/products/sequelink/slindex.asp). It will help you to get started a lot quicker. Regards, pascal chong Swapneel Dange wrote: hi there ~ i am curious to know as to how can u implement ORACLE database in the TOMCAT. and can somebody tell me as to where i can read the DOCUMENTATION for the implementation of the JDBC connectivity under TOMCAT. Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: JDBC ORACLE implementation !
hi ! frankly speaking, i am very new to the database concepts. i am trying to use the Oracle Database Personal Edition for my use here. But i can tell about the version of the TOMCAT, i am using , its 3.3.1a. if u could tell me in detail as to what ORACLE product i should use will be great. By the way i am using this database for the users to send data from 3 different states on the webserver i am running here. so that when that data which are huge files coem over here, i can massage them and get the particular details i want for my use. thanx ! Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange From: Andoni [EMAIL PROTECTED] Reply-To: Tomcat Users List [EMAIL PROTECTED] To: Tomcat Users List [EMAIL PROTECTED] Subject: Re: JDBC ORACLE implementation ! Date: Wed, 19 Feb 2003 10:09:16 - The implementation is based on which driver you use, which mostly depends on which version of Oracle you are running. What version numbers are you using for Oracle Tomcat - Original Message - From: Swapneel Dange [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Wednesday, February 19, 2003 9:47 AM Subject: JDBC ORACLE implementation ! hi there ~ i am curious to know as to how can u implement ORACLE database in the TOMCAT. and can somebody tell me as to where i can read the DOCUMENTATION for the implementation of the JDBC connectivity under TOMCAT. Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange _ MSN 8 with e-mail virus protection service: 2 months FREE* http://join.msn.com/?page=features/virus - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] _ Protect your PC - get McAfee.com VirusScan Online http://clinic.mcafee.com/clinic/ibuy/campaign.asp?cid=3963 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: JDBC ORACLE implementation !
So, you don't really need tomcat per say to talk to oracle, you need your servlets/jsps to be able to talk to oracle. What type of data are you working with and how big is it? That's really the question to ask that'll tell you what version of oracle to use. If you using small data (standard types or lobs) then any version should work ok. But you'll want to be in the READ_COMMITED transaction isolation mode (default). If you're working with large lobs (25 megs) you'll probably want to be on 9.2.0.1.0 or better (or a patched 9.1, but not an 8.x version I think). And you'll want to make sure you're using the jdbc driver supplied by the database, or one from a newer version, the older ones with newer databases tend to cause problems. Personal/Standard/Enterprise may not matter. --mikej -=- mike jackson [EMAIL PROTECTED] -Original Message- From: Swapneel Dange [mailto:[EMAIL PROTECTED]] Sent: Wednesday, February 19, 2003 4:11 PM To: [EMAIL PROTECTED] Subject: Re: JDBC ORACLE implementation ! hi ! frankly speaking, i am very new to the database concepts. i am trying to use the Oracle Database Personal Edition for my use here. But i can tell about the version of the TOMCAT, i am using , its 3.3.1a. if u could tell me in detail as to what ORACLE product i should use will be great. By the way i am using this database for the users to send data from 3 different states on the webserver i am running here. so that when that data which are huge files coem over here, i can massage them and get the particular details i want for my use. thanx ! Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange From: Andoni [EMAIL PROTECTED] Reply-To: Tomcat Users List [EMAIL PROTECTED] To: Tomcat Users List [EMAIL PROTECTED] Subject: Re: JDBC ORACLE implementation ! Date: Wed, 19 Feb 2003 10:09:16 - The implementation is based on which driver you use, which mostly depends on which version of Oracle you are running. What version numbers are you using for Oracle Tomcat - Original Message - From: Swapneel Dange [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Wednesday, February 19, 2003 9:47 AM Subject: JDBC ORACLE implementation ! hi there ~ i am curious to know as to how can u implement ORACLE database in the TOMCAT. and can somebody tell me as to where i can read the DOCUMENTATION for the implementation of the JDBC connectivity under TOMCAT. Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange _ MSN 8 with e-mail virus protection service: 2 months FREE* http://join.msn.com/?page=features/virus - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] _ Protect your PC - get McAfee.com VirusScan Online http://clinic.mcafee.com/clinic/ibuy/campaign.asp?cid=3963 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: JDBC ORACLE implementation !
hi mike ! the files i am using are quite small in size like 200-300 KB but sometimes i get files of the size 1MB or so. but the rate at which i get these files is huge, i get a file every 12 second so that makes 7200 files a day and all are useful files. i will then massgae the files and strip them of the un-necessary details. and then look out for keywords in the file. draw some conclusions based on that and then implement the solutions. moreoevr i could not understand as to what u wanted to convey through the sentense, So, you don't really need tomcat per say to talk to oracle, you need yourservlets/jsps to be able to talk to oracle. . i am a NOVICE, thats the reason that i didnt understans the meaning of the sentense there. anyways i can user the other JDBC drivers which are not supplied by ORACLE. some of the drivers suggested to me here are the ones by DATADIRECT. awaiting reply ! Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange From: Mike Jackson [EMAIL PROTECTED] Reply-To: Tomcat Users List [EMAIL PROTECTED] To: Tomcat Users List [EMAIL PROTECTED] Subject: RE: JDBC ORACLE implementation ! Date: Wed, 19 Feb 2003 16:24:25 -0800 So, you don't really need tomcat per say to talk to oracle, you need your servlets/jsps to be able to talk to oracle. What type of data are you working with and how big is it? That's really the question to ask that'll tell you what version of oracle to use. If you using small data (standard types or lobs) then any version should work ok. But you'll want to be in the READ_COMMITED transaction isolation mode (default). If you're working with large lobs (25 megs) you'll probably want to be on 9.2.0.1.0 or better (or a patched 9.1, but not an 8.x version I think). And you'll want to make sure you're using the jdbc driver supplied by the database, or one from a newer version, the older ones with newer databases tend to cause problems. Personal/Standard/Enterprise may not matter. --mikej -=- mike jackson [EMAIL PROTECTED] -Original Message- From: Swapneel Dange [mailto:[EMAIL PROTECTED]] Sent: Wednesday, February 19, 2003 4:11 PM To: [EMAIL PROTECTED] Subject: Re: JDBC ORACLE implementation ! hi ! frankly speaking, i am very new to the database concepts. i am trying to use the Oracle Database Personal Edition for my use here. But i can tell about the version of the TOMCAT, i am using , its 3.3.1a. if u could tell me in detail as to what ORACLE product i should use will be great. By the way i am using this database for the users to send data from 3 different states on the webserver i am running here. so that when that data which are huge files coem over here, i can massage them and get the particular details i want for my use. thanx ! Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange From: Andoni [EMAIL PROTECTED] Reply-To: Tomcat Users List [EMAIL PROTECTED] To: Tomcat Users List [EMAIL PROTECTED] Subject: Re: JDBC ORACLE implementation ! Date: Wed, 19 Feb 2003 10:09:16 - The implementation is based on which driver you use, which mostly depends on which version of Oracle you are running. What version numbers are you using for Oracle Tomcat - Original Message - From: Swapneel Dange [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Wednesday, February 19, 2003 9:47 AM Subject: JDBC ORACLE implementation ! hi there ~ i am curious to know as to how can u implement ORACLE database in the TOMCAT. and can somebody tell me as to where i can read the DOCUMENTATION for the implementation of the JDBC connectivity under TOMCAT. Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange _ MSN 8 with e-mail virus protection service: 2 months FREE* http://join.msn.com/?page=features/virus - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] _ Protect your PC - get McAfee.com VirusScan Online http://clinic.mcafee.com/clinic/ibuy/campaign.asp?cid=3963 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] _ The new MSN 8: advanced junk mail protection and 2 months FREE* http://join.msn.com/?page=features/junkmail
RE: JDBC ORACLE implementation !
Ok, lets start from the beginning, you're getting 1 file per 12 seconds,
roughtly 7200 files a day. Where does oracle come into this? You don't
really need oracle, you just need a directory and a way to generate unique
filenames. Then you can use grep or something scripted to massage them and
figure out what you want to keep.
Now, if you want to store the files in oracle and then use the oracle text
index on clobs to do keyword searching for stuff, then you're going to need
to connect to oracle somehow and put the files into the database. Since the
files aren't too large you shouldn't have problems with the version of
oracle, but you will have problems with the intermedia text index (it's
called something else now, but I forget what the current name is). The
index won't auto-update, so you'll have to make a dml call to update the
index. You'd probably only want to do this prior to when you're going to
work with the files.
Now, you need the servlet/jsp handling the uploads to be able to access
oracle, not tomcat. Tomcat could but doesn't need to come into the picture,
up to you. Either way, assuming you're going to put the files into oracle,
you're going to need a connection. You could do that simply by putting the
required code into the jsp/servlet, which would look something like this:
Class.forName( oracle.jdbc.driver.OracleDriver );
Connection con = DriverManager.getConnection(
jdbc:oracle:thin:@127.0.0.1:1521:CTMC, user, pass );
At this point you're going to have a connection. If you want to go the
datasource route you're going to have to ask someone else, I haven't setup
tomcat to provide the connection. Anyway, once you have the connection
you're going to write into the clob, which will look like this:
con.setAutoCommit( false );
Statement s = con.createStatement();
s.execute( insert into clob_table ( id, clob ) values ( 1,
empty_clob() );
ResultSet rs = s.executeQuery( select clob from clob_table where id = 1
for update );
if ( rs.next() ) {
oracle.sql.CLOB c = ( oracle.sql.CLOB ) rs.getClob( 1 );
java.io.PrintStream out = new java.io.PrintStream(
c.getAsciiOutputStream() );
out.println( file_data );
out.close();
}
s.close();
con.commit();
Now in some cases you'd want to use a different transactionIsolation mode,
the one you'll need to use when working with clobs under oracle is
READ_COMMITED. If you use SERIALIZED you'll have intermittent problems
(most likely, but not in all cases).
Does this answer your questions?
--mikej
-=-
mike jackson
[EMAIL PROTECTED]
-Original Message-
From: Swapneel Dange [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, February 19, 2003 4:49 PM
To: [EMAIL PROTECTED]
Subject: RE: JDBC ORACLE implementation !
hi mike !
the files i am using are quite small in size like 200-300 KB but
sometimes i
get files of the size 1MB or so. but the rate at which i get
these files is
huge, i get a file every 12 second so that makes 7200 files a day and all
are useful files. i will then massgae the files and strip them of the
un-necessary details. and then look out for keywords in the file.
draw some
conclusions based on that and then implement the solutions.
moreoevr i could not understand as to what u wanted to convey through the
sentense, So, you don't really need tomcat per say to talk to
oracle, you
need yourservlets/jsps to be able to talk to oracle. . i am a NOVICE,
thats the reason that i didnt understans the meaning of the
sentense there.
anyways i can user the other JDBC drivers which are not supplied
by ORACLE.
some of the drivers suggested to me here are the ones by DATADIRECT.
awaiting reply !
Swapneel Dange
505-642-4126
http://www.cs.nmsu.edu/~sdange
From: Mike Jackson [EMAIL PROTECTED]
Reply-To: Tomcat Users List [EMAIL PROTECTED]
To: Tomcat Users List [EMAIL PROTECTED]
Subject: RE: JDBC ORACLE implementation !
Date: Wed, 19 Feb 2003 16:24:25 -0800
So, you don't really need tomcat per say to talk to oracle, you need your
servlets/jsps to be able to talk to oracle. What type of data are you
working with and how big is it? That's really the question to
ask that'll
tell you what version of oracle to use. If you using small data
(standard
types or lobs) then any version should work ok. But you'll want to be in
the READ_COMMITED transaction isolation mode (default). If
you're working
with large lobs (25 megs) you'll probably want to be on 9.2.0.1.0 or
better
(or a patched 9.1, but not an 8.x version I think). And you'll want to
make
sure you're using the jdbc driver supplied by the database, or one from a
newer version, the older ones with newer databases tend to cause
problems.
Personal/Standard/Enterprise may not matter.
--mikej
-=-
mike jackson
[EMAIL PROTECTED]
-Original Message-
From: Swapneel Dange
Re: JDBC ORACLE implementation !
First off, you probably should be use Oracle enterprise edition, unless you're on a box with less than 128meg of memory. Oracle personal edition for 8i and 9i is really designed for simple uses. The scenario you've described will probably mean storing the text as a clob or in multiple columns. keep in mind if you store it as a clob, it limits your ability to search performance. breaking the text into columns will allow you to index the content easier. If query time is important, you may want to generate summaries of the text and use that for your indexes instead. as far as connecting to oracle, it's fairly straight forward. databases are handy, but take care with how you implement the application. If you don't need to index the content, or do not need transaction capabilities, you're better off using file system to store the text. RDBMS are designed specifically to handle relational data. If your data isn't relational, using Oracle is a bit over kill. Using the right tool will make your life easier in the long run. peter Swapneel Dange [EMAIL PROTECTED] wrote:hi ! frankly speaking, i am very new to the database concepts. i am trying to use the Oracle Database Personal Edition for my use here. But i can tell about the version of the TOMCAT, i am using , its 3.3.1a. if u could tell me in detail as to what ORACLE product i should use will be great. By the way i am using this database for the users to send data from 3 different states on the webserver i am running here. so that when that data which are huge files coem over here, i can massage them and get the particular details i want for my use. thanx ! Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange From: Andoni Reply-To: Tomcat Users List To: Tomcat Users List Subject: Re: JDBC ORACLE implementation ! Date: Wed, 19 Feb 2003 10:09:16 - The implementation is based on which driver you use, which mostly depends on which version of Oracle you are running. What version numbers are you using for Oracle Tomcat - Original Message - From: Swapneel Dange To: Sent: Wednesday, February 19, 2003 9:47 AM Subject: JDBC ORACLE implementation ! hi there ~ i am curious to know as to how can u implement ORACLE database in the TOMCAT. and can somebody tell me as to where i can read the DOCUMENTATION for the implementation of the JDBC connectivity under TOMCAT. Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange _ MSN 8 with e-mail virus protection service: 2 months FREE* http://join.msn.com/?page=features/virus - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] _ Protect your PC - get McAfee.com VirusScan Online http://clinic.mcafee.com/clinic/ibuy/campaign.asp?cid=3963 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Do you Yahoo!? Yahoo! Shopping - Send Flowers for Valentine's Day
Re: JDBC ORACLE implementation !
Hi Swapneel ! I have to agree with Peter on this. Try to keep your application as simple as possible. Don't use a database (especially NOT Oracle, if you are new to databases) if you can help it, because things can get very, very complicated very quickly. I think your main problem will not be connecting to Oracle via JDBC, but rather : - how to install and tune Oracle - how to handle BLOBs/CLOBs - how to do text indexing in a database My suggestion is the same as Peter's : write to the filesystem. This way, you only need to learn one skill - how to read/write files in Java - rather than learning how to handle JDBC, install and administer Oracle (which took me almost a month the last time I did it, and this was with Oracle Consulting staff!) and learning how the text indexer in Oracle works. Hate to say this, but you may be reaching a little too far on this one. Regards, pascal chong Peter Lin wrote: First off, you probably should be use Oracle enterprise edition, unless you're on a box with less than 128meg of memory. Oracle personal edition for 8i and 9i is really designed for simple uses. The scenario you've described will probably mean storing the text as a clob or in multiple columns. keep in mind if you store it as a clob, it limits your ability to search performance. breaking the text into columns will allow you to index the content easier. If query time is important, you may want to generate summaries of the text and use that for your indexes instead. as far as connecting to oracle, it's fairly straight forward. databases are handy, but take care with how you implement the application. If you don't need to index the content, or do not need transaction capabilities, you're better off using file system to store the text. RDBMS are designed specifically to handle relational data. If your data isn't relational, using Oracle is a bit over kill. Using the right tool will make your life easier in the long run. peter - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: JDBC ORACLE implementation !
hey peter ! ur right, that there is no transaction involved in this process here. only thing i will be doing is receiving files on the server using the servlets. now may be it was too much thinking on my part to say that i will use ORAVCLE. what do u say that for atleast 7200 files a day of size max 1MB, shouldnt i use ORACLE ? should i try some other options and if YES then what kind of database can i implement. right now i have the FILE SYSTEM implemented here. but i think it has limited my ability to do pattern searching and data mining, thats why i was trying to move to something more stable and robust such as a database which can support TOUGHER queries. awaiting reaply ! Swapneel Dange 505-642-4126 http://www.cs.nmsu.edu/~sdange From: Peter Lin [EMAIL PROTECTED] Reply-To: Tomcat Users List [EMAIL PROTECTED] To: Tomcat Users List [EMAIL PROTECTED] Subject: Re: JDBC ORACLE implementation ! Date: Wed, 19 Feb 2003 18:09:36 -0800 (PST) First off, you probably should be use Oracle enterprise edition, unless you're on a box with less than 128meg of memory. Oracle personal edition for 8i and 9i is really designed for simple uses. The scenario you've described will probably mean storing the text as a clob or in multiple columns. keep in mind if you store it as a clob, it limits your ability to search performance. breaking the text into columns will allow you to index the content easier. If query time is important, you may want to generate summaries of the text and use that for your indexes instead. as far as connecting to oracle, it's fairly straight forward. databases are handy, but take care with how you implement the application. If you don't need to index the content, or do not need transaction capabilities, you're better off using file system to store the text. RDBMS are designed specifically to handle relational data. If your data isn't relational, using Oracle is a bit over kill. Using the right tool will make your life easier in the long run. peter _ Help STOP SPAM with the new MSN 8 and get 2 months FREE* http://join.msn.com/?page=features/junkmail - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
