Re: [U2] Consuming web services [not-secure]
Sorry this is late to the conversation, but you may want to look at http://www.pilotfishtechnology.com/ It's the swiss army knife of data integration. I'm using it in several different ways. You could have it go consume a service and bring the response back however you prefer. -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Daniel McGrath Sent: Wednesday, October 03, 2012 5:10 PM To: U2 Users List Subject: Re: [U2] Consuming web services Starting at UniData 7.3.0 and in UniVerse 11.1.9, we now have U2 Dynamic Objects which does help with the parsing 'till I'm blue in the face when consuming JSON requests. Regards, Dan -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Jeff Schasny Sent: Wednesday, October 03, 2012 2:56 PM To: U2-Users@listserver.u2ug.org Subject: [U2] Consuming web services I know we have both the SOAP and RESTful web services development for publishing web services from Universe but how are folks consuming other peoples web services into the database? Are there tools for this or am I going to just open a socket, read, and parse 'till I'm blue in the face? -- Jeff Schasny - Denver, Co, USA jschasny at gmail dot com ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users CONFIDENTIAL INFORMATION: The information contained in this e-mail may be confidential and protected from general disclosure. If the recipient or reader of this e-mail is not the intended recipient or a person responsible to receive this e-mail for the intended recipient, please do not disseminate, distribute or copy it. If you received this e-mail in error, please notify the sender by replying to this message and delete this e-mail immediately. We will take immediate and appropriate action to see to it that this mistake is corrected.[*LD*] ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Consuming Web Services
Will I don't understand what's wrong with indexing, can you clarify this point, and I'll wipe out a fix in three days :) Well for a start I didn't say there's anything wrong, I said it could be improved - not the same thing! But as to specifics, take the following scenario (UniVerse specific): - Grab a transaction file for say, 10 million records. - Assume a reasonable key length say 10 chars. - Add a field with two states (open/closed, male/female, that kind of thing). - Index it, and watch what happens to the performance on that file. - Better still, don't use an existing file! Create a new file and a program to copy or build the content in basic and show a counter every 1000 records. At the start it will be quick. After about 500,000 you can grab a beer in between the counters. The problem is, that a UniVerse index is very good at establishing the index key: it has a nice B+tree format with a decent level of fan-out. But when it comes to the list of primary keys being indexed against each index key, that's really just treated as a block of data. If you have a good ratio with a lot of index keys (date*type*something_else) each of which gives a relatively short list of primary keys you can get very good indexing performance. But it isn't very clever when you have a small number of index keys to a large list of primary keys. So every time you changed the flag value in the file above it would have to load up the two lists (one for old value, one for new), locate and delete from the old and locate/fail/append to the new, each list averaging 11 byte * 5 million entries. And then write it back to a succession of oversize blocks in the index file. Now you might say - well, you wouldn't index a transaction file like that. And you would be right - because of the design of the index. But it's a perfectly legitimate and reasonable thing to want to do. How to better manage a large index list is, of course, the question. Since it is a large list into which elements are potentially inserted/deleted in order, the list itself could be made into a set of B+Tree pages over a certain threshold, reducing the cost of location/insertion and location/deletion. Other databases use techniques such as key deltas and compression to alleviate this. And I'm sure there are better options if I could be bothered to research them. So there you go, Will. Your job for the weekend. Redesign the UniVerse indexing so it works for large lists, and get Rocket to adopt it. :) Brian -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Wjhonson Sent: 04 October 2012 16:43 To: u2-users@listserver.u2ug.org Subject: Re: [U2] Consuming Web Services ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Consuming Web Services
Will and Brian, I had a similar experience. I indexed a file that had over a million records with the same code. All was ok for a couple of hours and then Universe crashed with something like out of string space. It was ugly! Had this worked, it would have been a great benefit but . I expermented with combining 3 attributes to reduce the max count to around 100,000 but the boss isn't willing to try again. Don Robinson --- From: Brian Leach br...@brianleach.co.uk To: 'U2 Users List' u2-users@listserver.u2ug.org Sent: Friday, October 5, 2012 8:59 AM Subject: Re: [U2] Consuming Web Services Will I don't understand what's wrong with indexing, can you clarify this point, and I'll wipe out a fix in three days :) Well for a start I didn't say there's anything wrong, I said it could be improved - not the same thing! But as to specifics, take the following scenario (UniVerse specific): - Grab a transaction file for say, 10 million records. - Assume a reasonable key length say 10 chars. - Add a field with two states (open/closed, male/female, that kind of thing). - Index it, and watch what happens to the performance on that file. - Better still, don't use an existing file! Create a new file and a program to copy or build the content in basic and show a counter every 1000 records. At the start it will be quick. After about 500,000 you can grab a beer in between the counters. The problem is, that a UniVerse index is very good at establishing the index key: it has a nice B+tree format with a decent level of fan-out. But when it comes to the list of primary keys being indexed against each index key, that's really just treated as a block of data. If you have a good ratio with a lot of index keys (date*type*something_else) each of which gives a relatively short list of primary keys you can get very good indexing performance. But it isn't very clever when you have a small number of index keys to a large list of primary keys. So every time you changed the flag value in the file above it would have to load up the two lists (one for old value, one for new), locate and delete from the old and locate/fail/append to the new, each list averaging 11 byte * 5 million entries. And then write it back to a succession of oversize blocks in the index file. Now you might say - well, you wouldn't index a transaction file like that. And you would be right - because of the design of the index. But it's a perfectly legitimate and reasonable thing to want to do. How to better manage a large index list is, of course, the question. Since it is a large list into which elements are potentially inserted/deleted in order, the list itself could be made into a set of B+Tree pages over a certain threshold, reducing the cost of location/insertion and location/deletion. Other databases use techniques such as key deltas and compression to alleviate this. And I'm sure there are better options if I could be bothered to research them. So there you go, Will. Your job for the weekend. Redesign the UniVerse indexing so it works for large lists, and get Rocket to adopt it. :) Brian -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Wjhonson Sent: 04 October 2012 16:43 To: u2-users@listserver.u2ug.org Subject: Re: [U2] Consuming Web Services ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Consuming Web Services
The first part is relatively trivial for a person of my level of genius :) The second part is so utterly Byzantine and unfathomable (and redundant apparently) that I would never even make the attempt. What Rocket adopts and also what they discard, has never been comprehendable. So there you go, Will. Your job for the weekend. Redesign the UniVerse indexing so it works for large lists, and get Rocket to adopt it. -Original Message- From: Brian Leach br...@brianleach.co.uk To: 'U2 Users List' u2-users@listserver.u2ug.org Sent: Fri, Oct 5, 2012 5:59 am Subject: Re: [U2] Consuming Web Services Will I don't understand what's wrong with indexing, can you clarify this point, and I'll wipe out a fix in three days :) Well for a start I didn't say there's anything wrong, I said it could be improved - not the same thing! But as to specifics, take the following scenario (UniVerse specific): - Grab a transaction file for say, 10 million records. - Assume a reasonable key length say 10 chars. - Add a field with two states (open/closed, male/female, that kind of thing). - Index it, and watch what happens to the performance on that file. - Better still, don't use an existing file! Create a new file and a program to copy or build the content in basic and show a counter every 1000 records. At the start it will be quick. After about 500,000 you can grab a beer in between the counters. The problem is, that a UniVerse index is very good at establishing the index key: it has a nice B+tree format with a decent level of fan-out. But when it comes to the list of primary keys being indexed against each index key, that's really just treated as a block of data. If you have a good ratio with a lot of index keys (date*type*something_else) each of which gives a relatively short list of primary keys you can get very good indexing performance. But it isn't very clever when you have a small number of index keys to a large list of primary keys. So every time you changed the flag value in the file above it would have to load up the two lists (one for old value, one for new), locate and delete from the old and locate/fail/append to the new, each list averaging 11 byte * 5 million entries. And then write it back to a succession of oversize blocks in the index file. Now you might say - well, you wouldn't index a transaction file like that. And you would be right - because of the design of the index. But it's a perfectly legitimate and reasonable thing to want to do. How to better manage a large index list is, of course, the question. Since it is a large list into which elements are potentially inserted/deleted in order, the list itself could be made into a set of B+Tree pages over a certain threshold, reducing the cost of location/insertion and location/deletion. Other databases use techniques such as key deltas and compression to alleviate this. And I'm sure there are better options if I could be bothered to research them. So there you go, Will. Your job for the weekend. Redesign the UniVerse indexing so it works for large lists, and get Rocket to adopt it. :) Brian -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Wjhonson Sent: 04 October 2012 16:43 To: u2-users@listserver.u2ug.org Subject: Re: [U2] Consuming Web Services ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Consuming web services
Hi, If I understand correctly, you're asking how the data embedded in the xml gets parsed out and written to a uv file, correct? If so, we use a uv function called XMLTODB, it's in the bp of the uv account. We accept orders from college book stores and we do that by consuming their web services. We use XMLTODB and the XMAP api to parse out the data that writes to our uv file. Works great. We've been using it for a few years now. I know we all hate documentation, but it is well documented in the BasicExt pdf. HTH, rudy Message: 2 Date: Wed, 03 Oct 2012 14:56:05 -0600 From: Jeff Schasny jscha...@gmail.commailto:jscha...@gmail.com To: U2-Users@listserver.u2ug.orgmailto:U2-Users@listserver.u2ug.org Subject: [U2] Consuming web services Message-ID: 506ca665.5020...@gmail.commailto:506ca665.5020...@gmail.com Content-Type: text/plain; charset=ISO-8859-1; format=flowed I know we have both the SOAP and RESTful web services development for publishing web services from Universe but how are folks consuming other peoples web services into the database? Are there tools for this or am I going to just open a socket, read, and parse 'till I'm blue in the face? Are you a fan of SAGE? Show us at www.facebook.com/SAGEPublicationshttp://www.facebook.com/SAGEPublications. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Consuming Web Services (U2 Indexing)
Brian: I was under the impression that UniData uses a real B-Tree indexing system while UniVerse uses some kind of linked list. e.g. UV has a single item for, say, male/female and the item would look like ID: male 001 1]2]3]4]5]6]...]999 ...which would perform exactly as you say. I don't think UniData performs that way at all. Bill - Original Message - *From:* br...@brianleach.co.uk *To:* 'U2 Users List' u2-users@listserver.u2ug.org *Date:* 10/5/2012 5:59 AM *Subject:* Re: [U2] Consuming Web Services Will I don't understand what's wrong with indexing, can you clarify this point, and I'll wipe out a fix in three days :) Well for a start I didn't say there's anything wrong, I said it could be improved - not the same thing! But as to specifics, take the following scenario (UniVerse specific): - Grab a transaction file for say, 10 million records. - Assume a reasonable key length say 10 chars. - Add a field with two states (open/closed, male/female, that kind of thing). - Index it, and watch what happens to the performance on that file. - Better still, don't use an existing file! Create a new file and a program to copy or build the content in basic and show a counter every 1000 records. At the start it will be quick. After about 500,000 you can grab a beer in between the counters. The problem is, that a UniVerse index is very good at establishing the index key: it has a nice B+tree format with a decent level of fan-out. But when it comes to the list of primary keys being indexed against each index key, that's really just treated as a block of data. If you have a good ratio with a lot of index keys (date*type*something_else) each of which gives a relatively short list of primary keys you can get very good indexing performance. But it isn't very clever when you have a small number of index keys to a large list of primary keys. So every time you changed the flag value in the file above it would have to load up the two lists (one for old value, one for new), locate and delete from the old and locate/fail/append to the new, each list averaging 11 byte * 5 million entries. And then write it back to a succession of oversize blocks in the index file. Now you might say - well, you wouldn't index a transaction file like that. And you would be right - because of the design of the index. But it's a perfectly legitimate and reasonable thing to want to do. How to better manage a large index list is, of course, the question. Since it is a large list into which elements are potentially inserted/deleted in order, the list itself could be made into a set of B+Tree pages over a certain threshold, reducing the cost of location/insertion and location/deletion. Other databases use techniques such as key deltas and compression to alleviate this. And I'm sure there are better options if I could be bothered to research them. So there you go, Will. Your job for the weekend. Redesign the UniVerse indexing so it works for large lists, and get Rocket to adopt it. :) Brian ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Consuming Web Services (U2 Indexing)
Bill I *did* say UniVerse specific :) Yes, it uses a really nice and well-designed B+Tree for the index keys but once you're down to the data (the primary keys) they are stored in a regular record format with @FM between each key. You can see that easily enough as you can create a pointer to the INDEX.nnn record and just read/write it like any other type 25 file. Which is lots of luurrvvelley out of line record blocks to fill up when you do an insertion into the middle of a huge index list. Brian -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Bill Haskett Sent: 05 October 2012 17:15 To: U2 Users List Subject: Re: [U2] Consuming Web Services (U2 Indexing) Brian: I was under the impression that UniData uses a real B-Tree indexing system while UniVerse uses some kind of linked list. e.g. UV has a single item for, say, male/female and the item would look like ID: male 001 1]2]3]4]5]6]...]999 ...which would perform exactly as you say. I don't think UniData performs that way at all. Bill - Original Message - *From:* br...@brianleach.co.uk *To:* 'U2 Users List' u2-users@listserver.u2ug.org *Date:* 10/5/2012 5:59 AM *Subject:* Re: [U2] Consuming Web Services Will I don't understand what's wrong with indexing, can you clarify this point, and I'll wipe out a fix in three days :) Well for a start I didn't say there's anything wrong, I said it could be improved - not the same thing! But as to specifics, take the following scenario (UniVerse specific): - Grab a transaction file for say, 10 million records. - Assume a reasonable key length say 10 chars. - Add a field with two states (open/closed, male/female, that kind of thing). - Index it, and watch what happens to the performance on that file. - Better still, don't use an existing file! Create a new file and a program to copy or build the content in basic and show a counter every 1000 records. At the start it will be quick. After about 500,000 you can grab a beer in between the counters. The problem is, that a UniVerse index is very good at establishing the index key: it has a nice B+tree format with a decent level of fan-out. But when it comes to the list of primary keys being indexed against each index key, that's really just treated as a block of data. If you have a good ratio with a lot of index keys (date*type*something_else) each of which gives a relatively short list of primary keys you can get very good indexing performance. But it isn't very clever when you have a small number of index keys to a large list of primary keys. So every time you changed the flag value in the file above it would have to load up the two lists (one for old value, one for new), locate and delete from the old and locate/fail/append to the new, each list averaging 11 byte * 5 million entries. And then write it back to a succession of oversize blocks in the index file. Now you might say - well, you wouldn't index a transaction file like that. And you would be right - because of the design of the index. But it's a perfectly legitimate and reasonable thing to want to do. How to better manage a large index list is, of course, the question. Since it is a large list into which elements are potentially inserted/deleted in order, the list itself could be made into a set of B+Tree pages over a certain threshold, reducing the cost of location/insertion and location/deletion. Other databases use techniques such as key deltas and compression to alleviate this. And I'm sure there are better options if I could be bothered to research them. So there you go, Will. Your job for the weekend. Redesign the UniVerse indexing so it works for large lists, and get Rocket to adopt it. :) Brian ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Consuming Web Services (U2 Indexing)
Gives you a sort of Dilbert like impression that at the end of the project, once all the B tree bugs were worked out, they were shouting Just make it work, we'll make it better next version! Only next version never came. Yes Brian a B-tree on a B-tree probably makes the most sense from a beautiful standpoint. You could also put a B-tree on top of a dynamic file or even a B-tree on top of a Pascal-like-linked-list (not a UV multi-valued list as it is presently). No reason why you couldn't code true linked lists into uv, and they don't have issues with insertion-in-the-middle, you just break the chain, insert, and reset the forward and backward pointers. The most you're rewriting, is your item or group if you use a block concept, not the entire list as your example here. So yeah it could be done. It's conceptually not hard. Now get Rocket to put some money into the budget for that. -Original Message- From: Brian Leach br...@brianleach.co.uk To: 'U2 Users List' u2-users@listserver.u2ug.org Sent: Fri, Oct 5, 2012 9:25 am Subject: Re: [U2] Consuming Web Services (U2 Indexing) Bill I *did* say UniVerse specific :) Yes, it uses a really nice and well-designed B+Tree for the index keys but once you're down to the data (the primary keys) they are stored in a regular record format with @FM between each key. You can see that easily enough as you can create a pointer to the INDEX.nnn record and just read/write it like any other type 25 file. Which is lots of luurrvvelley out of line record blocks to fill up when you do an insertion into the middle of a huge index list. Brian -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Bill Haskett Sent: 05 October 2012 17:15 To: U2 Users List Subject: Re: [U2] Consuming Web Services (U2 Indexing) Brian: I was under the impression that UniData uses a real B-Tree indexing system while UniVerse uses some kind of linked list. e.g. UV has a single item for, say, male/female and the item would look like ID: male 001 1]2]3]4]5]6]...]999 ...which would perform exactly as you say. I don't think UniData performs that way at all. Bill - Original Message - *From:* br...@brianleach.co.uk *To:* 'U2 Users List' u2-users@listserver.u2ug.org *Date:* 10/5/2012 5:59 AM *Subject:* Re: [U2] Consuming Web Services Will I don't understand what's wrong with indexing, can you clarify this point, and I'll wipe out a fix in three days :) Well for a start I didn't say there's anything wrong, I said it could be improved - not the same thing! But as to specifics, take the following scenario (UniVerse specific): - Grab a transaction file for say, 10 million records. - Assume a reasonable key length say 10 chars. - Add a field with two states (open/closed, male/female, that kind of thing). - Index it, and watch what happens to the performance on that file. - Better still, don't use an existing file! Create a new file and a program to copy or build the content in basic and show a counter every 1000 records. At the start it will be quick. After about 500,000 you can grab a beer in between the counters. The problem is, that a UniVerse index is very good at establishing the index key: it has a nice B+tree format with a decent level of fan-out. But when it comes to the list of primary keys being indexed against each index key, that's really just treated as a block of data. If you have a good ratio with a lot of index keys (date*type*something_else) each of which gives a relatively short list of primary keys you can get very good indexing performance. But it isn't very clever when you have a small number of index keys to a large list of primary keys. So every time you changed the flag value in the file above it would have to load up the two lists (one for old value, one for new), locate and delete from the old and locate/fail/append to the new, each list averaging 11 byte * 5 million entries. And then write it back to a succession of oversize blocks in the index file. Now you might say - well, you wouldn't index a transaction file like that. And you would be right - because of the design of the index. But it's a perfectly legitimate and reasonable thing to want to do. How to better manage a large index list is, of course, the question. Since it is a large list into which elements are potentially inserted/deleted in order, the list itself could be made into a set of B+Tree pages over a certain threshold, reducing the cost of location/insertion and location/deletion. Other databases use techniques such as key deltas and compression to alleviate this. And I'm sure there are better options if I could be bothered to research them. So there you go, Will. Your job for the weekend. Redesign the UniVerse indexing so it works for large lists, and get Rocket to adopt it. :) Brian
Re: [U2] Consuming Web Services
But we all know U2 is much more than a DB, in fact it is also much less than a DB. The concept of having the DB just do DB stuff works fine for SQL server and MySQL where the DB itself actually does a lot in terms of the data, and the clients, written in .NET, C, Python Java etc do all the business logic. For these stored procedures do exist but are usually kept to a minimum and certainly only for data interaction. The U2 databases are actually quite stupid, they store and retrieve data from tables in a (hopefully) quick and efficient manner, but they certainly don't do any of the clever stuff you find inside Sql server like map reduce, caching, indexing etc. The clever bits are what we do in the unibasic code. So I firmly believe there IS a case for the clever Unibasic code we write to access data over http, read in xml and to then process all of these different data streams. Yeah sure there are things which in theory are easier to do outside It does not mean you should always do things outside of unibasic. -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Bill Brutzman Sent: 03 October 2012 23:48 To: U2 Users List Subject: Re: [U2] Consuming Web Services Tony and Ben are right on... Invoking Stephen Colbert's concept of... The Word... the word is... MiddleWare. As legacy U2 was ahead of its time... by building-in middleware technologies... it is easy to lose sight of what belongs where. --Bill -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Tony Gravagno Sent: Wednesday, October 03, 2012 6:31 PM Subject: Re: [U2] Consuming web services I do the same as Ben but with .NET: -- Local U2 .NET client web service Remote server Pick is a database server, not a communications protocol end-point. We simply should not be doing direct comms from this platform anymore, given the huge number of mainstream options. This is coming from someone who has written all of the bi-directional comms interfaces to/from MV with sockets and cURL and plugins and anything else I could create - because (in the 90's) people said it wasn't possible and because I figured it would be kewl just to do it. It _is_ all technically possible, but that doesn't mean it makes good business sense: Doing things like this in BASIC creates a maintenance hassle later. It leaves the environment subject to DBMS-specific issues that often aren't addressed for quite a long time. Using mainstream tools vastly increases the resources available for getting help and solving problems. So these days I get MV to push a query or data payload out to a middle-tier that uses the latest communications methods available. The request goes out, the response comes back, it all just works - that That should be our bottom line here. HTH T From: Ben Souther We use a middle layer written in Java. Jeff Schasny wrote: I know we have both the SOAP and RESTful web services development for publishing web services from Universe but how are folks consuming other peoples web services into the database? Are there tools for this or am I going to just open a socket, read, and parse 'till I'm blue in the face? ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Consuming Web Services
The U2 database does do indexing. -Original Message- From: Symeon Breen syme...@gmail.com To: 'U2 Users List' u2-users@listserver.u2ug.org Sent: Thu, Oct 4, 2012 12:56 am Subject: Re: [U2] Consuming Web Services But we all know U2 is much more than a DB, in fact it is also much less than a DB. The concept of having the DB just do DB stuff works fine for SQL server and MySQL where the DB itself actually does a lot in terms of the data, and the clients, written in .NET, C, Python Java etc do all the business logic. For these stored procedures do exist but are usually kept to a minimum and certainly only for data interaction. The U2 databases are actually quite stupid, they store and retrieve data from tables in a (hopefully) quick and efficient manner, but they certainly don't do any of the clever stuff you find inside Sql server like map reduce, caching, indexing etc. The clever bits are what we do in the unibasic code. So I firmly believe there IS a case for the clever Unibasic code we write to access data over http, read in xml and to then process all of these different data streams. Yeah sure there are things which in theory are easier to do outside It does not mean you should always do things outside of unibasic. -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Bill Brutzman Sent: 03 October 2012 23:48 To: U2 Users List Subject: Re: [U2] Consuming Web Services Tony and Ben are right on... Invoking Stephen Colbert's concept of... The Word... the word is... MiddleWare. As legacy U2 was ahead of its time... by building-in middleware technologies... it is easy to lose sight of what belongs where. --Bill -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Tony Gravagno Sent: Wednesday, October 03, 2012 6:31 PM Subject: Re: [U2] Consuming web services I do the same as Ben but with .NET: -- Local U2 .NET client web service Remote server Pick is a database server, not a communications protocol end-point. We simply should not be doing direct comms from this platform anymore, given the huge number of mainstream options. This is coming from someone who has written all of the bi-directional comms interfaces to/from MV with sockets and cURL and plugins and anything else I could create - because (in the 90's) people said it wasn't possible and because I figured it would be kewl just to do it. It _is_ all technically possible, but that doesn't mean it makes good business sense: Doing things like this in BASIC creates a maintenance hassle later. It leaves the environment subject to DBMS-specific issues that often aren't addressed for quite a long time. Using mainstream tools vastly increases the resources available for getting help and solving problems. So these days I get MV to push a query or data payload out to a middle-tier that uses the latest communications methods available. The request goes out, the response comes back, it all just works - that That should be our bottom line here. HTH T From: Ben Souther We use a middle layer written in Java. Jeff Schasny wrote: I know we have both the SOAP and RESTful web services development for publishing web services from Universe but how are folks consuming other peoples web services into the database? Are there tools for this or am I going to just open a socket, read, and parse 'till I'm blue in the face? ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Consuming Web Services
I know - but you have to set it up against specific attributes and it creates a fairly standard index for the specified attributes in hand, you do that on sql as well, but it does its own indexing to make row retrieval optimised, has its own referential integrity rules around that etc. It's quite a different beast altogether. I am not saying it is better, but it does a lot more inside than U2 does. -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Wjhonson Sent: 04 October 2012 09:05 To: u2-users@listserver.u2ug.org Subject: Re: [U2] Consuming Web Services The U2 database does do indexing. -Original Message- From: Symeon Breen syme...@gmail.com To: 'U2 Users List' u2-users@listserver.u2ug.org Sent: Thu, Oct 4, 2012 12:56 am Subject: Re: [U2] Consuming Web Services But we all know U2 is much more than a DB, in fact it is also much less than a DB. The concept of having the DB just do DB stuff works fine for SQL server and MySQL where the DB itself actually does a lot in terms of the data, and the clients, written in .NET, C, Python Java etc do all the business logic. For these stored procedures do exist but are usually kept to a minimum and certainly only for data interaction. The U2 databases are actually quite stupid, they store and retrieve data from tables in a (hopefully) quick and efficient manner, but they certainly don't do any of the clever stuff you find inside Sql server like map reduce, caching, indexing etc. The clever bits are what we do in the unibasic code. So I firmly believe there IS a case for the clever Unibasic code we write to access data over http, read in xml and to then process all of these different data streams. Yeah sure there are things which in theory are easier to do outside It does not mean you should always do things outside of unibasic. -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Bill Brutzman Sent: 03 October 2012 23:48 To: U2 Users List Subject: Re: [U2] Consuming Web Services Tony and Ben are right on... Invoking Stephen Colbert's concept of... The Word... the word is... MiddleWare. As legacy U2 was ahead of its time... by building-in middleware technologies... it is easy to lose sight of what belongs where. --Bill -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Tony Gravagno Sent: Wednesday, October 03, 2012 6:31 PM Subject: Re: [U2] Consuming web services I do the same as Ben but with .NET: -- Local U2 .NET client web service Remote server Pick is a database server, not a communications protocol end-point. We simply should not be doing direct comms from this platform anymore, given the huge number of mainstream options. This is coming from someone who has written all of the bi-directional comms interfaces to/from MV with sockets and cURL and plugins and anything else I could create - because (in the 90's) people said it wasn't possible and because I figured it would be kewl just to do it. It _is_ all technically possible, but that doesn't mean it makes good business sense: Doing things like this in BASIC creates a maintenance hassle later. It leaves the environment subject to DBMS-specific issues that often aren't addressed for quite a long time. Using mainstream tools vastly increases the resources available for getting help and solving problems. So these days I get MV to push a query or data payload out to a middle-tier that uses the latest communications methods available. The request goes out, the response comes back, it all just works - that That should be our bottom line here. HTH T From: Ben Souther We use a middle layer written in Java. Jeff Schasny wrote: I know we have both the SOAP and RESTful web services development for publishing web services from Universe but how are folks consuming other peoples web services into the database? Are there tools for this or am I going to just open a socket, read, and parse 'till I'm blue in the face? ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Consuming Web Services
A U2 Table *can* also have referential integrity, it's an option. And SQL Table don't force referential integrity, you can turn that off :) -Original Message- From: Symeon Breen syme...@gmail.com To: 'U2 Users List' u2-users@listserver.u2ug.org Sent: Thu, Oct 4, 2012 1:17 am Subject: Re: [U2] Consuming Web Services I know - but you have to set it up against specific attributes and it creates a fairly standard index for the specified attributes in hand, you do that on sql as well, but it does its own indexing to make row retrieval optimised, has its own referential integrity rules around that etc. It's quite a different beast altogether. I am not saying it is better, but it does a lot more inside than U2 does. -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Wjhonson Sent: 04 October 2012 09:05 To: u2-users@listserver.u2ug.org Subject: Re: [U2] Consuming Web Services The U2 database does do indexing. -Original Message- From: Symeon Breen syme...@gmail.com To: 'U2 Users List' u2-users@listserver.u2ug.org Sent: Thu, Oct 4, 2012 12:56 am Subject: Re: [U2] Consuming Web Services But we all know U2 is much more than a DB, in fact it is also much less than a DB. The concept of having the DB just do DB stuff works fine for SQL server and MySQL where the DB itself actually does a lot in terms of the data, and the clients, written in .NET, C, Python Java etc do all the business logic. For these stored procedures do exist but are usually kept to a minimum and certainly only for data interaction. The U2 databases are actually quite stupid, they store and retrieve data from tables in a (hopefully) quick and efficient manner, but they certainly don't do any of the clever stuff you find inside Sql server like map reduce, caching, indexing etc. The clever bits are what we do in the unibasic code. So I firmly believe there IS a case for the clever Unibasic code we write to access data over http, read in xml and to then process all of these different data streams. Yeah sure there are things which in theory are easier to do outside It does not mean you should always do things outside of unibasic. -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Bill Brutzman Sent: 03 October 2012 23:48 To: U2 Users List Subject: Re: [U2] Consuming Web Services Tony and Ben are right on... Invoking Stephen Colbert's concept of... The Word... the word is... MiddleWare. As legacy U2 was ahead of its time... by building-in middleware technologies... it is easy to lose sight of what belongs where. --Bill -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Tony Gravagno Sent: Wednesday, October 03, 2012 6:31 PM Subject: Re: [U2] Consuming web services I do the same as Ben but with .NET: -- Local U2 .NET client web service Remote server Pick is a database server, not a communications protocol end-point. We simply should not be doing direct comms from this platform anymore, given the huge number of mainstream options. This is coming from someone who has written all of the bi-directional comms interfaces to/from MV with sockets and cURL and plugins and anything else I could create - because (in the 90's) people said it wasn't possible and because I figured it would be kewl just to do it. It _is_ all technically possible, but that doesn't mean it makes good business sense: Doing things like this in BASIC creates a maintenance hassle later. It leaves the environment subject to DBMS-specific issues that often aren't addressed for quite a long time. Using mainstream tools vastly increases the resources available for getting help and solving problems. So these days I get MV to push a query or data payload out to a middle-tier that uses the latest communications methods available. The request goes out, the response comes back, it all just works - that That should be our bottom line here. HTH T From: Ben Souther We use a middle layer written in Java. Jeff Schasny wrote: I know we have both the SOAP and RESTful web services development for publishing web services from Universe but how are folks consuming other peoples web services into the database? Are there tools for this or am I going to just open a socket, read, and parse 'till I'm blue in the face? ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Consuming Web Services
Ok so perhaps there is this and that. My point is a database like SQL server is architected in a very different manner to U2 - it is the actual central sqlserver.exe process that does the majority of the work, it accesses the tables and presents them to the api, its caching often used data, it obeys the schema, transaction, referential, indexing rules, it map reduces collations and totals, it looks after security, pools, licencing a whole plethora of things. Thereis a lot more out of the box inside SQL server than there is from U2 (again not saying it is better, just different) The central U2 processes look after licencing, locks, shared memory subroutines and a few other things. It is the udt or uvsh client process in which your databasic runs that has built into it the DB access code this is the key difference I feel, and hence why I feel clever unibasic code has a right to do clever things with sockets and xml etc. -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Wjhonson Sent: 04 October 2012 09:20 To: u2-users@listserver.u2ug.org Subject: Re: [U2] Consuming Web Services A U2 Table *can* also have referential integrity, it's an option. And SQL Table don't force referential integrity, you can turn that off :) -Original Message- From: Symeon Breen syme...@gmail.com To: 'U2 Users List' u2-users@listserver.u2ug.org Sent: Thu, Oct 4, 2012 1:17 am Subject: Re: [U2] Consuming Web Services I know - but you have to set it up against specific attributes and it creates a fairly standard index for the specified attributes in hand, you do that on sql as well, but it does its own indexing to make row retrieval optimised, has its own referential integrity rules around that etc. It's quite a different beast altogether. I am not saying it is better, but it does a lot more inside than U2 does. -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Wjhonson Sent: 04 October 2012 09:05 To: u2-users@listserver.u2ug.org Subject: Re: [U2] Consuming Web Services The U2 database does do indexing. -Original Message- From: Symeon Breen syme...@gmail.com To: 'U2 Users List' u2-users@listserver.u2ug.org Sent: Thu, Oct 4, 2012 12:56 am Subject: Re: [U2] Consuming Web Services But we all know U2 is much more than a DB, in fact it is also much less than a DB. The concept of having the DB just do DB stuff works fine for SQL server and MySQL where the DB itself actually does a lot in terms of the data, and the clients, written in .NET, C, Python Java etc do all the business logic. For these stored procedures do exist but are usually kept to a minimum and certainly only for data interaction. The U2 databases are actually quite stupid, they store and retrieve data from tables in a (hopefully) quick and efficient manner, but they certainly don't do any of the clever stuff you find inside Sql server like map reduce, caching, indexing etc. The clever bits are what we do in the unibasic code. So I firmly believe there IS a case for the clever Unibasic code we write to access data over http, read in xml and to then process all of these different data streams. Yeah sure there are things which in theory are easier to do outside It does not mean you should always do things outside of unibasic. -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Bill Brutzman Sent: 03 October 2012 23:48 To: U2 Users List Subject: Re: [U2] Consuming Web Services Tony and Ben are right on... Invoking Stephen Colbert's concept of... The Word... the word is... MiddleWare. As legacy U2 was ahead of its time... by building-in middleware technologies... it is easy to lose sight of what belongs where. --Bill -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Tony Gravagno Sent: Wednesday, October 03, 2012 6:31 PM Subject: Re: [U2] Consuming web services I do the same as Ben but with .NET: -- Local U2 .NET client web service Remote server Pick is a database server, not a communications protocol end-point. We simply should not be doing direct comms from this platform anymore, given the huge number of mainstream options. This is coming from someone who has written all of the bi-directional comms interfaces to/from MV with sockets and cURL and plugins and anything else I could create - because (in the 90's) people said it wasn't possible and because I figured it would be kewl just to do it. It _is_ all technically possible, but that doesn't mean it makes good business sense: Doing things like this in BASIC creates a maintenance hassle later. It leaves the environment subject to DBMS-specific issues that often aren't addressed for quite a long time. Using mainstream tools vastly increases the resources
Re: [U2] Consuming Web Services
On 04/10/12 09:43, Symeon Breen wrote: Ok so perhaps there is this and that. My point is a database like SQL server is architected in a very different manner to U2 But that's why U2 is superior :-) - it is the actual central sqlserver.exe process that does the majority of the work, it accesses the tables and presents them to the api, its caching often used data, Which U2 actually does far better - (a) it leaves it to the OS, and (b) with a disk hit rate approaching 100% it doesn't need the cache that often (that's badly put, but I know what I mean :-) it obeys the schema, transaction, referential, indexing rules, it map reduces collations and totals, it looks after security, pools, licencing a whole plethora of things. Thereis a lot more out of the box inside SQL server than there is from U2 (again not saying it is better, just different) That's what gets me - U2 is often assumed to be inferior because it doesn't have the things SQL databases have, when the reality is it doesn't have them because it doesn't need them! The central U2 processes look after licencing, locks, shared memory subroutines and a few other things. It is the udt or uvsh client process in which your databasic runs that has built into it the DB access code this is the key difference I feel, and hence why I feel clever unibasic code has a right to do clever things with sockets and xml etc. But it would be nice if more of these things (even if written in BASIC, which is a *good* thing) actually came out of the box with U2. Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Consuming Web Services
I was not trying to say sql server is better or worse than U2 just Different - hence supporting my argument that xml and socket calls are OK in unibasic -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Wols Lists Sent: 04 October 2012 10:20 To: u2-users@listserver.u2ug.org Subject: Re: [U2] Consuming Web Services On 04/10/12 09:43, Symeon Breen wrote: Ok so perhaps there is this and that. My point is a database like SQL server is architected in a very different manner to U2 But that's why U2 is superior :-) - it is the actual central sqlserver.exe process that does the majority of the work, it accesses the tables and presents them to the api, its caching often used data, Which U2 actually does far better - (a) it leaves it to the OS, and (b) with a disk hit rate approaching 100% it doesn't need the cache that often (that's badly put, but I know what I mean :-) it obeys the schema, transaction, referential, indexing rules, it map reduces collations and totals, it looks after security, pools, licencing a whole plethora of things. Thereis a lot more out of the box inside SQL server than there is from U2 (again not saying it is better, just different) That's what gets me - U2 is often assumed to be inferior because it doesn't have the things SQL databases have, when the reality is it doesn't have them because it doesn't need them! The central U2 processes look after licencing, locks, shared memory subroutines and a few other things. It is the udt or uvsh client process in which your databasic runs that has built into it the DB access code this is the key difference I feel, and hence why I feel clever unibasic code has a right to do clever things with sockets and xml etc. But it would be nice if more of these things (even if written in BASIC, which is a *good* thing) actually came out of the box with U2. Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Consuming Web Services
Very true and I agree that the DB Server (SQL, et al) and the DB client engine (UV) models each have pros and cons. UV is a lightweight DB and application engine with some very good capabilities in the DB and application area. Plus of course some truly powerful features too! It is not the best or worse in any of these fields. It fills its own little niche very nicely. UV has had some truly good native SQL engine abilities (schemas, tables, triggers, integrity constraints, stored procedures, dynamic normalisation of multi-valued fields, etc) for a long time. Not tacked on like the UD counterparts. But next to no one uses them. As they would break your legacy application. Plus if you created a new application/DB using it - you may as well use a read SQL database server! Some of the nifty features of true SQL DB servers are really to work around the limitations of the relational model and how you store your data (requiring complex joins, queries, etc). Which tends to make those nifty features essential. Whereas SQL DB servers are progress in leaps and bounds in many different and diverse ways and this is where they truly diverge - be it to cater for massive databases or distributed or failover-active/active database clusters, etc., of any data type or structure you could imagine. Whereas the U2 database engine has not really progressed or evolved much at all in reality - that is not a bad thing. Adding various extensions to the traditional PICK/U2 application interfaces is a very good thing. It does add lots of functions that are truly useful - requiring minimal effort and zero install or cost (a key requirement for lots of sites). If you need something better AND you have local talent to build/maintain it (or consultancy budget) - then go again and do it. :) There are many examples of organisations using their own interfaces/extensions - because they can afford something better that works for them. Your mileage and budgetary will vary! Hence, as you imply - use each database - exploiting their inherent strengths. Use the right tool for the right job. Every tool has its sweet spot! ;) I won't mention the number of MS Access databases out there doing real work...groan! -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Symeon Breen Sent: Thursday, 4 October 2012 5:56 PM To: 'U2 Users List' Subject: Re: [U2] Consuming Web Services But we all know U2 is much more than a DB, in fact it is also much less than a DB. The concept of having the DB just do DB stuff works fine for SQL server and MySQL where the DB itself actually does a lot in terms of the data, and the clients, written in .NET, C, Python Java etc do all the business logic. For these stored procedures do exist but are usually kept to a minimum and certainly only for data interaction. The U2 databases are actually quite stupid, they store and retrieve data from tables in a (hopefully) quick and efficient manner, but they certainly don't do any of the clever stuff you find inside Sql server like map reduce, caching, indexing etc. The clever bits are what we do in the unibasic code. So I firmly believe there IS a case for the clever Unibasic code we write to access data over http, read in xml and to then process all of these different data streams. Yeah sure there are things which in theory are easier to do outside It does not mean you should always do things outside of unibasic. -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Bill Brutzman Sent: 03 October 2012 23:48 To: U2 Users List Subject: Re: [U2] Consuming Web Services Tony and Ben are right on... Invoking Stephen Colbert's concept of... The Word... the word is... MiddleWare. As legacy U2 was ahead of its time... by building-in middleware technologies... it is easy to lose sight of what belongs where. --Bill -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Tony Gravagno Sent: Wednesday, October 03, 2012 6:31 PM Subject: Re: [U2] Consuming web services I do the same as Ben but with .NET: -- Local U2 .NET client web service Remote server Pick is a database server, not a communications protocol end-point. We simply should not be doing direct comms from this platform anymore, given the huge number of mainstream options. This is coming from someone who has written all of the bi-directional comms interfaces to/from MV with sockets and cURL and plugins and anything else I could create - because (in the 90's) people said it wasn't possible and because I figured it would be kewl just to do it. It _is_ all technically possible, but that doesn't mean it makes good business sense: Doing things like this in BASIC creates a maintenance hassle later. It leaves the environment subject to DBMS-specific
Re: [U2] Consuming Web Services
Getting back to the original question: In most, but not all cases, I use: UniVerse = (socket) = .NET Service - Web Service Where the .net service is not necessarily on the same machine as UniVerse (security/firewalling restrictions). Using UniVerse sockets to drive services is something I do in a lot of my applications (mvPDF, mvDistributor etc.) and works well. I also have routines for calling web services directly over UniVerse sockets, but you don't get the advantages of WSDL import and XML/JSON serialization built into .Net. The CallHTTP stuff is a pretty rubbish set of functions over the top of the socket API and XDOM is horrible so it's easier and more flexible to do your own if you're going that way :) As for the other stuff: we all love the u2 model (or we wouldn't be here) but that shouldn't blind us to the fact that there are areas where the implementation could be improved: indexing is one of those. Using cache better is probably another, but restricted by the fact that this is a multi-process model: caching would be much easier for single process, multithreaded servers that don't have to load shared memory segments to make data visible to all participants. In a replication scenario, for example, being able to replicate directly to a subscribers' cache could have advantages if you don't need the subscriber to guarantee that it has committed to disk in good time (unless it suddenly becomes a publisher/fail-over when it has to sync the cache).. Brian ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Consuming web services
createRequest setRequestHeader addRequestParameter SubmitRequest All unibasic functions for working with http -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Jeff Schasny Sent: 03 October 2012 21:56 To: U2-Users@listserver.u2ug.org Subject: [U2] Consuming web services I know we have both the SOAP and RESTful web services development for publishing web services from Universe but how are folks consuming other peoples web services into the database? Are there tools for this or am I going to just open a socket, read, and parse 'till I'm blue in the face? -- Jeff Schasny - Denver, Co, USA jschasny at gmail dot com ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Consuming Web Services
I don't understand what's wrong with indexing, can you clarify this point, and I'll wipe out a fix in three days :) -Original Message- From: Brian Leach br...@brianleach.co.uk To: 'U2 Users List' u2-users@listserver.u2ug.org Sent: Thu, Oct 4, 2012 4:05 am Subject: Re: [U2] Consuming Web Services Getting back to the original question: In most, but not all cases, I use: UniVerse = (socket) = .NET Service - Web Service Where the .net service is not necessarily on the same machine as UniVerse (security/firewalling restrictions). Using UniVerse sockets to drive services is something I do in a lot of my applications (mvPDF, mvDistributor etc.) and works well. I also have routines for calling web services directly over UniVerse sockets, but you don't get the advantages of WSDL import and XML/JSON serialization built into .Net. The CallHTTP stuff is a pretty rubbish set of functions over the top of the socket API and XDOM is horrible so it's easier and more flexible to do your own if you're going that way :) As for the other stuff: we all love the u2 model (or we wouldn't be here) but that shouldn't blind us to the fact that there are areas where the implementation could be improved: indexing is one of those. Using cache better is probably another, but restricted by the fact that this is a multi-process model: caching would be much easier for single process, multithreaded servers that don't have to load shared memory segments to make data visible to all participants. In a replication scenario, for example, being able to replicate directly to a subscribers' cache could have advantages if you don't need the subscriber to guarantee that it has committed to disk in good time (unless it suddenly becomes a publisher/fail-over when it has to sync the cache).. Brian ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Consuming Web Services
The closest we get is we have a WCF service host that maintains a block of logins for all of our .net apps that use UniObjects and handles running the subs through the service reference. -Original Message- From: Brian Leach br...@brianleach.co.uk To: 'U2 Users List' u2-users@listserver.u2ug.org Sent: Thu, Oct 4, 2012 4:05 am Subject: Re: [U2] Consuming Web Services Getting back to the original question: In most, but not all cases, I use: UniVerse = (socket) = .NET Service - Web Service Where the .net service is not necessarily on the same machine as UniVerse (security/firewalling restrictions). Using UniVerse sockets to drive services is something I do in a lot of my applications (mvPDF, mvDistributor etc.) and works well. I also have routines for calling web services directly over UniVerse sockets, but you don't get the advantages of WSDL import and XML/JSON serialization built into .Net. The CallHTTP stuff is a pretty rubbish set of functions over the top of the socket API and XDOM is horrible so it's easier and more flexible to do your own if you're going that way :) As for the other stuff: we all love the u2 model (or we wouldn't be here) but that shouldn't blind us to the fact that there are areas where the implementation could be improved: indexing is one of those. Using cache better is probably another, but restricted by the fact that this is a multi-process model: caching would be much easier for single process, multithreaded servers that don't have to load shared memory segments to make data visible to all participants. In a replication scenario, for example, being able to replicate directly to a subscribers' cache could have advantages if you don't need the subscriber to guarantee that it has committed to disk in good time (unless it suddenly becomes a publisher/fail-over when it has to sync the cache).. Brian ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
[U2] Consuming web services
I know we have both the SOAP and RESTful web services development for publishing web services from Universe but how are folks consuming other peoples web services into the database? Are there tools for this or am I going to just open a socket, read, and parse 'till I'm blue in the face? -- Jeff Schasny - Denver, Co, USA jschasny at gmail dot com ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Consuming web services
Starting at UniData 7.3.0 and in UniVerse 11.1.9, we now have U2 Dynamic Objects which does help with the parsing 'till I'm blue in the face when consuming JSON requests. Regards, Dan -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Jeff Schasny Sent: Wednesday, October 03, 2012 2:56 PM To: U2-Users@listserver.u2ug.org Subject: [U2] Consuming web services I know we have both the SOAP and RESTful web services development for publishing web services from Universe but how are folks consuming other peoples web services into the database? Are there tools for this or am I going to just open a socket, read, and parse 'till I'm blue in the face? -- Jeff Schasny - Denver, Co, USA jschasny at gmail dot com ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Consuming web services
We use a middle layer written in Java. On Wed, 2012-10-03 at 14:56 -0600, Jeff Schasny wrote: I know we have both the SOAP and RESTful web services development for publishing web services from Universe but how are folks consuming other peoples web services into the database? Are there tools for this or am I going to just open a socket, read, and parse 'till I'm blue in the face? -- Ben Souther Manager, Web and Web Service Development bsout...@fwdco.com | 508.927.8147 FWDavison Company, Inc. 50 Resnik Road, Suite 200 Plymouth, MA 02360-7220 CONFIDENTIALITY NOTICE: This e-mail message, and any accompanying documents, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure, distribution or copying is prohibited. If you are not the intended recipient, please contact our office by e-mail or by telephone at (508) 747-7261 and immediately destroy all copies of the original message. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Consuming Web Services
How about... [Browser] - |ColdFusion] - [UniVerse] |__| - [Other SQL] --Bill -Original Message- Sent: Wednesday, October 03, 2012 4:56 PM I know we have both the SOAP and RESTful web services development for publishing web services from Universe but how are folks consuming other peoples web services into the database? Are there tools for this or am I going to just open a socket, read, and parse 'till I'm blue in the face? Jeff Schasny - Denver, Co, USA jschasny at gmail dot com ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Consuming web services
There are a whole suite of BASIC verbs and functions for consuming web services, like SOAPCreateRequest, SOAPSubmitRequest, etc. It's not a terribly difficult thing to implement a web service consumer. Larry Hiscock Western Computer Services -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Jeff Schasny Sent: Wednesday, October 03, 2012 1:56 PM To: U2-Users@listserver.u2ug.org Subject: [U2] Consuming web services I know we have both the SOAP and RESTful web services development for publishing web services from Universe but how are folks consuming other peoples web services into the database? Are there tools for this or am I going to just open a socket, read, and parse 'till I'm blue in the face? -- Jeff Schasny - Denver, Co, USA jschasny at gmail dot com ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Consuming web services
Since I'm on Linux, I use wget to access the webservice (can also handle SOAP headers too), then Capture the output...and parse away. I've written some helpful internal parsing tools for XML - but it requires you to know what your looking for. You can parse for a value like topnode1node2node3elementname and it will drill down and return the value. Or just cut out node1 from top. Also have a tagcount routine to let me know how many times a node repeats Then I can loop through, and pull a specific node as it's own XML, then just parse that one. Right now, the XML parser is for very straightforward XML. It might work with namespaces. George -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Jeff Schasny Sent: Wednesday, October 03, 2012 4:56 PM To: U2-Users@listserver.u2ug.org Subject: [U2] Consuming web services I know we have both the SOAP and RESTful web services development for publishing web services from Universe but how are folks consuming other peoples web services into the database? Are there tools for this or am I going to just open a socket, read, and parse 'till I'm blue in the face? -- Jeff Schasny - Denver, Co, USA jschasny at gmail dot com ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Consuming web services
Hi Jeff, We have UniData, not Universe, but here is a sample code snippet I was playing with for consuming a SOAP web service in a UniBasic subroutine. It uses SOAPCreateRequest to manage the actual communication. This is based on some sample code I found with a google search, but I don't remember exactly which site I found the example on. It may have been http://permalink.gmane.org/gmane.comp.db.u2.general/52800 but I might be mistaken. Anyway, my test web service expected a product key as a parameter, then looked up that key in a database and returned a product description. It's a really basic example, but I hope it helps. Cheers, Jim Stoner ** $INCLUDE /usr/ud72/sys/INCLUDE/XML.H Desc= NodeName= TextName= RespHeaders = '' RespData = '' SoapStatus = '' URL = http://insert-url-for-wsdl SoapAction = GETDESC ;* replace with the web service action SoapNS = urn:DefaultNamespace ;* replace with the web service name space SoapMethod = GETDESC ;* replace with the web service method SoapParams = KEY:@VM:A.KEY:@VM:xsd:string ;* replace with the required parameters for the web service call. In my test case, the subroutine is passed in a single value which gets sent to the web service as a string param called KEY Timeout = 3 * create the request IF SOAPCreateRequest(URL, SoapAction, SoapReq) 0 THEN PRINT SOAPCreateRequest failed. RETURN END Status = SOAPSetParameters(SoapReq, SoapNS, SoapMethod, SoapParams ) Status = SOAPSubmitRequest(SoapReq, Timeout, RespHeaders, RespData, SoapStatus) PRINT SOAPSubmitRequest Status: : Status RespData = RespData:CHAR(10) ;* add a line-feed at the end of the xml string PRINT Response: : RespData IF XDOMOpen(RespData, XML.FROM.STRING, domFile) XML.SUCCESS THEN ;* domh is the document handle PRINT XDOMOpen failed. RETURN END IF XDOMLocate(domFile, '//GETDESCReturn', '', nodeHandle) XML.SUCCESS THEN PRINT XDOMLocate failed. RETURN END Status=XDOMGetNodeName(nodeHandle, NodeName) PRINT Node Name: : NodeName IF XDOMLocateNode(nodeHandle, XDOM.CHILD, XDOM.FIRST.CHILD, XDOM.TEXT.NODE , textHandle) XML.SUCCESS THEN PRINT XDOMLocateNode failed. RETURN END IF XDOMGetNodeValue(textHandle, Desc) XML.SUCCESS THEN PRINT XDOMGetNodeValue failed. RETURN END PRINT Returned Description: : Desc RETURN ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Consuming web services
I do the same as Ben but with .NET: -- Local U2 .NET client web service Remote server Pick is a database server, not a communications protocol end-point. We simply should not be doing direct comms from this platform anymore, given the huge number of mainstream options. This is coming from someone who has written all of the bi-directional comms interfaces to/from MV with sockets and cURL and plugins and anything else I could create - because (in the 90's) people said it wasn't possible and because I figured it would be kewl just to do it. It _is_ all technically possible, but that doesn't mean it makes good business sense: Doing things like this in BASIC creates a maintenance hassle later. It leaves the environment subject to DBMS-specific issues that often aren't addressed for quite a long time. Using mainstream tools vastly increases the resources available for getting help and solving problems. So these days I get MV to push a query or data payload out to a middle-tier that uses the latest communications methods available. The request goes out, the response comes back, it all just works - that That should be our bottom line here. HTH T From: Ben Souther We use a middle layer written in Java. Jeff Schasny wrote: I know we have both the SOAP and RESTful web services development for publishing web services from Universe but how are folks consuming other peoples web services into the database? Are there tools for this or am I going to just open a socket, read, and parse 'till I'm blue in the face? ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Consuming Web Services
Tony and Ben are right on... Invoking Stephen Colbert's concept of... The Word... the word is... MiddleWare. As legacy U2 was ahead of its time... by building-in middleware technologies... it is easy to lose sight of what belongs where. --Bill -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Tony Gravagno Sent: Wednesday, October 03, 2012 6:31 PM Subject: Re: [U2] Consuming web services I do the same as Ben but with .NET: -- Local U2 .NET client web service Remote server Pick is a database server, not a communications protocol end-point. We simply should not be doing direct comms from this platform anymore, given the huge number of mainstream options. This is coming from someone who has written all of the bi-directional comms interfaces to/from MV with sockets and cURL and plugins and anything else I could create - because (in the 90's) people said it wasn't possible and because I figured it would be kewl just to do it. It _is_ all technically possible, but that doesn't mean it makes good business sense: Doing things like this in BASIC creates a maintenance hassle later. It leaves the environment subject to DBMS-specific issues that often aren't addressed for quite a long time. Using mainstream tools vastly increases the resources available for getting help and solving problems. So these days I get MV to push a query or data payload out to a middle-tier that uses the latest communications methods available. The request goes out, the response comes back, it all just works - that That should be our bottom line here. HTH T From: Ben Souther We use a middle layer written in Java. Jeff Schasny wrote: I know we have both the SOAP and RESTful web services development for publishing web services from Universe but how are folks consuming other peoples web services into the database? Are there tools for this or am I going to just open a socket, read, and parse 'till I'm blue in the face? ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Consuming web services
I use either Python, Perl or PHP to broker the SOAP connection and pass info back through either statefile temp files or via stdio using EXECUTE's I/O handling. You could also pass data via pipes but it can get messy. Statefile temp files allow for queuing of transactions that don't have to be real time. Check out the Validating Addresses article I wrote for one of the Jan/Feb issues of Spectrum. (I don't remember the year! I must be getting old!) Regards, On 10/3/2012 4:56 PM, Jeff Schasny wrote: I know we have both the SOAP and RESTful web services development for publishing web services from Universe but how are folks consuming other peoples web services into the database? Are there tools for this or am I going to just open a socket, read, and parse 'till I'm blue in the face? -- Glen Batchelor IT Director/CIO/CTO All-Spec Industries phone: (910) 332-0424 fax: (910) 763-5664 E-mail: glen.batche...@all-spec.com Web: http://www.all-spec.com Mobile: http://m.all-spec.com Blog: http://blog.all-spec.com 24-hour Automated Voice Response. Get order status and tracking information 24-hours a day from any touch-tone phone. Call now: 877.404.6165 [910.550.2220] (you will need your 6-digit order# and the ship-to postal code of that order) D U Txt? Get order and tracking info via SMS/Text. Add your mobile# to MyAccount to activate. Text your 6-digit order# to 910.550.2220 to get order status. Text track, space and your 6-digit order# to 910.550.2220 for latest tracking info. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
