Re: [U2] UniVerse backups to disk
On 22/08/11 14:56, Robert Porter wrote: The snapshot disk space only needs to hold the amount of the changed data - not the whole filesystem. To the applications, it appears that a copy was made, but actually writes are being held behind the scenes. Don't think I'm explaining this well (Monday am), so lets try an example. Say that the UV DB is 10GB and it will take 1 hour to back it up. During 1 hour, only 1GB of data will be changed. The snapshot LV needs to be 1GB + a little for overhead - not 10GB. If you do a df while snapshot'ed it appears that 2 10GB partitions are mounted - the original and the snapshot. But the snapshot LV is actually only holding the pending writes. Unmounting the snapshot writes all the data (in order) to the real LV. While snapshot'ed the OS knows what's changed and what hasn't and UV never knows the difference. Interesting. The reason I suggested breaking the mirror was that mirroring is a common technique. As for snapshotting, there's a new linux btree-based file system on the way. It's completely copy-on-write, so effectively it's always taking snapshots until it runs out of disk and needs to delete old snapshots to make room. I don't know much about it, but I know the idea has been around for ages. There was a filesystem called TuxFS that did this, but that never made it out of beta. I think the new one owes a lot of its ideas but none of its code to TuxFS. Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] UniVerse backups to disk
Snapshotting doesn't get rid of mirroring just the need to break/merge them. I'd still suggest using mirrors. The risk of disk failure is too great. Guess you could use some other level of RAID to get there but it's hard to beat spindles plus mirrors (0+1) for databases. In fact our snapshot logical volumes are striped and mirrored as well. The snapshot volume is holding the writes, you have the same risk of a disk failure there. On the Linux front, are you talking about Btrfs (aka Better FS)?I've read some good things, but haven't gotten the nerve to play with it yet. Wols Lists antli...@youngman.org.uk 8/23/2011 6:28 AM On 22/08/11 14:56, Robert Porter wrote: ... Interesting. The reason I suggested breaking the mirror was that mirroring is a common technique. ... ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] [UV] Microsoft Team Foundation Server for Source Control
Thanks to everyone for some very interesting points. I am particularly intrigued by Stuart's handling of dictionaries using triggers. One particular challenge I may face is one particular hashed file (not a dictionary) which contains 2 1/2 million records that are under source control today. Copying them out to a directory might pose some problems with inode consumption. Perry -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Boydell, Stuart Sent: Friday, August 19, 2011 11:10 PM To: U2 Users List Subject: Re: [U2] [UV] Microsoft Team Foundation Server for Source Control Hi Perry, I use Perforce (P4) - similar (but different). Not sure what best practice is for TFS repositories is but with P4 and other Source Control (SC) systems something like the following should work. //function/branch/path/item 1. Starting with a functional area - a project or a defined set of artefacts which make a up a particular system (eg Payroll) 2. branch - this is usually MAIN (eg the current, main or head version) - we occasionally have different branch versions for Australia NZ or for a release branch. A commercial software developer would probably keep their major releases in different branches. 3. path - you should probably just start at the account directory. So if your complete path to the payroll account was: /usr/uvaccs/dev/perry/payroll/.. you would start at the *../payroll/..* level of the path and ignore the stuff before it. Not sure how TFS maps to the workspace, but I guess that you are able to give it a relative path like most SC systems. 4. The item under control. So you might end up with //Payroll/MAIN/PAYROLL/PAY.BP/PRINT.PAYSLIPS Also, not sure how TFS handles case sensitivity - this would be an issue if it regards Payslip PAYSLIP as the same item. As far as hashed item go: convert everything you need to control to type 19 directory files (except dictionaries - see below). Why? Because you should only need to do this in dev. For all other environments the items should only be copied in (deployed) from a release file - so will be able to be plonked into hashed files by UV copy. If you can't convert them for some reason (like you have binary in them?!), you'll probably need to put a trigger onto your hashed files to manage the controlled items. Dictionaries - have to be hashed because otherwise I-types won't compile or work. So, I have a trigger for the dictionary that writes out the item to a type 19 file under source control. It truncates the I-type attributes after about att 10 so you don't get the binary in the repository. It also manages which SB+ items get put in and out of SC. If the item is not checked out from SC for edit before you try to edit the dictionary, the trigger will rollback the copy and gracefully tell you that you need to check out the item before editing it. On save it updates the type 19 copy after which it needs to be checked in. In my case I have 1 file set up for all the dictionary items. I use an assigned key and a lookup file to manage them. The thinking behind it was to reduce the number of folders to manage in SC. In practice it probably doesn't matter, if you have lots of dictionary folders or one folder with lots of items as we do. Release strategy. I use a program, but we also looked at make and the Perforce equivalent. These are nice because they can check dependencies - however, it was overkill for us so I just have a program that copies everything from a 'drop' location into the target system and compiles and runs any setup programs. Hope that helps, Stuart -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Perry Taylor Sent: Saturday, 20 August 2011 00:57 To: U2-Users List Subject: [U2] [UV] Microsoft Team Foundation Server for Source Control Has anyone had any experience using Microsoft's Team Foundation Server for source control with UniVerse on a Linux server? I have the command line client functional and talking to the TFS server. I know I'll have to write some kind of interlude to manage those items in hashed files to get them out into the file system where they will be visible to the TFS client and to do the reverse upon checkout. What I'm looking for are some ideas for organizing in the TFS repository. Also, we're looking for a one-button deployment solution to be able to deploy our Windows/.NET software to the respective Windows servers along with the UniVerse software to the UniVerse server(s), run processes to create/delete files, index, etc. and compile and catalog BASIC programs. I know I'll probably have to build this thing to make this happen as I seriously doubt there is anything available off the shelf capable of doing this. Anyone been down this path? Thanks. Perry Taylor Zirmed, Inc. CONFIDENTIALITY NOTICE: This e-mail
Re: [U2] UniVerse backups to disk
Here is some Pie in the Sky info for ya... btrfs or Butter FS is being developed by Oracle for Linux. Its one of the reasons Oracle shutdown the development of ZFS when they bought Sun. (ZFS has many of those snapshotting features you are talking about). So now you have all of these spinoffs for ZFS with companies like: http://nexenta.com/corp/ http://www.ixsystems.com/ix/storage/titan-truenas-pro Nexenta is based on a Solaris kernel (with Debian's package management) TrueNAS is based on a BSD kernel with ZFS implemented there. ZFS never made it into the Linux kernel because of licensing issues... although there is still some attempt to do it. Apple at one point was talking about it, but, they shut the project down. So, btrfs, is the attempt to get those features you guys are talking about into the Linux kernel, and because its oracle, they will eventually do it, I'm sure (there is enough money behind it) However, as all filesystem development goes, they have been working on btrfs for some years now... and at present, it does not have a filesystem checker (fsck) that can fix errors. So its not recommend for production use. Unless of course they have just released it. I played with btrfs in its infancy a couple of years ago... at a time when you could not even boot from it. It was very finnicky, as anything new is sometimes. I use Nexenta free version for a couple of video cameras I got saddled into providing storage for. It was cheap enough to setup a spare machine I had with it. As far as using either for U2, Much testing and documenting would be required me thinks On Tue, Aug 23, 2011 at 10:30 AM, Robert Porter ropor...@ochsner.orgwrote: Snapshotting doesn't get rid of mirroring just the need to break/merge them. I'd still suggest using mirrors. The risk of disk failure is too great. Guess you could use some other level of RAID to get there but it's hard to beat spindles plus mirrors (0+1) for databases. In fact our snapshot logical volumes are striped and mirrored as well. The snapshot volume is holding the writes, you have the same risk of a disk failure there. On the Linux front, are you talking about Btrfs (aka Better FS)?I've read some good things, but haven't gotten the nerve to play with it yet. Wols Lists antli...@youngman.org.uk 8/23/2011 6:28 AM On 22/08/11 14:56, Robert Porter wrote: ... Interesting. The reason I suggested breaking the mirror was that mirroring is a common technique. ... ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users -- John Thompson ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] UniVerse backups to disk
On a relevant note. I was always told that: The best way to get an accurate backup on U2 is to: -Pause the writes -Do a logical volume snapshot (*nix only) -Resume the writes -Let the logical volume snapshot finish copying off to your backup space to disk. -On top of that you should still do a regular normal backup (like uvbackup, or tar, dd, or whatever) overnight, as snapshots were never intended to be a Full Backup solution. As far as windows is concerned I can't really say, other than, my experience with Symantec (Vertias) Backup Exec was always a horrible experience. So use something else. StorageCraft, or ComVault, I've heard good things about. StorageCraft being the way cheaper option. On Tue, Aug 23, 2011 at 1:16 PM, John Thompson jthompson...@gmail.comwrote: Here is some Pie in the Sky info for ya... btrfs or Butter FS is being developed by Oracle for Linux. Its one of the reasons Oracle shutdown the development of ZFS when they bought Sun. (ZFS has many of those snapshotting features you are talking about). So now you have all of these spinoffs for ZFS with companies like: http://nexenta.com/corp/ http://www.ixsystems.com/ix/storage/titan-truenas-pro Nexenta is based on a Solaris kernel (with Debian's package management) TrueNAS is based on a BSD kernel with ZFS implemented there. ZFS never made it into the Linux kernel because of licensing issues... although there is still some attempt to do it. Apple at one point was talking about it, but, they shut the project down. So, btrfs, is the attempt to get those features you guys are talking about into the Linux kernel, and because its oracle, they will eventually do it, I'm sure (there is enough money behind it) However, as all filesystem development goes, they have been working on btrfs for some years now... and at present, it does not have a filesystem checker (fsck) that can fix errors. So its not recommend for production use. Unless of course they have just released it. I played with btrfs in its infancy a couple of years ago... at a time when you could not even boot from it. It was very finnicky, as anything new is sometimes. I use Nexenta free version for a couple of video cameras I got saddled into providing storage for. It was cheap enough to setup a spare machine I had with it. As far as using either for U2, Much testing and documenting would be required me thinks On Tue, Aug 23, 2011 at 10:30 AM, Robert Porter ropor...@ochsner.orgwrote: Snapshotting doesn't get rid of mirroring just the need to break/merge them. I'd still suggest using mirrors. The risk of disk failure is too great. Guess you could use some other level of RAID to get there but it's hard to beat spindles plus mirrors (0+1) for databases. In fact our snapshot logical volumes are striped and mirrored as well. The snapshot volume is holding the writes, you have the same risk of a disk failure there. On the Linux front, are you talking about Btrfs (aka Better FS)?I've read some good things, but haven't gotten the nerve to play with it yet. Wols Lists antli...@youngman.org.uk 8/23/2011 6:28 AM On 22/08/11 14:56, Robert Porter wrote: ... Interesting. The reason I suggested breaking the mirror was that mirroring is a common technique. ... ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users -- John Thompson -- John Thompson ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
[U2] callHTTP creating Header
in a header with post-data I have to create the header-information like this: Content-type: text/xml;charset=UTF-8 Accept: text/xml, multipart/related, text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 When creating it with setRequestHeader or with addRequestParamete I will receive the error 415 (Unsupported media type) from the server. How can I create the correct header? Thanks in advance Karl-Heinz ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] [UV] Microsoft Team Foundation Server for Source Control
What source are you keeping that has 2.5 million records -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Perry Taylor Sent: Wednesday, 24 August 2011 01:39 To: U2 Users List Subject: Re: [U2] [UV] Microsoft Team Foundation Server for Source Control Thanks to everyone for some very interesting points. I am particularly intrigued by Stuart's handling of dictionaries using triggers. One particular challenge I may face is one particular hashed file (not a dictionary) which contains 2 1/2 million records that are under source control today. Copying them out to a directory might pose some problems with inode consumption. Perry ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] [UV] Microsoft Team Foundation Server for Source Control
Not source but code lists which are updated at different intervals. Some examples: ZIP, CPT, ICD9, etc. - Original Message - From: Boydell, Stuart [mailto:stuart.boyd...@spotless.com.au] Sent: Tuesday, August 23, 2011 07:49 PM To: U2 Users List u2-users@listserver.u2ug.org Subject: Re: [U2] [UV] Microsoft Team Foundation Server for Source Control What source are you keeping that has 2.5 million records -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Perry Taylor Sent: Wednesday, 24 August 2011 01:39 To: U2 Users List Subject: Re: [U2] [UV] Microsoft Team Foundation Server for Source Control Thanks to everyone for some very interesting points. I am particularly intrigued by Stuart's handling of dictionaries using triggers. One particular challenge I may face is one particular hashed file (not a dictionary) which contains 2 1/2 million records that are under source control today. Copying them out to a directory might pose some problems with inode consumption. Perry ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. ZirMed, Inc. has strict policies regarding the content of e-mail communications, specifically Protected Health Information, any communications containing such material will be returned to the originating party with such advisement noted. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] [UV] Microsoft Team Foundation Server for Source Control
Hi Perry, I'd strongly suggest normal backup would be a better strategy for those items. Source Control is really designed for those things that you use to build a system. Code, parameters, schema (file dictionary) generation scripts, etc. The data for that system would typically *not* be kept in source control (unless you're feeling masochistic ;). Just in case you haven't seen this: http://tfsguide.codeplex.com/ has guidelines for what you want to do and which is TFS specific. Also, from what I just read, TFS is not case sensitive which will be an issue if you ever have 2 items under control with differently cased equivalent names. Payroll.Rpt PAYROLL.RPT -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Perry Taylor Sent: Wednesday, 24 August 2011 10:50 To: 'u2-users@listserver.u2ug.org' Subject: Re: [U2] [UV] Microsoft Team Foundation Server for Source Control Not source but code lists which are updated at different intervals. Some examples: ZIP, CPT, ICD9, etc. - Original Message - From: Boydell, Stuart [mailto:stuart.boyd...@spotless.com.au] Sent: Tuesday, August 23, 2011 07:49 PM To: U2 Users List u2-users@listserver.u2ug.org Subject: Re: [U2] [UV] Microsoft Team Foundation Server for Source Control What source are you keeping that has 2.5 million records -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Perry Taylor Sent: Wednesday, 24 August 2011 01:39 To: U2 Users List Subject: Re: [U2] [UV] Microsoft Team Foundation Server for Source Control Thanks to everyone for some very interesting points. I am particularly intrigued by Stuart's handling of dictionaries using triggers. One particular challenge I may face is one particular hashed file (not a dictionary) which contains 2 1/2 million records that are under source control today. Copying them out to a directory might pose some problems with inode consumption. Perry ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. ZirMed, Inc. has strict policies regarding the content of e-mail communications, specifically Protected Health Information, any communications containing such material will be returned to the originating party with such advisement noted. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
