Hello Doug I am sorry if you were upset by my reply.
The move to MasterMap includes a rich set of attribute data - but a large number of users ( the majority ) will still primarily be interested in using the maps for a reference to position their own layers. The rich attribution will help to split the objects over multiple layers and to set the style of the objects. In such cases "drawing the stuff" is the essence of their proposed usage along with usage of the info tool. Vinnys original question related to the size limit problem and in particular with regard to MasterMap. I have pointed out that the DAT file does not need to be big, and thus the 2GB limit should affect the MAP file first. If you think that a high percentage of users will make SQL queries against non spatial data in the current topological layers and then further NOT want to view them then we also disagree on another point! You suggest my answer was misleading - again I disagree. As a MapInfo (only) Partner I am much more concerned by your misleading comments quoted below:- "You do need to be wary of creating the large MI files - they become extremely un-user friendly! eg. slow drawing " I think it is you misleading the list - it is not slow drawing - it is a superb graphics format and only now is Oracle 9i coming close for speed of drawing. Therefore your suggestion that users need to move to SQL Server or Oracle to work with MasterMap is again misleading the list. The MapInfo native format is more than up to the task - where the area covered is County or smaller. With all due respect I did cover both points ie DAT and MAP. With regard to MAP you now seem to be backing off and agree that drawing the graphics with MapInfo is not a problem. With regard to the DAT file you have misunderstood me. I pointed out that the DAT should be held relationally instead of in a flat file. Vinny is storing strings such as "rock;scrub;coniferous tree" in a single field and as you now say running SQL queries on this would be a nightmare. You would really need to use an Instr() type call and pass a lot of data from server to client. However if the data is stored relationally the DAT files can be very small - thus moving the file size problem to the MAP file ( graphics) and not the DAT file. I think with MasterMap data stored boils down to what sort of SQL query the users will want to make (IF ANY!) and what they need to retrieve through the info tool. One option we include is to turn the string round to be the field name so that the field content becomes true or false. For example instead of storing field MAKE to be "Manmade" store a field MANMADE to be TRUE or FALSE. This is a single byte, instead of a string of say 20 bytes. Also an SQL query on this would be much faster and a thematic on such field values would be possible but impossible within the concatenation Vinny is using. NOt all users will want to know the same attributes and the published layers would ideally be tailored to end users needs. As mentioned above, I think it very unlikely that many users will run SQL queries on data attributes in MasterMap. By colouring objects appropriately from data attributes you effectively have a thematic map removing many users need for data attributes. The most likely use of the GML tags will be with the info tool - for example the new addresspoint theme gives the address. The info tool will give us the house number etc. Not many people will want to select all houses where the house number = "71" for example but they will want to use info on the data. I have posted this reply back to the list as you put yours there and I think the list should be aware of the strengths of the product the list is focused on. The original question was to do with file size limit and several us have now answered that in detail. I would suggest if you want to discuss this in any greater detail we take it off line as our personal differences as to the relative merits of MapInfo/ESRI are not relevant to this list. Regards Bob In message <[EMAIL PROTECTED]>, GIS Coordinated <[EMAIL PROTECTED]> writes >Bob >try your sql queries on a 2gig dat file - its just half the story to draw >the stuff! (often users dont even need/or want to draw anything - just >query). This was my real general point. Users out there reading this list >will think they can just throw everything in a huge pile and expect great >performance as you suggest below (which is ok for drawing) but you should >always consider what exactly you want to do with the data, and then >carefully determine the best way to store it. > >regards > > >Doug > > >-----Original Message----- >From: bob young [mailto:[EMAIL PROTECTED]] >Sent: 20 September 2002 16:13 >To: Vinesh Govind >Cc: MapInfo Lists (E-mail) >Subject: Re: MI-L MapInfo file size limit > > >Hi Vinny > >Thanks for the reply. However if the data is stored relationally this >will not be a problem. > >The updates include "departed features" as well as new features. If your >data is split into current holding ( ie whats on the ground now ) and >history ( ie previous versions ) then the current holding can only >increase for the following reasons. > >The OS continue to refine the land usage. eg a 1 hectare field that is >described as one parcel in terms of geometry but data describes it as >containing coniferous and rock is sub divided into 3 0.3 hectare >polygons with more specific attribution. > >The solution to above is more MapInfo tables to hold the output. ie the >1 acre field would be on one layer. The three subdivisions could be on >three layers for natural, rock and coniferous land types. > >The increase in people living alone and the inevitable development of >green field sites leads to more toids ( ie geometric features). > >This problem will mean a creep towards the 2 GB limit. However if you >are currently around the 400GB per table that leaves a massive amount of >loss of countryside before you need worry about the 2 GB limit! > >Also the largest theme is natural land. The building layer has more >spare capacity at present. Building development will not increase what >at present is the largest layer. ( I doubt if it will reduce it either - >although area will reduce the coordinates will probably not ). > >I think the main thing to remember is that change only should settle >down to about 2 to 3 % change per year ( a figure I have had from OS ). >Much of this will replace what is there rather than be additional data. > >The Positional Accuracy Improvement program in certain areas will lead >to large volumes of change but should not increase your files sizes. One >set of geometry will be replaced with a more accurate set of geometry >but virtually the same number of coordinates. > >Although it is safe ( and advisable ) to remove attribution from your >published maps try and make sure your model stores full attribution >somewhere. Although you at present might think certain attribution is >not needed, future users might feel differently! If you are applying >update only and your model only carries forward reduced attribution this >might be a problem. I would recommend with "change only update" using a >local holding storing full attribution from which reduced attribution >layers for MapInfo can be derived for your current end users. This >approach gives you the best of both worlds. > >Regards > > >Bob > > > >In message <397693EFEA62D411806E00B0D04972310134F7C5@LCCNTEX3>, Vinesh >Govind <[EMAIL PROTECTED]> writes >> >> Bob >> >> Thanks very much for the informative reply. >> >> I should say from the outset that we have not yet reached the 2GB >> file size limit and more importantly our file sizes should >> gradually diminish as we remove some of the atrribute data that is >> not of any relevance to us. >> >> However, my concern is that as one receives "updates" to MasterMap, >> file sizes will gradually grow and the 2GB limit could get >> precariously close. If this is the case, then a further concern I >> have is that companies are selling translators for MasterMap but >> are not warning users of a future potential problem that could >> arise. >> >> My apologies to non UK listers who find this topic irrelevant. >> >> Thanks. >> >> Vinny >> >> -----Original Message----- >> From: bob young [mailto:[EMAIL PROTECTED]] >> Sent: 20 September 2002 14:27 >> To: Vinesh Govind >> Cc: MapInfo Lists (E-mail) >> Subject: Re: MI-L MapInfo file size limit >> >> >> Hi Vinny >> >> The ID file contains a list of 4 byte pointers into the MAP file >> thus >> allowing record n to find map object n. >> >> a 4 byte long will store +- 2GB and MapInfo have used a signed long >> thus >> limiting themselves to half of this. Therefore even on systems(eg >> Win2000) which allow files to be the size of the drive the >> addressing >> in the ID and MAP file limit the MAP file size to 2GB. >> >> I have asked senior staff at the MapInfo European Conference in >> Cannes >> this year if there are any plans to move to larger files and they >> certainly have not ruled it out. There is obviously strong support >> for >> Oracle 9i but many of the techies would like to improve the >> existing TAB >> format. >> >> I disagree totally with Doug that at 2 GB the MAP file gives bad >> performance. Quite the reverse. If you compare it with ArcView SHP >> file >> format with small coverages you will not see much difference >> between MAP >> and SHP. The larger the coverage the more dramatic the improvements >> Mapinfo R tree gives you compared to SHP. MapInfo reads fixed size >> index >> blocks of data from the drive to find map features. For just one >> extra >> block read it can read a file 25 times bigger. Lets say a depth of >> 4 >> reads gives you access to 1 MB then 5 reads will give you access to >> 25 >> MB and 6 reads gives access to 625 MB. This is probably the >> greatest >> feature of MapInfo - that it does handle very large MAP files >> exceptionally well. >> >> I personally believe MapInfo should re engineer their excellent >> format >> to make use of int64 ( 8 byte pointers ) on Win 2000 and beyond. >> >> Having said that I am surprised that the 2 GB limit concerns you at >> Leicestershire. If your Mastermap data is split into layers you >> should >> not be anywhere near this. The only time the 2 GB limit should be >> an >> issue is with coverages much larger than 1 County. eg Wales, >> Scotland or >> National Coverage. If you want to try one of our approaches to >> MasterMap >> translation into MapInfo native format you are welcome to a free >> evaluation. >> >> The answer to National Coverage is the use of seamless layers. In a >> way >> a seamless layer is the extension of the RTree index. First MapInfo >> finds the MBRs of the base tables that intersect the screen. For >> this it >> gets the benefits of the R Tree in the seamless layer. Then only >> for the >> base tables that intersect the screen it uses the bases tables R >> Trees >> to narrow its search to a small part of each base table. Hence >> performance is still very acceptable. The drawback of seamless >> tables is >> you lose the use of data indexes ( the IND files ) and you lose the >> ability to thematically map. Also if on a server network traffic >> will be >> higher as one R Tree index over the top of another R Tree index is >> not >> as efficient as if it was all in one - ie if they did introduce int >> 64 >> there would be less reads to get to the same data. >> >> If your limit is within the DAT file rather than the MAP file you >> should >> really be using relational joins on the GML tags. It is pointless >> holding long strings over and over again in a flat file when the >> same >> result can be achieved by the use of relational tables. The DAT >> files >> can be reduced to a size much smaller than the MAP files. Your >> limit >> should come from the MAP files and if you split these sensibly the >> largest needed for Leicestershire would be well under 1 GB never >> mind 2 >> GB. MapInfo is a full RDBMS. MapInfo is more than a match for >> MasterMap >> data but it now warrants using the full power of an RDBMS whereas >> with >> NTF data we easily got away with flat files as Landline was really >> just >> graphic data not full GIS data. >> >> Hope this helps in resolving your file size problems. Feel free to >> ring >> me if I can be of any help. >> >> I apologise to non UK listers who may think the above is irrelevant >> for >> them. However I do believe as GML becomes more pervasive and data >> attribution increases the question will become more relevant in >> other >> countries. >> >> Regards >> >> >> Bob >> (0044) 1633 881117 >> www.mapsbydesign.co.uk >> >> >> >> In message <397693EFEA62D411806E00B0D04972310134F7C2@LCCNTEX3>, >> Vinesh >> Govind <[EMAIL PROTECTED]> writes >> >Users >> > >> >I consider myself to be an intermediate user of MapInfo software >> so bear >> >with me if my query is too simplistic for some of the MapInfo >> gurus. >> > >> >I've been doing some translation of OS MasterMap data and have >> been told >> >that the maximum file size limit for a MapInfo table is 2GB. >> >Is this correct?......Why? >> >If this is indeed a correct assumption, then are there any plans >> for the >> >future by MapInfo Corp. to increase the file size limit. >> > >> >Thanks. >> > >> >Vinny >> > >> > >> >>_______________________________________________________________________ >> >The contents of this message do not necessarily represent the >> >opinions, views, policy or procedures of Leicestershire County >> Council. >> > >> >> -- >> bob young > >-- >bob young > >--------------------------------------------------------------------- >List hosting provided by Directions Magazine | www.directionsmag.com | >To unsubscribe, e-mail: [EMAIL PROTECTED] >For additional commands, e-mail: [EMAIL PROTECTED] >Message number: 3173 > > > -- bob young --------------------------------------------------------------------- List hosting provided by Directions Magazine | www.directionsmag.com | To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] Message number: 3179
