Re: Longitude Index: B-Tree or Cluster

2020-04-21 Thread Peter Bozek via 4D_Tech
On Tue, Apr 14, 2020 at 11:12 PM Tim Nevels via 4D_Tech < 4d_tech@lists.4d.com> wrote: > > > traversed. That make a perfect sense for foreign key indexes - when > loading > > 1-record, you have immediately a selection of N-records available, and > you > > (almost) never do a search for range of

Re: Longitude Index: B-Tree or Cluster

2020-04-19 Thread John J Foster via 4D_Tech
Hi Bernard, I did find a version with methods in english. But I’ll need to convert the text of the distance form so I can better understand it. Appreciate, John… > John, > > In Math4D, method names are all in English. > All descriptions are in English (and in French…) > > Forms have not

Re: Longitude Index: B-Tree or Cluster

2020-04-18 Thread Bernard Escaich via 4D_Tech
John, In Math4D, method names are all in English. All descriptions are in English (and in French…) Forms have not been translated yet, because some use old technologies as Draw and Graph. You don’t need them, if you have a great number of calculations, because you will use direct calls to the

Re: Longitude Index: B-Tree or Cluster

2020-04-15 Thread John J Foster via 4D_Tech
Hey Tim, > Many years ago at a 4D Summit I remember... Do you have a photographic memory? Good info. Thanks, John... > Many years ago at a 4D Summit I remember someone asking Laurent a question of > how Cluster B-Tree indexes were implemented. He gave a fairly detailed > answer. > >

Re: Longitude Index: B-Tree or Cluster

2020-04-15 Thread John J Foster via 4D_Tech
Hi Bernard, My scenario was based upon the way I saw someone else implement it. And I don’t have miles or kilometers and don't need them for my purposes. Yea it will either be a square or a circle not sure yet. Does anyone know if there is an updated v17 English version of the Math4D

Re: Longitude Index: B-Tree or Cluster

2020-04-14 Thread Bernard Escaich via 4D_Tech
ave split out the longitudes into a text and real component. I have >> formatted the longitudes as “000” and if west of GMT prefixed with “-“ and >> if east prefixed with “+“. I'll do something similar with latitudes and then >> try building some radial searches and see. >>

Re: Longitude Index: B-Tree or Cluster

2020-04-14 Thread Charles Miller via 4D_Tech
Should have been search On Tue, Apr 14, 2020 at 5:33 PM John J Foster via 4D_Tech < 4d_tech@lists.4d.com> wrote: > Hey Chuck, > > Not sure what “hash and stir” means? > > Matching data requires one kind of index for speed and doing a radial > search another. > > In the db where I only have 1/2

Re: Longitude Index: B-Tree or Cluster

2020-04-14 Thread John J Foster via 4D_Tech
Hey Chuck, Not sure what “hash and stir” means? Matching data requires one kind of index for speed and doing a radial search another. In the db where I only have 1/2 million records with longitude as a real the index returns a range quite quickly. So I’ll keep it for the radial search. For

Re: Longitude Index: B-Tree or Cluster

2020-04-14 Thread John J Foster via 4D_Tech
me radial searches and see. > > Since all string is text is there a still a way to tell 4D that a field and > thus an index is a fixed size? > > Anyway we shall see if theory and implementation matches! > > Appreciate, > John… > > > > > On Apr 14, 2020, at 12:00 PM

Re: Longitude Index: B-Tree or Cluster

2020-04-14 Thread Tim Nevels via 4D_Tech
On Apr 14, 2020, at 3:39 PM, Peter Bozek wrote: > Cluster index is suitable when the result of search is a (potentially > large) selection. While B-tree node stores a reference to a record, Cluster > index node stores a selection, so no selection building is needed. However, > if I would design

Re: Longitude Index: B-Tree or Cluster

2020-04-14 Thread Chuck Miller via 4D_Tech
Sure you hash and stir that, when searching for a particular lat/long combination you has entry and search This will not help in distance calculations How about this for a create a concatenated key that has two parts for each Named LatKey Lat_before decimal Lat_after decimal

Re: Longitude Index: B-Tree or Cluster

2020-04-14 Thread Peter Bozek via 4D_Tech
ech-requ...@lists.4d.com wrote: > > > > From: Arnaud de Montard mailto:arn...@init5.fr>> > > Subject: Re: Longitude Index: B-Tree or Cluster > > Date: April 14, 2020 at 2:23:26 AM PDT > > To: 4D iNug Technical <4d_tech@lists.4d.com <mailto:4d_tech@lists.4d.com &g

Re: Longitude Index: B-Tree or Cluster

2020-04-14 Thread John J Foster via 4D_Tech
Hey Chuck, Given my last response and my needs… I am trying to visualize how a hash would work? Can you give me an example using a longitude and latitude? Thanks, John… > You could use a hash and store it. I have a sequence that is too long to > query and that is what I did works like a

Re: Longitude Index: B-Tree or Cluster

2020-04-14 Thread John J Foster via 4D_Tech
Hi Arnaud, Hmmm... I think I would like the data to be obvious to my eyes when I look at it. My initial goal was simply to be able to match up a location with a missing Olson Timezone ID. Import the data, fix the data in the much larger db, and then export it back out to reimport into it’s

Re: Longitude Index: B-Tree or Cluster

2020-04-14 Thread John J Foster via 4D_Tech
@init5.fr>> > Subject: Re: Longitude Index: B-Tree or Cluster > Date: April 14, 2020 at 2:23:26 AM PDT > To: 4D iNug Technical <4d_tech@lists.4d.com <mailto:4d_tech@lists.4d.com>> > > > >> Le 13 avr. 2020 à 19:36, Bernard Escaich via 4D_Tech <4d_tech@l

Re: Longitude Index: B-Tree or Cluster

2020-04-14 Thread Chuck Miller via 4D_Tech
You could use a hash and store it. I have a sequence that is too long to query and that is what I did works like a charm. Regards Chuck Chuck Miller Voice: (617) 739-0306 Informed Solutions, Inc.

Re: Longitude Index: B-Tree or Cluster

2020-04-14 Thread Arnaud de Montard via 4D_Tech
> Le 13 avr. 2020 à 23:51, John J Foster via 4D_Tech <4d_tech@lists.4d.com> a > écrit : > > Hey Bernard, > > I would use text values if I didn’t think I needed to do range searches. It’s > not a “live” db so I can play and experiment to my hearts content. Considering 5 million pairs of

Re: Longitude Index: B-Tree or Cluster

2020-04-14 Thread Arnaud de Montard via 4D_Tech
> Le 13 avr. 2020 à 19:36, Bernard Escaich via 4D_Tech <4d_tech@lists.4d.com> a > écrit : > > Hi John, > > From the top of my head, indexes on long are less efficient than on text > values ; I have a doubt, perhaps only for updating. I'd think the opposite, an index based on a "fixed length"

Re: Longitude Index: B-Tree or Cluster

2020-04-13 Thread John J Foster via 4D_Tech
Hey Bernard, I would use text values if I didn’t think I needed to do range searches. It’s not a “live” db so I can play and experiment to my hearts content. Yea I could format the longitudes as “000” and the latitudes as “00” then they sort in order. Of course I have to keep in mind the

Re: Longitude Index: B-Tree or Cluster

2020-04-13 Thread Bernard Escaich via 4D_Tech
Hi John, From the top of my head, indexes on long are less efficient than on text values ; I have a doubt, perhaps only for updating. You could test adding two fields with longitude and latitude converted in text values with appropriate leading zéros for search ? Cordialement, Bernard Escaich

Re: Longitude Index: B-Tree or Cluster

2020-04-13 Thread John J Foster via 4D_Tech
Hey Bernard, Ah yes the Math4D component. I had forgotten about that! I’ll take a look. If I add a field and split the longitude into degrees I’d likely use a long. I think that would give me the most flexible approach. Then searching for specific values like 115 or -47 would be easy.

Re: Longitude Index: B-Tree or Cluster

2020-04-12 Thread Chuck Miller via 4D_Tech
Cluster is meant when there are many duplicates. To do queries I would make a duplicate field and use a string conversion or a long code version. and not a real. Reals in my opinion are the slowest when it comes to queries. Or manipulation Sent from my iPhone > On Apr 12, 2020, at 6:57 PM,

Re: Longitude Index: B-Tree or Cluster

2020-04-12 Thread Bernard Escaich via 4D_Tech
John, You have geodesic functions in Math4D, component available here : https://forums.4d.com/Post/EN/31847250/1/31847251#31847251 HTH Cordialement, Bernard Escaich > Le 10 avr. 2020 à 02:03, John J Foster via 4D_Tech <4d_tech@lists.4d.com> a > écrit : > > Hi All, > > I have a GEO

Re: Longitude Index: B-Tree or Cluster

2020-04-12 Thread John J Foster via 4D_Tech
Hi Randy, Thanks for the code. As I play with this data I am learning. The proximity approach evolved as I realized it would be nice to find al the citrus within a certain distance. Yea Google does this all the time but I need it to be non-web based. Thus the millions of database records.

Re: Longitude Index: B-Tree or Cluster

2020-04-12 Thread Randy Kaempen via 4D_Tech
John, > On Apr 12, 2020, at 10:33 AM, John J Foster via 4D_Tech > <4d_tech@lists.4d.com> wrote: > > If you happen to find any 4D bearing/distance code (anything related to > searching using longitude/latitude or both) I would enjoy looking at it. I had worked out these calculations with a

Re: Longitude Index: B-Tree or Cluster

2020-04-12 Thread John J Foster via 4D_Tech
Hey Jörg, Thanks! I was able to find a few other NUG entries using the word “bearing” in the search. Challenging for the NUG archive to read my mind and correctly interpret what I am looking for! If you happen to find any 4D bearing/distance code (anything related to searching using

Re: Longitude Index: B-Tree or Cluster

2020-04-10 Thread Jörg Knebel via 4D_Tech
> On 11 Apr 2020, at 12:24 AEST, John J Foster via 4D_Tech > <4d_tech@lists.4d.com> wrote: > > Couldn’t find anything in KB. Here is the post from 155 month and 19 days ago mentioned in the other post: But someone might think the following could be of some help, sorry its not 4D-code.

Re: Longitude Index: B-Tree or Cluster

2020-04-10 Thread Jörg Knebel via 4D_Tech
Hi John, > On 11 Apr 2020, at 12:24 AEST, John J Foster via 4D_Tech > <4d_tech@lists.4d.com> wrote: > > Yes I understand all that and use the longitude to calculate MGT offsets and > Latitiudes for Declinations and so on … > > I am fixing data - matching one set imported data from another

Re: Longitude Index: B-Tree or Cluster

2020-04-10 Thread John J Foster via 4D_Tech
Hi Jörg, Yes I understand all that and use the longitude to calculate MGT offsets and Latitiudes for Declinations and so on … I am fixing data - matching one set imported data from another source - and I also realize that I could use a good proximity routine (using the lon/lat and a distance

Re: Longitude Index: B-Tree or Cluster

2020-04-10 Thread Tom Benedict via 4D_Tech
The Google tells me that you are located in Coffs Harbour, New South Wales, Australia > Exercise: > Name the town where I’m located when my position is: > > - 30.30225° 153.1086° > ** 4D Internet Users Group (4D iNUG)

Re: Longitude Index: B-Tree or Cluster

2020-04-10 Thread Jörg Knebel via 4D_Tech
G’day John, > On 10 Apr 2020, at 10:03 AEST, John J Foster via 4D_Tech > <4d_tech@lists.4d.com> wrote: > > Many of the values are similar like 141.235 or 141 3.23 and so on. Have a look at a world map with a grid and consider this: Geodata are shown in degrees and fractions of it and can be

Re: Longitude Index: B-Tree or Cluster

2020-04-10 Thread John J Foster via 4D_Tech
Hey Kirk, Yea I have been mulling as a possibility. But I really want to put 4D through it’s motions first. Not worried about storage at this time as this is kind of a laboratory for testing large tables. Thanks for your ideas! John... > If absolute raw speed is what you are looking for I

Re: Longitude Index: B-Tree or Cluster

2020-04-10 Thread Kirk Brooks via 4D_Tech
John, If absolute raw speed is what you are looking for I would break each value into whole numbers - the integer and the decimal and store them separately as longints. Index the integer with a b-tree and the decimal with a cluster. You sacrifice space (maybe - haven't really done the math on 2

Re: Longitude Index: B-Tree or Cluster

2020-04-10 Thread John J Foster via 4D_Tech
Hi Tom, I didn’t give a complete scenario which is why the example looks so broad. In this case I have imported data to which I need to match with the correct Olson Time Zone. So in the example searching for Russia (RU) and then the longitude range returns many many records (aka, broad).

Re: Longitude Index: B-Tree or Cluster

2020-04-10 Thread Tom Benedict via 4D_Tech
John, Make sure you include testing a single query, if you haven’t already, then see what 4D’s query analyzer does with it in the query plan. Your current use of two queries may very well be optimal for your data set, since most countries don’t span much of the world in terms of longitude.

Re: Longitude Index: B-Tree or Cluster

2020-04-10 Thread John J Foster via 4D_Tech
Hi Arnaud, Just noticed that the search was typed in wrong. It should be: QUERY([City];[City]country code=$cntryCode) QUERY SELECTION([City];[City]longitude>=$lon_r;*) QUERY SELECTION([City]; & ;[City]longitude<=$lon_r+1) John… > In this case I am searching by

Re: Longitude Index: B-Tree or Cluster

2020-04-10 Thread John J Foster via 4D_Tech
Hi Arnaud, > what is the field type? The field type is a real number. There are so many similar values in terms of the integer part but many many variations of the decimal part. Currently I am using a B-Tree index. I’ve thought about breaking it the two parts but that just adds an extra

Re: Longitude Index: B-Tree or Cluster

2020-04-10 Thread Arnaud de Montard via 4D_Tech
> Le 10 avr. 2020 à 02:03, John J Foster via 4D_Tech <4d_tech@lists.4d.com> a > écrit : > > Hi All, > > I have a GEO database of almost 5 million cities. They include various data > and latitude and longitude. > > I need to use the Longitude file in searches. To speed it up I need to index.

Longitude Index: B-Tree or Cluster

2020-04-09 Thread John J Foster via 4D_Tech
Hi All, I have a GEO database of almost 5 million cities. They include various data and latitude and longitude. I need to use the Longitude file in searches. To speed it up I need to index. Many of the values are similar like 141.235 or 141 3.23 and so on. So for those who might know would a