Well, if you want to store octets individulaly sure. In that case, I would create a separate fastbit table that has the ip addresses associated with a 64 bit surrogate key. Do scans of that table, then use the bitmap that results from that to intersect with the other tables on the key.
--Justin On Tue, Feb 16, 2016 at 11:53 PM, Teryl Taylor <[email protected]> wrote: > Hi Ian, > > I've used fastbit with IP's a fair bit and using primitive data types such > as 8,16,32 or 64 bit integers is the way to go, as fastbit can query on > them very quickly. The size of the integer should really be determined by > what kinds of queries you are trying to do on the data. For example, are > you going to be doing a lot of range queries over subnets of a certain > size? I would probably first try using 4 32 bit integers and maybe adjust > the size depending on your needs. A couple things to consider while you do > this: > > 1.) you don't have to load up all the data columns of the IP address to > do searches, you can leverage the indices to do the searches, and then only > load up the data columns that you need. This would be handy if you are > doing a lot of subnet searches. > > 2.) Performance for column oriented stores will deteriorate as you load up > more columns. So if you are going to load up the entire IP address each > time, you probably don't want to divide it into 16 columns, because it will > likely be slower. If you only need to load a portion of the IP address > with each query, then 16 columns may be a good choice. > > 3.) You can use partitioning to your advantage. For example, if you are > only storing IP address for let's say 3 /64 subnets, then you could encode > the subnet in the directory name of the partition and have 3 partitions > while storing only the lower 64 bits of each individual IP address. Then > your query would be a two step process, find the partition that refers to a > subnet, and then query for a particular IP or IPs in that subnet. This > can help speed things up a lot, and allow you to store a lot less data! > > Hope that helps, > > Teryl > > > On Tue, Feb 16, 2016 at 10:00 PM, Ian Pye <[email protected]> wrote: > >> Right. Does make application side math a bit more complex. But thanks >> for the idea. >> >> On Tue, Feb 16, 2016 at 6:17 PM, Justin Swanhart <[email protected]> >> wrote: >> > Hi, >> > >> > The traditional way would to be to store the high/low 64 bits into two >> 64 >> > bit ints, or to use four 32 bit ints. That will make math more complex >> for >> > ranges but it is workable, might need some app-side math too. John may >> be >> > able to provide other solutions. >> > >> > --Justin >> > >> > On Tue, Feb 16, 2016 at 9:11 PM, Ian Pye <[email protected]> wrote: >> >> >> >> All, >> >> >> >> I'm looking for a way to store IPv6 addresses in fastbit. The catch is >> >> that they are 128 bits wide. I could use a blob type, but then nice >> >> operations like > and < don't work. I'm wondering if anyone has done >> >> this, and if so what techniques worked. >> >> >> >> Best, >> >> >> >> Ian >> >> _______________________________________________ >> >> FastBit-users mailing list >> >> [email protected] >> >> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >> > >> > >> > >> > _______________________________________________ >> > FastBit-users mailing list >> > [email protected] >> > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >> > >> _______________________________________________ >> FastBit-users mailing list >> [email protected] >> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >> > > > _______________________________________________ > FastBit-users mailing list > [email protected] > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users > >
_______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
