Re: [pmacct-discussion] Outputting DNS equivalent of src_host and dst_host IP addresses?

[Hiep Huynh]

Thank you for looking into it and letting us know that it's not on the roadmap.


From: pmacct-discussion <pmacct-discussion-boun...@pmacct.net> on behalf of 
Paolo Lucente <pa...@pmacct.net>
Sent: Wednesday, December 7, 2016 11:02 AM
To: pmacct-discussion@pmacct.net
Cc: Steven Sheehy; Mark Ponthier
Subject: Re: [pmacct-discussion] Outputting DNS equivalent of src_host and 
dst_host IP addresses?

Hi Hiep,

Unfortunately this is not possible today nor in the roadmap. The easiest
thing that comes to mind is a two-steps kind of export: you export from
pmacct into a script, running local, that enriches the records with DNS
lookups; from there you ship enriched records to your consumers for
presentation. The pipeline, depending on your preferences, could be
something as basic as based on files in CSV format or complicated
further (but more elegant).

Cheers,
Paolo

On Mon, Dec 05, 2016 at 09:57:07PM +, Hiep Huynh wrote:
> Bill,
>
>
> Rather than perform the lookup as the traffic arrives, we're interested in 
> have the lookup performed at the time of purge.  In our case the purge 
> interval is 60 minutes, so there are fewer aggregated data (IP addresses) to 
> perform the lookup on.  Also if the lookup results are cached, only the first 
> purge will have a significant impact on performance.
>
>
> But the reason why it's so critical for pmacct to perform it for us is our 
> consumers (ex. presentation) aren't in the same network or have access to the 
> same DNS servers where pmacct collected the data. To clarify, our consumer 
> can try to lookup the IP against its own DNS servers, but it won't find a 
> match for IP's that are localized to the network (and DNS servers) that 
> pmacct ran in.
>
>
> 
> From: pmacct-discussion <pmacct-discussion-boun...@pmacct.net> on behalf of 
> Bill Nash <bi...@billn.net>
> Sent: Monday, December 5, 2016 3:27 PM
> To: pmacct-discussion@pmacct.net
> Cc: Steven Sheehy; Mark Ponthier
> Subject: Re: [pmacct-discussion] Outputting DNS equivalent of src_host and 
> dst_host IP addresses?
>
> DNS lookups will effectively rate limit flow export, though, even if you're 
> hitting a cache. Do it after the fact in your presentation layer with a 
> cache, don't do it at the collection level, because you'll also have to store 
> it. I dunno what your flow volume is, but this is generally a bad idea. 
> You're increasing processing time per flow with a multi-millisecond block, 
> and you're increasing storage per flow by up to 64 bytes, in more egregious 
> cases. Per flow. This is a scale exercise that can get out of hand very 
> quickly.
>
> On Mon, Dec 5, 2016 at 9:10 AM, Hiep Huynh 
> <hhu...@firescope.com<mailto:hhu...@firescope.com>> wrote:
>
>
> When aggregating on src_host and dst_host, the outputs are IP addresses.  Is 
> it possible to also get DNS equivalent? Can pmacct perform a reverse DNS 
> lookup and output it along with the IP addresses?
>
>
> If not, is there a workaround involving the 'networks_file' option where both 
> IP address and its DNS/label are included in its output?
>
>
> Thanks.
>
> ___
> pmacct-discussion mailing list
> http://www.pmacct.net/#mailinglists
>
>
>
> --
>
> - billn

> ___
> pmacct-discussion mailing list
> http://www.pmacct.net/#mailinglists


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] Outputting DNS equivalent of src_host and dst_host IP addresses?

[Bill Nash]

DNS lookups will effectively rate limit flow export, though, even if you're
hitting a cache. Do it after the fact in your presentation layer with a
cache, don't do it at the collection level, because you'll also have to
store it. I dunno what your flow volume is, but this is generally a bad
idea. You're increasing processing time per flow with a multi-millisecond
block, and you're increasing storage per flow by up to 64 bytes, in more
egregious cases. Per flow. This is a scale exercise that can get out of
hand very quickly.

On Mon, Dec 5, 2016 at 9:10 AM, Hiep Huynh  wrote:

>
> When aggregating on src_host and dst_host, the outputs are IP addresses.
> Is it possible to also get DNS equivalent? Can pmacct perform a reverse DNS
> lookup and output it along with the IP addresses?
>
>
> If not, is there a workaround involving the 'networks_file' option where
> both IP address and its DNS/label are included in its output?
>
>
> Thanks.
>
> ___
> pmacct-discussion mailing list
> http://www.pmacct.net/#mailinglists
>



-- 

- billn
___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists