--- Begin Message ---
Hi,
i see the same behavior on my installation.
It begins shortly after when the scheduler for discoverall kicks in
---deployment.yml---
schedule:
discoverall:
when: '30 5,21 * * *'
the errors continue after the message "invalid input syntax for integer"
---postgresql-log---
2019-11-29 03:22:40.005 CET postgres netdata [63977]ERROR: canceling statement
due to statement time
out
2019-11-29 03:22:40.005 CET postgres netdata [63977]STATEMENT:
SELECT
checkpoints_timed AS checkpoint_scheduled,
checkpoints_req AS checkpoint_requested,
buffers_checkpoint * current_setting('block_size')::numeric
buffers_checkpoint,
buffers_clean * current_setting('block_size')::numeric
buffers_clean,
maxwritten_clean,
buffers_backend * current_setting('block_size')::numeric
buffers_backend,
buffers_alloc * current_setting('block_size')::numeric
buffers_alloc,
buffers_backend_fsync
FROM pg_stat_bgwriter;
2019-11-29 05:33:10.675 CET [19803]ERROR: canceling autovacuum task
2019-11-29 05:33:10.675 CET [19803]CONTEXT: automatic analyze of table
"netdisco.public.device_port"
2019-11-29 05:34:10.653 CET [20086]ERROR: canceling autovacuum task
2019-11-29 05:34:10.653 CET [20086]CONTEXT: automatic analyze of table
"netdisco.public.device_port"
2019-11-29 05:35:33.640 CET netdisco netdisco [20898]ERROR: invalid input
syntax for integer: "0.47.92.100.108.128.0"
2019-11-29 05:35:33.640 CET netdisco netdisco [20898]STATEMENT: INSERT INTO
device_port_properties ( ifindex, ip, port, raw_speed) VALUES ( $1, $2, $3, $4 )
2019-11-29 05:35:33.640 CET netdisco netdisco [20898]ERROR: current
transaction is aborted, commands ignored until end of transaction block
2019-11-29 05:35:33.640 CET netdisco netdisco [20898]STATEMENT: INSERT INTO
device_port_properties ( ifindex, ip, port, raw_speed) VALUES ( $1, $2, $3, $4 )
2019-11-29 05:35:33.640 CET netdisco netdisco [20898]ERROR: current
transaction is aborted, commands ignored until end of transaction block
2019-11-29 05:35:33.640 CET netdisco netdisco [20898]STATEMENT: INSERT INTO
device_port_properties ( ifindex, ip, port, raw_speed) VALUES ( $1, $2, $3, $4 )
2019-11-29 05:35:33.640 CET netdisco netdisco [20898]ERROR: current
transaction is aborted, commands ignored until end of transaction block
......
I dumped the sql-data in a file and could not find a string
"0.47.92.100.108.128.0" there.
greets
piotr
> Tobias Gerlach <[email protected]> hat am 28. November 2019 um 11:26
> geschrieben:
>
>
> After a full discovery run I can see in the postgres log tens of
> thousands of messages like these:
>
> 2019-11-28 11:18:26.296 CET [42973] STATEMENT: INSERT INTO
> device_port_properties ( ifindex, ip, port, raw_speed) VALUES ( $1,
> $2, $3, $4 )
> 2019-11-28 11:18:26.296 CET [42973] ERROR: current transaction is
> aborted, commands ignored until end of transaction block
> ...
>
> What does these errors mean? Could this be related to the stucked jobs?
>
>
> Am Mo., 25. Nov. 2019 um 17:05 Uhr schrieb Tobias Gerlach <[email protected]>:
> >
> > Hello,
> >
> > I start thinking whether the problem might not be an issue with the latest
> > 2.0044x'er versions of Netdisco, but could be an issue with my new RHEL
> > server. However I still have no clue what could the stuck jobs.
> > Last week I completely uninstalled Netdisco and installed version 2.42.10
> > from middle of this year, where I'm convinced I never faced these issues
> > with. Also I deleted all data from any Netdisco tables, except the "users"
> > table, to make sure the database contains no old entries. Even today the
> > "admin" table contains jobs which have been entered more than three days
> > ago but haven't started yet (status is NULL). On the other hand there are a
> > bunch of jobs which finished successfully.
> > On my old SLES server I used PostgreSQL v9.4 and on my current RHEL 7.6
> > server I'm running on PostgreSQL database server version 11.6. I've
> > optimized postgres.conf with values from pgtune based on my server
> > environment. Netdisco's deployment.yml I adopted from my old installation
> > and haven't been touched so far.
> > Any ideas to narrow down what's going on and what could be next steps to
> > troubleshoot?
> >
> > Jobs:
> > 3091997 2019-11-22 08:33:25.783195 NULL NULL 10.208.16.6 NULL arpnip NULL
> > queued netdisco 172.26.14.13 NULL NULL NULL
> > 3091068 2019-11-22 08:33:25.783195 NULL NULL 10.213.89.205 NULL arpnip NULL
> > queued netdisco 172.26.14.13 NULL NULL NULL
> > 3088950 2019-11-22 08:33:25.783195 NULL NULL 10.26.52.5 NULL arpnip NULL
> > queued netdisco 172.26.14.13 NULL NULL NULL
> > ...
> > 3093364 2019-11-22 08:33:25.783195 2019-11-22 13:15:38 2019-11-22 13:15:39
> > 149.223.209.234 NULL arpnip NULL done netdisco 172.26.14.13 Gathered arp
> > caches from 149.223.209.234 NULL NULL
> > 3094549 2019-11-22 08:33:25.783195 2019-11-22 13:15:16 2019-11-22 13:15:27
> > 10.219.155.41 NULL arpnip NULL done netdisco 172.26.14.13 Gathered arp
> > caches from 10.219.155.41 NULL NULL
> > 3089676 2019-11-22 08:33:25.783195 2019-11-22 13:15:22 2019-11-22 13:15:23
> > 10.166.0.58 NULL arpnip NULL done netdisco 172.26.14.13 Gathered arp caches
> > from 10.166.0.58 NULL NULL
> >
> > postgres.conf modified part:
> > DB TUNE
> > # DB Version: 11
> > # OS Type: linux
> > # DB Type: web
> > # Total Memory (RAM): 16 GB
> > # CPUs num: 4
> > # Connections num: 100
> > # Data Storage: ssd
> >
> > max_connections = 100
> > shared_buffers = 4GB
> > effective_cache_size = 12GB
> > maintenance_work_mem = 1GB
> > checkpoint_completion_target = 0.7
> > wal_buffers = 16MB
> > default_statistics_target = 100
> > random_page_cost = 1.1
> > effective_io_concurrency = 200
> > work_mem = 20971kB
> > min_wal_size = 1GB
> > max_wal_size = 2GB
> > max_worker_processes = 4
> > max_parallel_workers_per_gather = 2
> > max_parallel_workers = 4
> >
> > Thanks,
> > Tobias
> >
> >
> > Am Mi., 20. Nov. 2019 um 11:43 Uhr schrieb Oliver Gorwits <[email protected]>:
> >>
> >> Hi Tobias
> >>
> >> Yes, deleting the perl5 directory is enough, and then when you reinstall
> >> you can pick a specific version by doing:
> >>
> >> curl -L https://cpanmin.us/ | perl - --notest --local-lib ~/perl5
> >> App::[email protected]
> >>
> >> (or whatever version you want)
> >>
> >> Please do let us know how you get on, as so far we've not been able to
> >> reproduce the issue which is quite frustrating
> >>
> >> regards
> >> oliver.
> >>
> >> On Mon, 18 Nov 2019 at 09:04, Tobias Gerlach <[email protected]> wrote:
> >>>
> >>> Hello,
> >>> unfortunately the problems still exist with the latest version and also
> >>> all workarounds don't work work for me. I'm seriously considering to
> >>> downgrade Netdisco as the issue effects a few thousand devices in my case.
> >>> I think the problems didn't exist with version 2.043001 and started with
> >>> 2.044x but I'm no longer 100% sure. Can anyone confirm that please?
> >>> Would it be enough to delete the whole ~/perl5 directory for a complete
> >>> deinstallation or should the database be deleted as well? Perhaps it is
> >>> sufficient to delete the contents of some tables?
> >>> Thanks,
> >>> Tobias
> >>>
> >>> Am Do., 7. Nov. 2019 um 19:19 Uhr schrieb Nick Nauwelaerts
> >>> <[email protected]>:
> >>>>
> >>>> (sourceforge held my mail for being oversize, so trimmed old replies a
> >>>> bit)
> >>>>
> >>>> kinda late to the party, tried to make sense of the thread via the
> >>>> mailinglist archive.
> >>>>
> >>>> "Happens in my case on a lot of devices, not just a few. The stuck jobs
> >>>> doesn't finish after even days. "
> >>>>
> >>>>
> >>>> if you restart netdisco-backend, are your jobs then reported as complete
> >>>> in the poller performance report, but with very long runtimes?
> >>>>
> >>>>
> >>>> i've been seeing something similar from time to time but can't find a
> >>>> way to reproduce it:
> >>>> https://github.com/netdisco/netdisco/issues/466
> >>>> jobs that ran for "1 day 01:29:30".
> >>>>
> >>>>
> >>>> what i'm looking, which could be totally unrelated, is the timestamp
> >>>> precision, since some fields use high precision "2018-12-13
> >>>> 20:05:30.887272", some just up to the second: " 2018-12-13 20:05:30".
> >>>> this shouldn't be an issue if we compare using sql, but that's not
> >>>> always the case, for example in
> >>>>
> >>>> https://github.com/netdisco/netdisco/blob/92bc49f27444ff201cc2178132979e0d89c08850/lib/App/Netdisco/DB/Result/Admin.pm#L80-L82
> >>>> we use "$args->{foreign_alias}.last_defer" => { '>', \'(LOCALTIMESTAMP -
> >>>> ?::interval)' },
> >>>>
> >>>>
> >>>>
> >>>> but in other parts
> >>>> https://github.com/netdisco/netdisco/blob/92bc49f27444ff201cc2178132979e0d89c08850/lib/App/Netdisco/DB/ResultSet/Admin.pm#L61-L63
> >>>> we cast those timestamps: entered_stamp => \"to_char(entered,
> >>>> 'YYYY-MM-DD HH24:MI')",
> >>>>
> >>>>
> >>>> also think i saw 1 or 2 time comparisons done in perl.
> >>>>
> >>>>
> >>>> and for some reason sometimes the high precision timer has a leading 0
> >>>> for microseconds:
> >>>> no leading 0 -> "35119" --- macsuck | 2018-12-13 12:55:20.35119
> >>>> leading 0 -> "058855 " ---- arpnip | 2018-12-13 12:05:12.058855
> >>>>
> >>>>
> >>>> // nick
> >>>>
> >>>> From: Pavel Skovajsa [mailto:[email protected]]
> >>>> Sent: Monday, October 14, 2019 22:09
> >>>> To: Oliver Gorwits <[email protected]>
> >>>> Cc: Tobias Gerlach <[email protected]>;
> >>>> [email protected]
> >>>> Subject: Re: [Netdisco] Arpwalk, Macsuck or Discover not successful on
> >>>> some devices
> >>>>
> >>>> bump
> >>>>
> >>>> On Mon, Oct 7, 2019 at 6:39 PM Pavel Skovajsa <[email protected]>
> >>>> wrote:
> >>>> Hello,
> >>>>
> >>>> I think we have the same issue - the jobs haven't finished since the
> >>>> upgrade to 2.44.0. Last week I deleted all the job (using the web
> >>>> frontend trash bin icon). Poller performance only shows nbtstat job
> >>>> completed, nothing else is there since the update. The output for the
> >>>> slow queries and running queries is on https://pastebin.com/LPtEDD6h.
> >>>>
> >>>> On the other side, it looks like it is really working, because things
> >>>> are working based on the output below:
> >>>> query
> >>>> result
> >>>> select count(name) from device where vendor = 'cisco' AND (last_discover
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '2 hours') AND (CURRENT_TIMESTAMP
> >>>> - interval '1 hours'));
> >>>> 1809
> >>>> select count(name) from device where vendor = 'cisco' AND (last_discover
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '3 hours') AND (CURRENT_TIMESTAMP
> >>>> - interval '2 hours'));
> >>>> 1856
> >>>> select count(name) from device where vendor = 'cisco' AND (last_discover
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '4 hours') AND (CURRENT_TIMESTAMP
> >>>> - interval '3 hours'));
> >>>> 1815
> >>>> select count(name) from device where vendor = 'cisco' AND (last_discover
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '5 hours') AND (CURRENT_TIMESTAMP
> >>>> - interval '4 hours'));
> >>>> 1932
> >>>> select count(name) from device where vendor = 'cisco' AND (last_discover
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '6 hours') AND (CURRENT_TIMESTAMP
> >>>> - interval '5 hours'));
> >>>> 712
> >>>> select count(name) from device where vendor = 'cisco' AND (last_discover
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '7 hours') AND (CURRENT_TIMESTAMP
> >>>> - interval '6 hours'));
> >>>> 0
> >>>> select count(name) from device where vendor = 'cisco' AND (last_discover
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '8 hours') AND (CURRENT_TIMESTAMP
> >>>> - interval '7 hours'));
> >>>> 0
> >>>> select count(name) from device where vendor = 'cisco' AND (last_discover
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '9 hours') AND (CURRENT_TIMESTAMP
> >>>> - interval '8 hours'));
> >>>> 0
> >>>> select count(name) from device where vendor = 'cisco' AND (last_discover
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '10 hours') AND
> >>>> (CURRENT_TIMESTAMP - interval '9 hours'));
> >>>> 0
> >>>> select count(name) from device where vendor = 'cisco' AND (last_discover
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '11 hours') AND
> >>>> (CURRENT_TIMESTAMP - interval '10 hours'));
> >>>> 0
> >>>>
> >>>>
> >>>>
> >>>> select count(name) from device where vendor = 'cisco' AND (last_macsuck
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '2 hours') AND (CURRENT_TIMESTAMP
> >>>> - interval '1 hours'));
> >>>> 0
> >>>> select count(name) from device where vendor = 'cisco' AND (last_macsuck
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '3 hours') AND (CURRENT_TIMESTAMP
> >>>> - interval '2 hours'));
> >>>> 0
> >>>> select count(name) from device where vendor = 'cisco' AND (last_macsuck
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '4 hours') AND (CURRENT_TIMESTAMP
> >>>> - interval '3 hours'));
> >>>> 0
> >>>> select count(name) from device where vendor = 'cisco' AND (last_macsuck
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '5 hours') AND (CURRENT_TIMESTAMP
> >>>> - interval '4 hours'));
> >>>> 0
> >>>> select count(name) from device where vendor = 'cisco' AND (last_macsuck
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '6 hours') AND (CURRENT_TIMESTAMP
> >>>> - interval '5 hours'));
> >>>> 0
> >>>> select count(name) from device where vendor = 'cisco' AND (last_macsuck
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '7 hours') AND (CURRENT_TIMESTAMP
> >>>> - interval '6 hours'));
> >>>> 0
> >>>> select count(name) from device where vendor = 'cisco' AND (last_macsuck
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '8 hours') AND (CURRENT_TIMESTAMP
> >>>> - interval '7 hours'));
> >>>> 0
> >>>> select count(name) from device where vendor = 'cisco' AND (last_macsuck
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '9 hours') AND (CURRENT_TIMESTAMP
> >>>> - interval '8 hours'));
> >>>> 0
> >>>> select count(name) from device where vendor = 'cisco' AND (last_macsuck
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '10 hours') AND
> >>>> (CURRENT_TIMESTAMP - interval '9 hours'));
> >>>> 0
> >>>> select count(name) from device where vendor = 'cisco' AND (last_macsuck
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '11 hours') AND
> >>>> (CURRENT_TIMESTAMP - interval '10 hours'));
> >>>> 1066
> >>>> select count(name) from device where vendor = 'cisco' AND (last_macsuck
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '12 hours') AND
> >>>> (CURRENT_TIMESTAMP - interval '11hours'));
> >>>> 2021
> >>>> select count(name) from device where vendor = 'cisco' AND (last_macsuck
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '13 hours') AND
> >>>> (CURRENT_TIMESTAMP - interval '12 hours'));
> >>>> 1957
> >>>> select count(name) from device where vendor = 'cisco' AND (last_macsuck
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '14 hours') AND
> >>>> (CURRENT_TIMESTAMP - interval '13 hours'));
> >>>> 1913
> >>>> select count(name) from device where vendor = 'cisco' AND (last_macsuck
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '15 hours') AND
> >>>> (CURRENT_TIMESTAMP - interval '14 hours'));
> >>>> 1972
> >>>> select count(name) from device where vendor = 'cisco' AND (last_macsuck
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '16 hours') AND
> >>>> (CURRENT_TIMESTAMP - interval '15 hours'));
> >>>> 237
> >>>>
> >>>> select count(name) from device where vendor = 'cisco' AND (last_arpnip
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '2 hours') AND (CURRENT_TIMESTAMP
> >>>> - interval '1 hours'));
> >>>> 2010
> >>>> select count(name) from device where vendor = 'cisco' AND (last_arpnip
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '3 hours') AND (CURRENT_TIMESTAMP
> >>>> - interval '2 hours'));
> >>>> 2271
> >>>> select count(name) from device where vendor = 'cisco' AND (last_arpnip
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '4 hours') AND (CURRENT_TIMESTAMP
> >>>> - interval '3 hours'));
> >>>> 1301
> >>>> select count(name) from device where vendor = 'cisco' AND (last_arpnip
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '5 hours') AND (CURRENT_TIMESTAMP
> >>>> - interval '4 hours'));
> >>>> 0
> >>>> select count(name) from device where vendor = 'cisco' AND (last_arpnip
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '6 hours') AND (CURRENT_TIMESTAMP
> >>>> - interval '5 hours'));
> >>>> 0
> >>>> select count(name) from device where vendor = 'cisco' AND (last_arpnip
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '7 hours') AND (CURRENT_TIMESTAMP
> >>>> - interval '6 hours'));
> >>>> 0
> >>>> select count(name) from device where vendor = 'cisco' AND (last_arpnip
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '8 hours') AND (CURRENT_TIMESTAMP
> >>>> - interval '7 hours'));
> >>>> 0
> >>>> select count(name) from device where vendor = 'cisco' AND (last_arpnip
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '9 hours') AND (CURRENT_TIMESTAMP
> >>>> - interval '8 hours'));
> >>>> 0
> >>>> select count(name) from device where vendor = 'cisco' AND (last_arpnip
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '10 hours') AND
> >>>> (CURRENT_TIMESTAMP - interval '9 hours'));
> >>>> 0
> >>>> select count(name) from device where vendor = 'cisco' AND (last_arpnip
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '11 hours') AND
> >>>> (CURRENT_TIMESTAMP - interval '10 hours'));
> >>>> 0
> >>>> select count(name) from device where vendor = 'cisco' AND (last_arpnip
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '12 hours') AND
> >>>> (CURRENT_TIMESTAMP - interval '11hours'));
> >>>> 0
> >>>> select count(name) from device where vendor = 'cisco' AND (last_arpnip
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '13 hours') AND
> >>>> (CURRENT_TIMESTAMP - interval '12 hours'));
> >>>> 0
> >>>> select count(name) from device where vendor = 'cisco' AND (last_arpnip
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '14 hours') AND
> >>>> (CURRENT_TIMESTAMP - interval '13 hours'));
> >>>> 0
> >>>> select count(name) from device where vendor = 'cisco' AND (last_arpnip
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '15 hours') AND
> >>>> (CURRENT_TIMESTAMP - interval '14 hours'));
> >>>> 0
> >>>> select count(name) from device where vendor = 'cisco' AND (last_arpnip
> >>>> BETWEEN (CURRENT_TIMESTAMP - interval '16 hours') AND
> >>>> (CURRENT_TIMESTAMP - interval '15 hours'));
> >>>> 0
> >>>>
> >>>> netdisco=> select action,status,count (job) from admin group by
> >>>> action,status;
> >>>> action | status | count
> >>>> -------------+---------------------------+--------
> >>>> arpnip | done | 51663
> >>>> arpnip | queued | 248
> >>>> arpwalk | done | 6
> >>>> discover | done | 36793
> >>>> discover | error | 6635
> >>>> discover | info | 5
> >>>> discover | queued | 4076
> >>>> discoverall | done | 3
> >>>> expire | done | 4
> >>>> macsuck | done | 46684
> >>>> macsuck | queued | 299
> >>>> macsuck | queued-mdnetdisco | 2
> >>>> macwalk | done | 4
> >>>> nbtstat | done | 105197
> >>>> nbtstat | queued | 303
> >>>> nbtwalk | done | 9
> >>>> (16 rows)
> >>>>
> >>>> -pavel
> >>>>
> >>>> On Sat, Oct 5, 2019 at 1:17 PM Oliver Gorwits <[email protected]> wrote:
> >>>> It would be interesting to know what the backend server process table
> >>>> shows when the workers are not progressing. Output of ps should show
> >>>> what the workers are doing.
> >>>>
> >>>> Also, here is a way to show active queries against a database and at the
> >>>> same time I wonder if running this will show some blockage somewhere:
> >>>>
> >>>> -- show running queries (pre 9.2)
> >>>> SELECT procpid, age(clock_timestamp(), query_start), usename,
> >>>> current_query
> >>>> FROM pg_stat_activity
> >>>> WHERE current_query != '<IDLE>' AND current_query NOT ILIKE
> >>>> '%pg_stat_activity%'
> >>>> ORDER BY query_start desc;
> >>>>
> >>>> -- show running queries (9.2)
> >>>> SELECT pid, age(clock_timestamp(), query_start), usename, query
> >>>> FROM pg_stat_activity
> >>>> WHERE query != '<IDLE>' AND query NOT ILIKE '%pg_stat_activity%'
> >>>> ORDER BY query_start desc;
> >>>>
> >>>> -- long running (9.2)
> >>>> SELECT now() - query_start as "runtime", usename, datname, waiting,
> >>>> state, query
> >>>> FROM pg_stat_activity
> >>>> WHERE now() - query_start > '2 minutes'::interval
> >>>> ORDER BY runtime DESC;
> >>>>
> >>>> ________________________________
> >>>>
> >>>> Volg Aquafin op Facebook<https://www.facebook.com/AquafinNV> |
> >>>> Twitter<https://twitter.com/aquafinnv> |
> >>>> YouTube<http://www.youtube.com/channel/UCk_4P5BJ-MtEEDCkCsR_KqQ?feature=mhee>
> >>>> | LinkedIN<http://www.linkedin.com/company/aquafin/products> |
> >>>> Instagram<https://www.instagram.com/aquafin_nv/>
> >>>>
> >>>> In het kader van de uitoefening van onze taken verzamelen we bij Aquafin
> >>>> persoonsgegevens. Hoe we omgaan met deze gegevens en wat de rechten van
> >>>> de betrokkenen zijn, kan je nalezen in onze privacy
> >>>> policy<https://www.aquafin.be/nl-be/privacy-policy>.
> >>>>
> >>>> P Denk aan het milieu. Druk deze mail niet onnodig af.
>
>
> _______________________________________________
> Netdisco mailing list
> [email protected]
> https://sourceforge.net/p/netdisco/mailman/netdisco-users/
>
--- End Message ---
