Send Netdot-users mailing list submissions to
[email protected]
To subscribe or unsubscribe via the World Wide Web, visit
https://osl.uoregon.edu/mailman/listinfo/netdot-users
or, via email, send a message with subject or body 'help' to
[email protected]
You can reach the person managing the list at
[email protected]
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Netdot-users digest..."
Today's Topics:
1. Request for testing a script to migrate netdot installations
from MySQL to PostgreSQL (Anton Berezin)
2. Re: Request for testing a script to migrate netdot
installations from MySQL to PostgreSQL (Vincent Magnin)
3. Personnal attribute in common user interface (Romain Nuel)
4. Re: Request for testing a script to migrate netdot
installations from MySQL to PostgreSQL (Anton Berezin)
5. Re: Request for testing a script to migrate netdot
installations from MySQL to PostgreSQL (Vincent Magnin)
6. Re: Request for testing a script to migrate netdot
installations from MySQL to PostgreSQL (Anton Berezin)
----------------------------------------------------------------------
Message: 1
Date: Thu, 16 Jan 2014 22:29:53 +0100
From: Anton Berezin <[email protected]>
Subject: [Netdot-users] Request for testing a script to migrate netdot
installations from MySQL to PostgreSQL
To: [email protected]
Message-ID: <[email protected]>
Content-Type: text/plain; charset=us-ascii
Dear Netdot community,
Some time ago, after much discussion among Netdot developers, with input
from the community (see, for example
http://osl.uoregon.edu/pipermail/netdot-users/2012-September/001345.html),
it was decided to discontinue support for MySQL, and to concentrate on what
can be achieved when the only supported DBMS is PostgreSQL.
Since, from the look of things, the majority of existing Netdot
installations run MySQL, a painless migration path needs to be provided.
The idea for the future Netdot versions is first to discontinue any support
for MySQL - while of course providing a way to migrate to Pg, and once this
is out of the way, to start taking advantage of the features PostgreSQL
provides, like native datatypes for IP address storage and so on.
I'd like to present a script which can be used for MySQL -> PostgreSQL
migration of existing Netdot installations. It has seen substantial
internal testing, and it is now fit for a release to a wider audience. The
more usage it gets, the better are the chances that no remaining bugs are
going to slip through to the final version, after which MySQL support is
officially dropped.
The most recent version of the script can be fetched from
https://raw.github.com/tobez/Netdot/master/upgrade/migrate-mysql-pg
It is meant to live under upgrade/ directory of Netdot sources.
Please consider giving it a go, and report back successes and failures
(especially failures!)
Migration Howto
---------------
1. Fetch, configure, and install Netdot using Pg as the DBMS backend.
The version you install MUST be the same version as your existing
installation that uses MySQL as the backend, since the migration procedure
does not support DB schema changes that are not related to the DB backend
change itself.
Otherwise, the procedure should work for both 1.0.4 and 1.0.5 RC1.
It can be wise to choose installation paths different from your existing
Netdot installation, to temporarily disable existing installation's
cronjobs, and to make sure the Web frontend is not being used - at least
not for any modifications.
2. Make a dump of your Netdot MySQL database:
# mysqldump -u ${DB_DBA} -p ${DB_DATABASE} >netdot-mysql.dump
(substitute correct values from etc/Site.conf of the *existing* MySQL Netdot
installation)
3. Copy migrate-mysql-pg to the upgrade/ directory of the Netdot sources
used to install Pg Netdot.
4. Run the migration script.
Change to the directory the script was copied to in step 3, and run
$ ./migrate-mysql-pg -n -o /where/to/put/netdot-pg.sql
/location/of/netdot-mysql.dump
Look at the output for any abnormalities.
5. Use the dump generated in step 4 to populate PostgreSQL database.
$ psql -q ${DB_DATABASE} ${DB_DBA} </location/of/netdot-pg.sql
(substitute correct values from etc/Site.conf of the *new* Pg Netdot
installation)
Look at the output for any abnormalities. The dump is designed to bail out
immediately in case of any error, on a theory that it is better to have no
data than broken data.
If there are problems, please report them to this list. At any rate, before
repeating steps 4 and 5, it is necessary run make installdb for the new Pg
Netdot installation, since failed step 5 might leave the database in an
inconsistent state.
6. Activate the cronjobs, the Web frontend etc.
I would like to thank Carlos Vicente for guidance, discussion, and testing,
and Network Startup Resource Center for support.
\Anton.
--
Our society can survive even a large amount of irrational regulation.
-- John McCarthy
------------------------------
Message: 2
Date: Fri, 17 Jan 2014 07:25:53 +0000
From: Vincent Magnin <[email protected]>
Subject: Re: [Netdot-users] Request for testing a script to migrate
netdot installations from MySQL to PostgreSQL
To: [email protected]
Message-ID:
<[email protected]>
Content-Type: text/plain; charset=UTF-8; format=flowed; DelSp=Yes
Hi Anton,
Your script seems to works fine, but step 5 failed in my install:
# psql -q netdot105 postgres < netdot-pgsql.dump
Before anything can be done, relax some constraints
Handling table accessright
Handling table asn
Handling table asset
ERROR: date/time field value out of range: "0000-00-00"
CONTEXTE : COPY asset, line 341, column maint_from: "0000-00-00"
Regards,
Vincent
--
------------------------------------------------------------------------
Vincent Magnin [email protected]
Ing?nieur R?seau & T?l?com +41 21 692 22 48
UNIL, Centre Informatique, 1015 Lausanne
Switzerland
------------------------------
Message: 3
Date: Fri, 17 Jan 2014 09:53:09 +0100 (CET)
From: Romain Nuel <[email protected]>
Subject: [Netdot-users] Personnal attribute in common user interface
To: [email protected]
Message-ID: <[email protected]>
Content-Type: text/plain; charset="iso-8859-1"
Hello,
I'm testing Netdot in this moment and I would like to know if is it possible to
add/create a personnal attribute which can be reachable by "common" user (other
than admin) when we add records.
The attribute that I want to add is a sort of comment which is linked to one
host scope, and I would like it to appear in DHCP config file after
exportation.
I succeed to add an attribute linked to the PhysAddr, unfortunately it create
nothing in the DHCP export file. And I fail to create a one linked to a host
scope.
I insisted that I would like this attribute is accessible/available from the
user interface.
Thank you
Have a nice day
--
Romain
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://osl.uoregon.edu/pipermail/netdot-users/attachments/20140117/6511bbf5/attachment-0001.html
------------------------------
Message: 4
Date: Fri, 17 Jan 2014 10:38:07 +0100
From: Anton Berezin <[email protected]>
Subject: Re: [Netdot-users] Request for testing a script to migrate
netdot installations from MySQL to PostgreSQL
To: Vincent Magnin <[email protected]>
Cc: [email protected]
Message-ID: <[email protected]>
Content-Type: text/plain; charset=us-ascii
Hi Vincent,
On Fri, Jan 17, 2014 at 07:25:53AM +0000, Vincent Magnin wrote:
> Your script seems to works fine, but step 5 failed in my install:
>
> # psql -q netdot105 postgres < netdot-pgsql.dump
> Before anything can be done, relax some constraints
> Handling table accessright
> Handling table asn
> Handling table asset
> ERROR: date/time field value out of range: "0000-00-00"
> CONTEXTE : COPY asset, line 341, column maint_from: "0000-00-00"
Should be fixed in the fresh version of the script, please try again.
I'd like to emphasize that botched import like this would leave your fresh
Pg database in a bad shape (indexes and constraints are dropped),
so before repeating steps 4 and 5 you need to `make installdb` your Netdot
Pg installation.
Thanks for testing it!
\Anton.
--
Our society can survive even a large amount of irrational regulation.
-- John McCarthy
------------------------------
Message: 5
Date: Fri, 17 Jan 2014 09:49:52 +0000
From: Vincent Magnin <[email protected]>
Subject: Re: [Netdot-users] Request for testing a script to migrate
netdot installations from MySQL to PostgreSQL
To: [email protected]
Message-ID:
<[email protected]>
Content-Type: text/plain; charset=UTF-8; format=flowed; DelSp=Yes
Hi Anton,
I had also issues on other places (after I fixed this previous one in
my dump):
1. table asset, column "maint_from", value "0000-00-00" => I'll test your fix.
2. table backbonecable, column "type" contains null values
3. table cablestrand, column "status" contains null values
4. table closetpicture/datacache/sitepicture: Issue on handling bindata
5. table contact, column "contacttype" contains null values
Best Regards,
Vincent
--
------------------------------------------------------------------------
Vincent Magnin [email protected]
Ing?nieur R?seau & T?l?com +41 21 692 22 48
UNIL, Centre Informatique, 1015 Lausanne
Switzerland
------------------------------
Message: 6
Date: Fri, 17 Jan 2014 11:50:12 +0100
From: Anton Berezin <[email protected]>
Subject: Re: [Netdot-users] Request for testing a script to migrate
netdot installations from MySQL to PostgreSQL
To: Vincent Magnin <[email protected]>
Cc: [email protected]
Message-ID: <[email protected]>
Content-Type: text/plain; charset=us-ascii
Hi Vincent,
On Fri, Jan 17, 2014 at 09:49:52AM +0000, Vincent Magnin wrote:
> I had also issues on other places (after I fixed this previous one in
> my dump):
>
> 1. table asset, column "maint_from", value "0000-00-00" => I'll test your fix.
> 2. table backbonecable, column "type" contains null values
> 3. table cablestrand, column "status" contains null values
> 4. table closetpicture/datacache/sitepicture: Issue on handling bindata
> 5. table contact, column "contacttype" contains null values
This is interesting. Which version of Netdot are you running?
In my 1.0.4 installation, I have:
mysql> describe backbonecable;
| Field | Type | Null | Key | Default | Extra |
| type | bigint(20) | NO | MUL | NULL | |
Similarly for cablestrand(status) and contact(contacttype).
This means that NULL values should not be in those columns in MySQL.
There are two way to cleanup this mess:
A. Make sure there are no NULLs in not-nullable columns in MySQL (and
execute corresponding ALTER TABLE statements to make sure it does not
happen again in the future).
B. After doing make installdb for the Netdot/Pg, drop the NOT NULL
constraints for those columns in PostgreSQL.
The (A) is more correct, but (B) is certainly easier.
As for item (4), this is a bit more complex to deal with. In the earlier
versions of the migrator I had issues with binary data, but I was pretty
sure these were fixed. In order to debug this, would it be possible for you
to make available to me the portion of MySQL dump corresponding to one of
those tables?
Thanks for your feedback,
\Anton.
--
Our society can survive even a large amount of irrational regulation.
-- John McCarthy
------------------------------
_______________________________________________
Netdot-users mailing list
[email protected]
https://osl.uoregon.edu/mailman/listinfo/netdot-users
End of Netdot-users Digest, Vol 62, Issue 6
*******************************************
