[MediaWiki-commits] [Gerrit] explicitely set input reader format in osm2pgsql - change (operations/puppet)

[Gehel (Code Review)]

Gehel has submitted this change and it was merged.

Change subject: explicitely set input reader format in osm2pgsql
..


explicitely set input reader format in osm2pgsql

Latest version of osm2pgsql does not seem to autodetect format correctly
in our replicate-osm script. As the format is known in advanced, let's
make it explicit.

Bug: T112423
Change-Id: I0482b718a496cfabf647295b413bce5e2a453375
---
M modules/osm/manifests/planet_sync.pp
M modules/osm/spec/defines/osm_planet_sync_spec.rb
M modules/osm/templates/replicate-osm.erb
3 files changed, 37 insertions(+), 11 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/osm/manifests/planet_sync.pp 
b/modules/osm/manifests/planet_sync.pp
index 19a1651..e9b3ebf 100644
--- a/modules/osm/manifests/planet_sync.pp
+++ b/modules/osm/manifests/planet_sync.pp
@@ -30,6 +30,10 @@
 #  Memory in megabytes osm2pgsql should occupy
 #   $num_threads
 #  Number of threads to use during sync
+#   $input_reader_format
+#  Format passed to osm2pgsql as --input-reader parameter. osm2pgsql < 0.90
+#  needs 'libxml2' (which is default) and osm2pgsql >= 0.90 needs 'xml'.
+#  osm2pgsql == 0.90 is used on Jessie only at this point.
 #
 # Actions:
 #   sync with planet.osm
@@ -53,6 +57,10 @@
 $expire_levels='15',
 $memory_limit=floor($::memorysize_mb)/12,
 $num_threads=$::processorcount,
+$input_reader_format = os_version('Debian >= Jessie')? {
+true=> 'xml',
+default => 'libxml2',
+},
 ) {
 include ::osm::users
 
diff --git a/modules/osm/spec/defines/osm_planet_sync_spec.rb 
b/modules/osm/spec/defines/osm_planet_sync_spec.rb
index a669991..b5badb2 100644
--- a/modules/osm/spec/defines/osm_planet_sync_spec.rb
+++ b/modules/osm/spec/defines/osm_planet_sync_spec.rb
@@ -2,17 +2,34 @@
 
 describe 'osm::planet_sync', :type => :define do
 let(:title) { 'somedb' }
-let(:facts) { {
-:memoryfree => '1000 MB',
-}
-}
-let(:params) { {
-:osmosis_dir => '/srv/osmosis',
-:period => 'minute',
-}
-}
+
 context 'with ensure present' do
-it { should contain_cron('planet_sync-somedb') }
-it { should 
contain_file('/srv/osmosis/configuration.txt').with_content(/minute/) }
+let(:params) { {
+:osmosis_dir => '/srv/osmosis',
+:period  => 'minute',
+:pg_password => 'secret',
+} }
+
+context 'on Ubuntu Precise' do
+let(:facts) { {
+:lsbdistrelease => 'Precise',
+:lsbdistid  => 'Ubuntu',
+:memorysize_mb => 64420.94,
+}}
+
+it { should contain_cron('planet_sync-somedb') }
+it { should 
contain_file('/srv/osmosis/configuration.txt').with_content(/minute/) }
+it { should 
contain_file('/usr/local/bin/replicate-osm').with_content(/--input-reader 
libxml2/) }
+end
+
+context 'on Debian Jessie' do
+let(:facts) { {
+:lsbdistrelease => 'Jessie',
+:lsbdistid  => 'Debian',
+:memorysize_mb => 64420.94,
+}}
+it { should 
contain_file('/usr/local/bin/replicate-osm').with_content(/--input-reader xml/) 
}
+end
 end
+
 end
diff --git a/modules/osm/templates/replicate-osm.erb 
b/modules/osm/templates/replicate-osm.erb
index 827fc3e..875973b 100644
--- a/modules/osm/templates/replicate-osm.erb
+++ b/modules/osm/templates/replicate-osm.erb
@@ -18,6 +18,7 @@
 | /usr/bin/osm2pgsql \
   --hstore \
   --slim \
+  --input-reader <%= @input_reader_format %> \
   --cache <%= @memory_limit %> \
   --number-processes <%= @num_threads %> \
   --expire-tiles <%= @expire_levels %> \

-- 
To view, visit https://gerrit.wikimedia.org/r/293475
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I0482b718a496cfabf647295b413bce5e2a453375
Gerrit-PatchSet: 4
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: Alexandros Kosiaris 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: MaxSem 
Gerrit-Reviewer: Yurik 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] explicitely set input reader format in osm2pgsql - change (operations/puppet)

[Gehel (Code Review)]

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/293475

Change subject: explicitely set input reader format in osm2pgsql
..

explicitely set input reader format in osm2pgsql

Latest version of osm2pgsql does not seem to autodetect format correctly
in our replicate-osm script. As the format is known in advanced, let's
make it explicit.

Bug: T112423
Change-Id: I0482b718a496cfabf647295b413bce5e2a453375
---
M modules/osm/templates/replicate-osm.erb
1 file changed, 1 insertion(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/75/293475/1

diff --git a/modules/osm/templates/replicate-osm.erb 
b/modules/osm/templates/replicate-osm.erb
index 421593e..b162754 100644
--- a/modules/osm/templates/replicate-osm.erb
+++ b/modules/osm/templates/replicate-osm.erb
@@ -16,6 +16,7 @@
 | /usr/bin/osm2pgsql \
   --hstore \
   --slim \
+  --input-reader xml \
   --cache <%= @memory_limit %> \
   --number-processes <%= @num_threads %> \
   --expire-tiles <%= @expire_levels %> \

-- 
To view, visit https://gerrit.wikimedia.org/r/293475
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I0482b718a496cfabf647295b413bce5e2a453375
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits