Re: [Freeipa-devel] [PATCH] 253 Enable mod_deflate

2013-01-09 Thread Petr Viktorin

On 01/09/2013 08:41 AM, Martin Kosek wrote:

On 01/08/2013 06:05 PM, Petr Vobornik wrote:

On 01/08/2013 05:58 PM, Rob Crittenden wrote:

Petr Vobornik wrote:

Design page: http://www.freeipa.org/page/V3/WebUI_gzip_compression

Enabled mod_deflate for:
* text/html (HTML files)
* text/plain (for future use)
* text/css (CSS files)
* text/xml (XML RPC)
* application/javascript (JavaScript files)
* application/json (JSON RPC)
* application/x-font-woff (woff fonts)

Added proper mime type for woff fonts.
Disabled etag header because it doesn't work with mod_deflate.

https://fedorahosted.org/freeipa/ticket/3326


Should this be enabled on upgrades as well?


Yes, I don't see a reason not to.


This should be enabled on upgrades as is, since Petr bumped VERSION in
install/conf/ipa.conf.

We should carefully check that enabling it also for xmlrpc/json does not cause
any grief.

Martin



HTTP libraries won't ask for gzip if they can't handle it, so there 
shouldn't be any grief.
I tested the UI, installing client  replica, and the CLI tool. All work 
fine.


Just one thing: WOFF is already compressed so we shouldn't gzip it again.


--
PetrĀ³

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 253 Enable mod_deflate

2013-01-09 Thread Petr Vobornik

On 01/09/2013 11:59 AM, Petr Viktorin wrote:

On 01/09/2013 08:41 AM, Martin Kosek wrote:

On 01/08/2013 06:05 PM, Petr Vobornik wrote:

On 01/08/2013 05:58 PM, Rob Crittenden wrote:

Petr Vobornik wrote:

Design page: http://www.freeipa.org/page/V3/WebUI_gzip_compression

Enabled mod_deflate for:
* text/html (HTML files)
* text/plain (for future use)
* text/css (CSS files)
* text/xml (XML RPC)
* application/javascript (JavaScript files)
* application/json (JSON RPC)
* application/x-font-woff (woff fonts)

Added proper mime type for woff fonts.
Disabled etag header because it doesn't work with mod_deflate.

https://fedorahosted.org/freeipa/ticket/3326


Should this be enabled on upgrades as well?


Yes, I don't see a reason not to.


This should be enabled on upgrades as is, since Petr bumped VERSION in
install/conf/ipa.conf.

We should carefully check that enabling it also for xmlrpc/json does
not cause
any grief.

Martin



HTTP libraries won't ask for gzip if they can't handle it, so there
shouldn't be any grief.
I tested the UI, installing client  replica, and the CLI tool. All work
fine.

Just one thing: WOFF is already compressed so we shouldn't gzip it again.



Thanks. Compression for application/x-font-woff removed. Updated patch 
attached.

--
Petr Vobornik
From 8fcc8f6e74153892b2b647e41aeee7f6668cd2d5 Mon Sep 17 00:00:00 2001
From: Petr Vobornik pvobo...@redhat.com
Date: Tue, 4 Dec 2012 13:24:58 +0100
Subject: [PATCH] Enable mod_deflate

Enabled mod_deflate for:
* text/html (HTML files)
* text/plain (for future use)
* text/css (CSS files)
* text/xml (XML RPC)
* application/javascript (JavaScript files)
* application/json (JSON RPC)
* application/x-font-woff (woff fonts)

Added proper mime type for woff fonts.
Disabled etag header because it doesn't work with mod_deflate.

https://fedorahosted.org/freeipa/ticket/3326
---
 install/conf/ipa.conf | 14 +-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/install/conf/ipa.conf b/install/conf/ipa.conf
index d3f3446b0fdcddd3400d784b7b46368db682d335..a936c7fe6b7a957683e803d51b8685fb6187ca31 100644
--- a/install/conf/ipa.conf
+++ b/install/conf/ipa.conf
@@ -1,5 +1,5 @@
 #
-# VERSION 10 - DO NOT REMOVE THIS LINE
+# VERSION 11 - DO NOT REMOVE THIS LINE
 #
 # This file may be overwritten on upgrades.
 #
@@ -22,6 +22,18 @@ LimitRequestFieldSize 10
 AddType application/java-archivejar
 AddType application/x-xpinstall xpi
 
+# Proper header for .woff fonts
+AddType application/x-font-woff woff
+
+# Enable compression
+AddOutputFilterByType DEFLATE text/html text/plain text/xml \
+ application/javascript application/json text/css
+
+# Disable etag http header. Doesn't work well with mod_deflate
+# https://issues.apache.org/bugzilla/show_bug.cgi?id=45023
+# Usage of last-modified header and modified-since validator is sufficient.
+Header unset ETag
+FileETag None
 
 # FIXME: WSGISocketPrefix is a server-scope directive.  The mod_wsgi package
 # should really be fixed by adding this its /etc/httpd/conf.d/wsgi.conf:
-- 
1.7.11.7

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel

Re: [Freeipa-devel] [PATCH] 253 Enable mod_deflate

2013-01-09 Thread Petr Viktorin

On 01/09/2013 02:06 PM, Petr Vobornik wrote:

On 01/09/2013 11:59 AM, Petr Viktorin wrote:

On 01/09/2013 08:41 AM, Martin Kosek wrote:

On 01/08/2013 06:05 PM, Petr Vobornik wrote:

On 01/08/2013 05:58 PM, Rob Crittenden wrote:

Petr Vobornik wrote:

Design page: http://www.freeipa.org/page/V3/WebUI_gzip_compression

Enabled mod_deflate for:
* text/html (HTML files)
* text/plain (for future use)
* text/css (CSS files)
* text/xml (XML RPC)
* application/javascript (JavaScript files)
* application/json (JSON RPC)
* application/x-font-woff (woff fonts)

Added proper mime type for woff fonts.
Disabled etag header because it doesn't work with mod_deflate.

https://fedorahosted.org/freeipa/ticket/3326


Should this be enabled on upgrades as well?


Yes, I don't see a reason not to.


This should be enabled on upgrades as is, since Petr bumped VERSION in
install/conf/ipa.conf.

We should carefully check that enabling it also for xmlrpc/json does
not cause
any grief.

Martin



HTTP libraries won't ask for gzip if they can't handle it, so there
shouldn't be any grief.
I tested the UI, installing client  replica, and the CLI tool. All work
fine.

Just one thing: WOFF is already compressed so we shouldn't gzip it again.



Thanks. Compression for application/x-font-woff removed. Updated patch
attached.



ACK


--
PetrĀ³

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


[Freeipa-devel] RFC 6594 DNS SSHFP records design doc

2013-01-09 Thread Jan Cholasta

Hi,

you can find the design doc at 
http://freeipa.org/page/V3/RFC_6594_SSHFP_DNS_records. It's also 
inlined below.


Honza


= Overview =

IPA supports automatic update of SSHFP DNS records for managed hosts in 
the ttipa-client-install/tt script and in tthost-*/tt commands. 
The support is currently limited to the original SSHFP specification 
from RFC 4255; SSHFP records generated by IPA contain SHA-1 fingerprints 
of RSA and DSS host keys.


Recently, RFC 6594 was released. It extends the original SSHFP 
specification with support for SHA-256 fingerprints and ECDSA host keys.


Add support for RFC 6594 SSHFP records to IPA, generate both SHA-1 and 
SHA-256 fingerprints for RSA, DSS and ECDSA host keys.


= Use Cases =

Automatic generation of SSHFP DNS records on IPA client install:
 # ipa-client-install
 Discovery was successful!
 Hostname: host1.example.com
 Realm: EXAMPLE.COM
 DNS Domain: example.com
 IPA Server: ipa.example.com
 BaseDN: dc=example,dc=com

 Continue to configure the system with these values? [no]: yes
 User authorized to enroll computers: admin
 Synchronizing time with KDC...
 Password for ad...@example.com:
 Enrolled in IPA realm EXAMPLE.COM
 Created /etc/ipa/default.conf
 New SSSD config will be created
 Configured /etc/sssd/sssd.conf
 Configured /etc/krb5.conf for IPA realm EXAMPLE.COM
 trying https://ipa.example.com/ipa/xml
 Hostname (host1.example.com) not found in DNS
 DNS server record set to: host1.example.com - 192.168.1.1
 Adding SSH public key from /etc/ssh/ssh_host_rsa_key.pub
 Adding SSH public key from /etc/ssh/ssh_host_dsa_key.pub
 Forwarding 'host_mod' to server u'https://ipa.example.com/ipa/xml'
 SSSD enabled
 Configured /etc/openldap/ldap.conf
 NTP enabled
 Configured /etc/ssh/ssh_config
 Configured /etc/ssh/sshd_config
 Client configuration complete.

 $ dig host1.example.com SSHFP +short
 2 2 0E04A7E09D037934492108ED5590612416BE736AD1BCAEAE1EA4148E 80C956E2
 2 1 F2A1353FF919AD785B6BD42B588F6236D1F67459
 1 2 3E475EEAF17975C36EE1413DDD659275FDD19C97C2C74A3651BA12F7 52E12A18
 1 1 A308B1B02A8B43CB5192E26FA50280F752BB3A14

Automatic generation of SSHFP DNS records when modifying a host:
 $ ipa host-mod host2.example.com --updatedns --sshpubkey='ssh-rsa 
data' --sshpubkey='ssh-dss data' --sshpubkey='ecdsa-sha2-nistp256 
data'

 -
 Modified host host2.example.com
 -
   Host name: host2.example.com
   Principal name: host/host2.example@example.com
   MAC address: 00:11:22:33:44:55
   SSH public key: ecdsa-sha2-nistp256 data,
   ssh-dss data,
   ssh-rsa data
   Keytab: True
   Managed by: host2.example.com
   SSH public key fingerprint: 
6C:9F:07:51:63:36:32:8B:ED:CF:8C:4C:5F:F2:BF:AE (ecdsa-sha2-nistp256),


07:5D:0D:55:64:62:A3:FE:02:AE:FC:CD:F6:ED:E1:D9 (ssh-dss),

8C:C3:27:A8:40:9F:80:01:61:99:D2:25:55:A3:52:30 (ssh-rsa)

 $ dig host2.example.com SSHFP +short
 2 2 43FFD792089442F08892CA753059FD8B7FA939E990CE4687A3D1FB75 E0B8F6DE
 2 1 4C2C50EDEAE6BC6107A37EAE7A05694C15CFEC53
 3 1 B1D733A262E29B44A4D8A9FAF4B3B9E78302D1DB
 1 2 E5382308CFD60DE4F0ACF3BCB0366314EECFC71030A28AAF75280041 5FDF81A8
 3 2 545055E921E94128AF6BFE68E6E2804333628F7808B8EAE10E297B11 3270862F
 1 1 DA7A6687AE4B2C242E12A67DACDC67D26E374AD5

= Design=

Implement support for SHA-256 fingerprints and ECDSA keys in SSHFP 
records in the ttipapython.ssh/tt module (add new method 
ttfingerprint_dns_sha256/tt).


Extend ttipa-client-install/tt and the tthost/tt plugin to add 
all types of SSHFP records to DNS.


= Implementation =

N/A

= Feature Managment =

N/A

= Major configuration options and enablement =

N/A

= Replication =

N/A

= Updates and Upgrades =

N/A

= Dependencies =

N/A

= External Impact =

N/A

--
Jan Cholasta

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


[Freeipa-devel] [PATCH] 89 Raise ValidationError on invalid CSV values

2013-01-09 Thread Jan Cholasta

Hi,

this patch fixes https://fedorahosted.org/freeipa/ticket/3323.

Honza

--
Jan Cholasta
From 9c8a97f8b1ae843679b4deda8b19e6026e532b32 Mon Sep 17 00:00:00 2001
From: Jan Cholasta jchol...@redhat.com
Date: Tue, 8 Jan 2013 16:32:41 +0100
Subject: [PATCH] Raise ValidationError on invalid CSV values.

https://fedorahosted.org/freeipa/ticket/3323
---
 ipalib/parameters.py | 13 ++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/ipalib/parameters.py b/ipalib/parameters.py
index 670e036..63fa2f6 100644
--- a/ipalib/parameters.py
+++ b/ipalib/parameters.py
@@ -694,9 +694,16 @@ class Param(ReadOnly):
 delimiter=self.csv_separator, quotechar='',
 skipinitialspace=self.csv_skipspace,
 **kwargs)
-for row in csv_reader:
-# decode UTF-8 back to Unicode, cell by cell:
-yield [unicode(cell, 'utf-8') for cell in row]
+try:
+for row in csv_reader:
+# decode UTF-8 back to Unicode, cell by cell:
+yield [unicode(cell, 'utf-8') for cell in row]
+except csv.Error, e:
+raise ValidationError(
+name=self.get_param_name(),
+value=unicode_csv_data,
+error=_(Improperly formatted CSV value (%s) % e)
+)
 
 def split_csv(self, value):
 Split CSV strings into individual values.
-- 
1.8.1

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel

[Freeipa-devel] [PATCH] 90 Run interactive_prompt callbacks after CSV values are split

2013-01-09 Thread Jan Cholasta

Hi,

this patch fixes https://fedorahosted.org/freeipa/ticket/3334.

Honza

--
Jan Cholasta
From 43343dae9ed8cbac6806dd3f7f548b28c872c365 Mon Sep 17 00:00:00 2001
From: Jan Cholasta jchol...@redhat.com
Date: Wed, 9 Jan 2013 18:09:10 +0100
Subject: [PATCH] Run interactive_prompt callbacks after CSV values are split.

https://fedorahosted.org/freeipa/ticket/3334
---
 ipalib/cli.py | 16 
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/ipalib/cli.py b/ipalib/cli.py
index ac0eb05..899220d 100644
--- a/ipalib/cli.py
+++ b/ipalib/cli.py
@@ -1034,6 +1034,14 @@ class cli(backend.Executioner):
 if self.env.interactive:
 self.prompt_interactively(cmd, kw)
 kw = cmd.split_csv(**kw)
+if self.env.interactive:
+try:
+callbacks = cmd.get_callbacks('interactive_prompt')
+except AttributeError:
+pass
+else:
+for callback in callbacks:
+callback(cmd, kw)
 kw['version'] = API_VERSION
 self.load_files(cmd, kw)
 return kw
@@ -1195,14 +1203,6 @@ class cli(backend.Executioner):
 param.label, param.confirm
 )
 
-try:
-callbacks = cmd.get_callbacks('interactive_prompt')
-except AttributeError:
-pass
-else:
-for callback in callbacks:
-callback(cmd, kw)
-
 def load_files(self, cmd, kw):
 
 Load files from File parameters.
-- 
1.8.1

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel

[Freeipa-devel] [PATCHES] 91-92 Add support for RFC 6594 SSHFP DNS records

2013-01-09 Thread Jan Cholasta

Hi,

Patch 91 removes module ipapython.compat. The code that uses it doesn't 
work with ancient Python versions anyway, so there's no need to keep it 
around.


Patch 92 adds support for automatic generation of RFC 6594 SSHFP DNS 
records to ipa-client-install and host plugin, as described in 
http://freeipa.org/page/V3/RFC_6594_SSHFP_DNS_records. Note that 
https://fedorahosted.org/freeipa/ticket/2642#comment:7 still applies.


https://fedorahosted.org/freeipa/ticket/2642

Honza

--
Jan Cholasta
From ad6fb49b3e4ac1b5a66d15ee3c8c1075be322e5d Mon Sep 17 00:00:00 2001
From: Jan Cholasta jchol...@redhat.com
Date: Tue, 8 Jan 2013 16:11:05 +0100
Subject: [PATCH 1/2] Drop ipapython.compat.

---
 ipapython/certdb.py| 2 +-
 ipapython/ssh.py   | 3 +--
 ipaserver/install/certs.py | 3 +--
 ipaserver/rpcserver.py | 5 ++---
 tests/test_ipaserver/test_rpcserver.py | 3 ++-
 5 files changed, 7 insertions(+), 9 deletions(-)

diff --git a/ipapython/certdb.py b/ipapython/certdb.py
index 2c0529b..7a06da5 100644
--- a/ipapython/certdb.py
+++ b/ipapython/certdb.py
@@ -21,7 +21,7 @@ from ipapython import ipautil
 from ipapython import nsslib
 from ipalib import pkcs10
 import tempfile
-from ipapython.compat import sha1
+from hashlib import sha1
 import shutil
 import os
 
diff --git a/ipapython/ssh.py b/ipapython/ssh.py
index 6686e91..3294aa4 100644
--- a/ipapython/ssh.py
+++ b/ipapython/ssh.py
@@ -25,8 +25,7 @@ SSH utilities.
 import base64
 import re
 import struct
-
-from ipapython.compat import md5, sha1
+from hashlib import md5, sha1
 
 __all__ = ['SSHPublicKey']
 
diff --git a/ipaserver/install/certs.py b/ipaserver/install/certs.py
index bfbba08..76782d5 100644
--- a/ipaserver/install/certs.py
+++ b/ipaserver/install/certs.py
@@ -28,6 +28,7 @@ import xml.dom.minidom
 import pwd
 import fcntl
 import base64
+from hashlib import sha1
 
 from ipapython import nsslib
 from ipapython import dogtag
@@ -48,8 +49,6 @@ import nss.nss as nss
 
 from ipalib import api
 
-from ipapython.compat import sha1
-
 # Apache needs access to this database so we need to create it
 # where apache can reach
 NSS_DIR = /etc/httpd/alias
diff --git a/ipaserver/rpcserver.py b/ipaserver/rpcserver.py
index 8bce48b..581c30b 100644
--- a/ipaserver/rpcserver.py
+++ b/ipaserver/rpcserver.py
@@ -23,7 +23,6 @@ RPC server.
 Also see the `ipalib.rpc` module.
 
 
-from cgi import parse_qs
 from xml.sax.saxutils import escape
 from xmlrpclib import Fault
 from wsgiref.util import shift_path_info
@@ -34,6 +33,7 @@ import datetime
 from decimal import Decimal
 import urlparse
 import time
+import json
 
 from ipalib import plugable
 from ipalib.backend import Executioner
@@ -43,7 +43,6 @@ from ipalib.rpc import xml_dumps, xml_loads
 from ipalib.util import parse_time_duration, normalize_name
 from ipapython.dn import DN
 from ipaserver.plugins.ldap2 import ldap2
-from ipapython.compat import json
 from ipalib.session import session_mgr, AuthManager, get_ipa_ccache_name, load_ccache_data, bind_ipa_ccache, release_ipa_ccache, fmt_time, default_max_session_duration
 from ipalib.backend import Backend
 from ipalib.krb_utils import krb5_parse_ccache, KRB5_CCache, krb_ticket_expiration_threshold, krb5_format_principal_name
@@ -211,7 +210,7 @@ def extract_query(environ):
 qstr = environ['QUERY_STRING']
 if qstr:
 query = dict(nicify_query(
-parse_qs(qstr)#, keep_blank_values=True)
+urlparse.parse_qs(qstr)#, keep_blank_values=True)
 ))
 else:
 query = {}
diff --git a/tests/test_ipaserver/test_rpcserver.py b/tests/test_ipaserver/test_rpcserver.py
index f423f97..a75a85e 100644
--- a/tests/test_ipaserver/test_rpcserver.py
+++ b/tests/test_ipaserver/test_rpcserver.py
@@ -21,11 +21,12 @@
 Test the `ipaserver.rpc` module.
 
 
+import json
+
 from tests.util import create_test_api, assert_equal, raises, PluginTester
 from tests.data import unicode_str
 from ipalib import errors, Command
 from ipaserver import rpcserver
-from ipapython.compat import json
 
 
 class StartResponse(object):
-- 
1.8.1

From 49b21577101936120f708ac7bc884085ae498dba Mon Sep 17 00:00:00 2001
From: Jan Cholasta jchol...@redhat.com
Date: Tue, 8 Jan 2013 16:13:07 +0100
Subject: [PATCH 2/2] Add support for RFC 6594 SSHFP DNS records.

https://fedorahosted.org/freeipa/ticket/2642
---
 ipa-client/ipa-install/ipa-client-install |  3 +++
 ipalib/plugins/host.py|  6 ++
 ipapython/ssh.py  | 18 +++---
 3 files changed, 24 insertions(+), 3 deletions(-)

diff --git a/ipa-client/ipa-install/ipa-client-install b/ipa-client/ipa-install/ipa-client-install
index a38c828..cc7bae9 100755
--- a/ipa-client/ipa-install/ipa-client-install
+++ b/ipa-client/ipa-install/ipa-client-install
@@ -1267,6 +1267,9 @@ def update_ssh_keys(server, hostname, ssh_dir, create_sshfp):
 sshfp = pubkey.fingerprint_dns_sha1()
 if