Stephen Winnall wrote:
> On 5 Jan 2008, at 04:39, Dan Langille wrote:
>
>> Dan Langille wrote:
>>> Stephen Winnall wrote:
>>>> I have been using Bacula for over two years quite happily on an
>>>> old Red Hat 9 server. The last version of Bacula that I used was
>>>> a hand- compiled 2.0.0 with PostgreSQL 7.3.9.
>>>>
>>>> This server is the data storage for my Mac OS X and Windows
>>>> clients , which it serves with Netatalk and Samba. So any given
>>>> file can be accessed via Netatalk, Samba or directly as a Linux
>>>> file. Filenames can be English or German (so may contain umlauts).
>>>>
>>>> I decided to migrate the system to Fedora 8 and go with the RPMs
>>>> in that distro (Bacula 2.0.3 and PostgreSQL 8.2.5). I had
>>>> previously done upgrades from Bacula 1.3.x to 2.0.0, and I've
>>>> migrated other PostgreSQL databases across release versions, so I
>>>> thought I knew what I was doing. I dumped the Bacula database
>>>> from PG 7.3.9 using the PostgreSQL facility in Webmin and
>>>> restored it to a new UTF-8 database in PG 8.2.5.
>>>>
>>>> I had to make a few minor alterations to my bacula-*.conf files
>>>> (I built stuff under "/opt/bacula" and the Fedora distro has it
>>>> under "/") and I was ready to go.
>>>>
>>>> Given the time of year, the first task to throw at the new system
>>>> was the annual full backup. This helped me to shake out the usual
>>>> silly errors that one has, but it's also thrown me up against
>>>> something which I don't know how to handle in Bacula.
>>>>
>>>> I have some filenames which contain lowercase-a-umlaut or
>>>> lowercase-u- umlaut which Netatalk has encoded in MacRoman (<8A>
>>>> and <9F> respectively). PostgreSQL takes exception to these and
>>>> Bacula generates messages like the following:
>>>>
>>>> 04-Jan 15:31 hub-dir: Annual_Backup.2008-01-04_15.26.38 Fatal
>>>> error: sql_create.c:870 sql_create.c:870 insert INSERT INTO
>>>> Filename (Name) VALUES ('2004-04-29 Z<9F>rich 0002.jpeg') failed:
>>>> ERROR: invalid byte sequence for encoding "UTF8": 0x9f
>>>> HINT: This error can also happen if the byte sequence does not
>>>> match the encoding expected by the server, which is controlled
>>>> by "client_encoding".
>>>>
>>>> because they fall into an area of UTF-8/Unicode/ISO-8859-1 which
>>>> seems to be unassigned (<82>-<8C> and <90>-<9F>).
>>>>
>>>> These files are not new and I didn't have this problem with my
>>>> old configuration, so something has presumably changed in Bacula
>>>> or PostgreSQL.
>>> Can you put one of those files (or just the filename) into a
>>> tarball and email it to me please? I will try to back it up here.
>> Read this slowly to see where I differ from you. I have not had any
>> backup failures yet.
>>
>> I have the files. Good news: The backup succeeded here. Here is
>> some of the job output:
>>
>> $ echo 'list files jobid=15984' | bconsole | grep nem
>> | /home/dan/bacula-nemesis/2004-05-05 Erg�nzung
>> 0002.jpeg | /home/dan/
>> bacula-nemesis/2004-05-05 Erg�nzung
>> 0001.jpeg | /home/dan/bacula-
>> nemesis/2004-04-29 Z�rich
>> 0002.jpeg | /home/dan/bacula-
>> nemesis/2004-04-29 Z�rich
>> 0001.jpeg | /home/dan/bacula-
>> nemesis/ | /home/dan/bacula-nemesis.tar.gz
>>
>> But then, I'm using SQL_ASCII here. Let me try UTF8 on another
>> system.
>>
>> Oh.. it worked there too:
>>
>> [EMAIL PROTECTED]:~] $ psql -l
>> List of databases
>> Name | Owner | Encoding
>> -----------+--------+-----------
>> bacula | bacula | UTF8
>> postgres | pgsql | SQL_ASCII
>> regress | dan | SQL_ASCII
>> template0 | pgsql | SQL_ASCII
>> template1 | pgsql | SQL_ASCII
>> (5 rows)
>>
>> [EMAIL PROTECTED]:~] $ bconsole
>> Connecting to Director ducky:9101
>> 1000 OK: ducky-dir Version: 2.2.6 (10 November 2007)
>> Enter a period to cancel a command.
>> *restore
>> Automatically selected Catalog: MyCatalog
>> Using Catalog "MyCatalog"
>>
>> First you select one or more JobIds that contain files
>> to be restored. You will be presented several methods
>> of specifying the JobIds. Then you will be allowed to
>> select which files from those JobIds are to be restored.
>>
>> To select the JobIds, you have the following choices:
>> 1: List last 20 Jobs run
>> 2: List Jobs where a given File is saved
>> 3: Enter list of comma separated JobIds to select
>> 4: Enter SQL list command
>> 5: Select the most recent backup for a client
>> 6: Select backup for a client before a specified time
>> 7: Enter a list of files to restore
>> 8: Enter a list of files to restore before a specified time
>> 9: Find the JobIds of the most recent backup for a client
>> 10: Find the JobIds for a backup for a client before a specified
>> time
>> 11: Enter a list of directories to restore for found JobIds
>> 12: Cancel
>> Select item: (1-12): 5
>> Automatically selected Client: ducky-fd
>> Automatically selected FileSet: Full Set
>> +-------+-------+----------+-------------+---------------------
>> +------------+
>> | jobid | level | jobfiles | jobbytes | starttime |
>> volumename |
>> +-------+-------+----------+-------------+---------------------
>> +------------+
>> | 2 | F | 20,689 | 344,879,617 | 2008-01-04 22:13:12 |
>> TestVol001 |
>> +-------+-------+----------+-------------+---------------------
>> +------------+
>> You have selected the following JobId: 2
>>
>> Building directory tree for JobId 2 ... +++++++++++++++++++++++++++++
>> ++++++++++++++++++
>> 1 Job, 19,459 files inserted into the tree.
>>
>> You are now entering file selection mode where you add (mark) and
>> remove (unmark) files to be restored. No files are initially added,
>> unless
>> you used the "all" keyword on the command line.
>> Enter "done" to leave this mode.
>>
>> cwd is: /
>> $ cd /home/dan
>> cwd is: /home/dan/
>> $ ls 2*
>> 2004-04-29 Z�rich 0001.jpeg
>> 2004-04-29 Z�rich 0002.jpeg
>> 2004-05-05 Erg�nzung 0001.jpeg
>> 2004-05-05 Erg�nzung 0002.jpeg
>> $
>>
>> $ [EMAIL PROTECTED]:~] $ psql bacula
>> Welcome to psql 8.2.5, the PostgreSQL interactive terminal.
>>
>> Type: \copyright for distribution terms
>> \h for help with SQL commands
>> \? for help with psql commands
>> \g or terminate with semicolon to execute query
>> \q to quit
>>
>> bacula=#
>>
>>
>> In short, I'm on a different version of PostgreSQL and Bacula.
>>
>> OK, let me upgrade Bacula to 2.2.7. Nope, that's OK too. So I'm on
>> the same Bacula and same PostgreSQL as you.
>>
>> Don't know what to tell you. :)
> Hi Dan
>
> Files created with the following Perl script reproduce the problem for
> me. I think it may give a more accurate test basis than the TGZ file I
> sent:
>
> #! /usr/bin/perl -w
>
> touch( "2004-04-29\ Z\237rich\ 0001.jpeg" );
> touch( "2004-04-29\ Z\237rich\ 0002.jpeg" );
> touch( "2004-05-05\ Erg\212nzung\ 0001.jpeg" );
> touch( "2004-05-05\ Erg\212nzung\ 0002.jpeg" );
>
> sub touch {
> my $filename = shift;
> open FILE, ">$filename";
> close FILE;
> }
I have confirmed a bug: the job silently fails without reporting the
following error, which is logged in /var/log/messages:
ERROR: invalid byte sequence for encoding "UTF8": 0x9f
HINT: This error can also happen if the byte sequence does not match
the encoding expected by the server is controlled by "client_encoding".
No entries are added to the file table. The error should be logged and
the backup at least not flagged as OK.
As for your situation, there are two questions remaining:
- is the file name valid UTF8?
- if it is, I wonder if we need to do a "set client_encoding 'utf-8'" first
--
Dan Langille - http://www.langille.org/
BSDCan - The Technical BSD Conference: http://www.bsdcan.org/
PGCon - The PostgreSQL Conference: http://www.pgcon.org/
-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
