Unfortunately, this will not go into the head branch soon as I had started the merging process shortly before that patch. It will be there next week, though...
Oliver
Jacob Lund wrote:
Sorry missed the file :)
-----Original Message-----
From: Oliver Zeigermann [mailto:[EMAIL PROTECTED] Sent: 29. januar 2004 10:47
To: Slide Users Mailing List
Subject: Re: TXFileStore and local filesystem
So, I think we have two problems now, I am endangered to mix up:
(1) The filestore has a problem with file names (2) The dabase stores have a problem as well, which is yet unclear to me
Concerning (1): Could you send the new exception after the patch was applied? At least the file name given in the exceptions head followed by "Can not create resource at " should look different for me to see what might be be going on.
Concerning (2): Could you describe this a bit more in order to make my rusty mind understand?
Concerning the Unicode vs. UTF-8 issue: How would you decode a string before storing into the database? Into what? The JDBC method accepts a string, so you will have to pass it one. As I said, you can only decode/encode into/from bytes...
Oliver
Jacob Lund wrote:
The patch did not make any difference - it still throws the same
exception!
What I meant about converting from UTF-8 to Unicode is that the database driver can handle Unicode. In the filestore UTF-8 is converted to local character set in order to create the files and this is why the filestore
(I
think) has a problem. If the database could store the data in Unicode then there would be no problem. Since java is using Unicode in strings the task would simply be to decode the strings before they are stored in the
database
and then make sure that all text fields in the database are Unicode (or widechar or nchar).
Please tell me if I am way off here!
/Jacob
-----Original Message-----
From: Oliver Zeigermann [mailto:[EMAIL PROTECTED] Sent: 29. januar 2004 10:02
To: Slide Users Mailing List
Subject: Re: TXFileStore and local filesystem
Jacob Lund wrote:
No, the filestore works correctly.
OK, shall I check in the patch? Did it work for you?
From what I can see the filestore converts from UTF-8 to local before itstores data. This I why UTF-8 works fine for me when I upload files with Danish letters in the filename, and also why if fails when it stores files with characters not supported by the codepage.
Windows XP use Unicode, but in "dos mode" it will use the old codepage types. The only thing that I can imagine is that java will use this
codepage
when it is doing IO operations towards the filesystem. This problem might
be
a problem that only appears on windows systems.
I do not think that the problem is in the fill data into the database that has a problem. Some place in slide it will convert that data (in this case the uri) to UTF-8 before it is send to the client. The data stored in the database is UTF-8, and I believe that java is using Unicode. So the
solution
might be to convert data fetched from the database back to Unicode as soon as it arrives to the store class.
The correct solution might be to convert from UTF-8 to Unicode before storing the data and then change the database scheme to Unicode char in
all
fields containing strings.
Hmmmm. You might be confusing certain things here. On one side there is Unicode having a number for each character. On the other side there is the representation in bytes. Now, UTF-8 *is* Unicode, but on the other side, i.e. the representation in bytes. Thus it does not make too much sense to compare Unicode with UTF-8. Do you agree?
I am guessing here since I do not have any idea of how the stores are structured in slide. I you want I would be happy to do some debugging, but
I
will need a short introduction to how the datastores are designed in
slide.
I know, proper documentation is a major problem. I will try to prepare something like a short introduction and will post it to the list as soon as it is done. This may take a while though :(
Oliver
/Jacob
-----Original Message-----
From: Oliver Zeigermann [mailto:[EMAIL PROTECTED] Sent: 28. januar 2004 16:40
To: Slide Users Mailing List
Subject: Re: TXFileStore and local filesystem
Jacob Lund wrote:
Sorry about that - yes I am talking about the URI!
If I look in a record in the database, each Danish character is stored as two "funny looking" characters corresponding to the unescaped UTF-8
encoded
version - so this looks correct! However when I do a propfind on the collection I which I place this file, then I get something like this /files/%C3%83%C2%B8 - and this should have been representing one Danish character. If I take the above and convert from UTF8 to my local, then I
get
what is store in the database - If I then convert from UTF8 to local
again
the I get the correct Danish letter.
I could not find anything that might have converted the URI strings. They are just plainly filled into the SQL like in
"select 1 from OBJECT o, URI u where
o.URI_ID=u.URI_ID and u.URI_STRING=?");
statement.setString(1, uri.toString());
So, maybe this is a more general problem...
I seem that slide converts the URI's from the db to UTF8, but they are already stored in unescaped UTF-8!
Does this happen with the file store as well?
Oliver
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
.
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
.
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
------------------------------------------------------------------------
/**********************************************************************/ /****** DROP SLIDE TABLES ******/ /**********************************************************************/
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[OBJECT]') and OBJECTPROPERTY(id, N'IsUserTable') = 1) DROP TABLE [dbo].[OBJECT] GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[BINDING]') and OBJECTPROPERTY(id, N'IsUserTable') = 1) DROP TABLE [dbo].[BINDING] GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[PARENT_BINDING]') and OBJECTPROPERTY(id, N'IsUserTable') = 1) DROP TABLE [dbo].[PARENT_BINDING] GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[CHILDREN]') and OBJECTPROPERTY(id, N'IsUserTable') = 1) DROP TABLE [dbo].[CHILDREN] GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[LINKS]') and OBJECTPROPERTY(id, N'IsUserTable') = 1) DROP TABLE [dbo].[LINKS] GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[LOCKS]') and OBJECTPROPERTY(id, N'IsUserTable') = 1) DROP TABLE [dbo].[LOCKS] GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[VERSION_CONTENT]') and OBJECTPROPERTY(id, N'IsUserTable') = 1) DROP TABLE [dbo].[VERSION_CONTENT] GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[PROPERTIES]') and OBJECTPROPERTY(id, N'IsUserTable') = 1) DROP TABLE [dbo].[PROPERTIES] GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[PERMISSIONS]') and OBJECTPROPERTY(id, N'IsUserTable') = 1) DROP TABLE [dbo].[PERMISSIONS] GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[VERSION_PREDS]') and OBJECTPROPERTY(id, N'IsUserTable') = 1) DROP TABLE [dbo].[VERSION_PREDS] GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[VERSION_LABELS]') and OBJECTPROPERTY(id, N'IsUserTable') = 1) DROP TABLE [dbo].[VERSION_LABELS] GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[VERSION_HISTORY]') and OBJECTPROPERTY(id, N'IsUserTable') = 1) DROP TABLE [dbo].[VERSION_HISTORY] GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[VERSION]') and OBJECTPROPERTY(id, N'IsUserTable') = 1) DROP TABLE [dbo].[VERSION] GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[BRANCH]') and OBJECTPROPERTY(id, N'IsUserTable') = 1) DROP TABLE [dbo].[BRANCH] GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[LABEL]') and OBJECTPROPERTY(id, N'IsUserTable') = 1) DROP TABLE [dbo].[LABEL] GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[URI]') AND OBJECTPROPERTY(id, N'IsUserTable') = 1) DROP TABLE [dbo].[URI] GO
/**********************************************************************/ /****** DROP EXISTING USER DEFINED DATA TYPES ******/ /**********************************************************************/
sp_droptype id_type GO
sp_droptype uri_str_type GO
sp_droptype revision_no_type GO
sp_droptype hash_type GO
sp_droptype literal_str_type GO
sp_droptype value_str_type GO
/**********************************************************************/ /****** ADD USER DEFINED DATA TYPES ******/ /**********************************************************************/
sp_addtype id_type, bigint GO
sp_addtype uri_str_type, "nvarchar(800)" GO
sp_addtype revision_no_type, "nvarchar(20)" GO
sp_addtype hash_type, bigint GO
sp_addtype literal_str_type, "nvarchar(3000)" GO
sp_addtype value_str_type, "nvarchar(255)" GO
/**********************************************************************/ /****** CREATE SLIDE TABLES ******/ /**********************************************************************/
CREATE TABLE dbo.URI ( URI_ID id_type IDENTITY UNIQUE NOT NULL, URI_STRING uri_str_type UNIQUE NOT NULL, -- UNIQUE NONCLUSTERED (URI_ID) ) GO
CREATE INDEX XUID
ON URI(URI_ID) GO
CREATE INDEX XUSTRING
ON URI(URI_STRING) GO
CREATE TABLE dbo.OBJECT (
URI_ID id_type PRIMARY KEY,
CLASS_NAME nvarchar(255) NOT NULL,
CONSTRAINT FK_OBJECT_URI_ID
FOREIGN KEY (URI_ID)
REFERENCES URI (URI_ID)
)
GO
CREATE TABLE dbo.BINDING ( URI_ID id_type NOT NULL REFERENCES URI (URI_ID), NAME uri_str_type NOT NULL, CHILD_UURI_ID id_type NOT NULL REFERENCES URI (URI_ID), UNIQUE CLUSTERED (URI_ID, NAME, CHILD_UURI_ID) ) GO
CREATE TABLE dbo.PARENT_BINDING (
URI_ID id_type NOT NULL
REFERENCES URI (URI_ID),
NAME uri_str_type NOT NULL,
PARENT_UURI_ID id_type NOT NULL
REFERENCES URI (URI_ID),
UNIQUE CLUSTERED (URI_ID, NAME, PARENT_UURI_ID)
) GO
CREATE TABLE dbo.LINKS ( URI_ID id_type NOT NULL FOREIGN KEY REFERENCES URI (URI_ID), LINK_TO_ID id_type NOT NULL FOREIGN KEY REFERENCES URI (URI_ID), --UNIQUE CLUSTERED (URI_ID, LINK_TO_ID) ) GO
CREATE INDEX XURI_ID
ON LINKS(URI_ID) GO
CREATE INDEX XLINK_TO_ID
ON LINKS(LINK_TO_ID) GO
CREATE TABLE dbo.LOCKS (
LOCK_ID id_type PRIMARY KEY,
OBJECT_ID id_type NOT NULL FOREIGN KEY
REFERENCES URI (URI_ID),
SUBJECT_ID id_type NOT NULL FOREIGN KEY
REFERENCES URI (URI_ID),
TYPE_ID id_type NOT NULL FOREIGN KEY
REFERENCES URI (URI_ID),
EXPIRATION_DATE numeric(14, 0) NOT NULL,
IS_INHERITABLE bit NOT NULL, IS_EXCLUSIVE bit NOT NULL,
OWNER nvarchar(255),
CONSTRAINT FK_LOCKS_LOCK_ID
FOREIGN KEY (LOCK_ID)
REFERENCES URI (URI_ID)
)
GO
CREATE TABLE dbo.BRANCH ( BRANCH_ID id_type IDENTITY UNIQUE NOT NULL,
BRANCH_STRING nvarchar(255) UNIQUE NOT NULL,
UNIQUE NONCLUSTERED (BRANCH_ID)
)
GO
CREATE TABLE dbo.LABEL ( LABEL_ID id_type IDENTITY UNIQUE NOT NULL, LABEL_STRING nvarchar(255) NOT NULL, UNIQUE NONCLUSTERED (LABEL_ID) ) GO
CREATE TABLE dbo.VERSION (
URI_ID id_type PRIMARY KEY,
IS_VERSIONED bit NOT NULL, CONSTRAINT FK_VERSION_URI_ID
FOREIGN KEY (URI_ID)
REFERENCES URI (URI_ID)
)
GO
CREATE TABLE dbo.VERSION_HISTORY (
VERSION_ID id_type IDENTITY UNIQUE NOT NULL,
URI_ID id_type NOT NULL FOREIGN KEY
REFERENCES VERSION (URI_ID),
BRANCH_ID id_type NOT NULL FOREIGN KEY
REFERENCES BRANCH (BRANCH_ID),
REVISION_NO nVARCHAR(20) NOT NULL,
--UNIQUE CLUSTERED (URI_ID, BRANCH_ID, REVISION_NO)
)
GO
CREATE INDEX XVERSION_HISTORY1 ON VERSION_HISTORY(URI_ID, BRANCH_ID, REVISION_NO) GO
CREATE TABLE dbo.VERSION_PREDS (
VERSION_ID id_type NOT NULL FOREIGN KEY REFERENCES VERSION_HISTORY (VERSION_ID),
PREDECESSOR_ID id_type NOT NULL FOREIGN KEY
REFERENCES VERSION_HISTORY (VERSION_ID),
UNIQUE CLUSTERED (VERSION_ID, PREDECESSOR_ID)
)
GO
CREATE INDEX XVERSION_PREDS1 ON VERSION_PREDS(VERSION_ID, PREDECESSOR_ID) GO
CREATE TABLE dbo.VERSION_LABELS (
VERSION_ID id_type NOT NULL FOREIGN KEY
REFERENCES VERSION_HISTORY (VERSION_ID),
LABEL_ID id_type NOT NULL FOREIGN KEY
REFERENCES LABEL (LABEL_ID), UNIQUE CLUSTERED (VERSION_ID, LABEL_ID)
)
GO
CREATE TABLE dbo.VERSION_CONTENT (
VERSION_ID id_type PRIMARY KEY,
CONTENT image NOT NULL,
CONSTRAINT FK_VC_VERSION_ID FOREIGN KEY (VERSION_ID)
REFERENCES VERSION_HISTORY (VERSION_ID),
)
GO
CREATE TABLE dbo.PROPERTIES (
VERSION_ID id_type NOT NULL FOREIGN KEY
REFERENCES VERSION_HISTORY (VERSION_ID), PROPERTY_NAMESPACE nvarchar(50) NOT NULL, PROPERTY_NAME nvarchar(50) NOT NULL, PROPERTY_VALUE nvarchar(255) NOT NULL,
PROPERTY_TYPE nvarchar(50) NOT NULL, IS_PROTECTED bit NOT NULL,
UNIQUE CLUSTERED (VERSION_ID, PROPERTY_NAMESPACE, PROPERTY_NAME)
)
GO
CREATE TABLE dbo.PERMISSIONS ( OBJECT_ID id_type NOT NULL FOREIGN KEY REFERENCES URI (URI_ID), SUBJECT_ID id_type NOT NULL FOREIGN KEY REFERENCES URI (URI_ID), ACTION_ID id_type NOT NULL FOREIGN KEY REFERENCES URI (URI_ID), VERSION_NO nVARCHAR(20) NULL, IS_INHERITABLE bit NOT NULL, IS_NEGATIVE bit NOT NULL, -- Both order and sequence would be more suitable, but can not be used SUCCESSION int NOT NULL, UNIQUE CLUSTERED (OBJECT_ID, SUBJECT_ID, ACTION_ID), UNIQUE (OBJECT_ID, SUCCESSION) ) GO
------------------------------------------------------------------------
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
