Sorry missed the file :)
-----Original Message-----
From: Oliver Zeigermann [mailto:[EMAIL PROTECTED]
Sent: 29. januar 2004 10:47
To: Slide Users Mailing List
Subject: Re: TXFileStore and local filesystem
So, I think we have two problems now, I am endangered to mix up:
(1) The filestore has a problem with file names
(2) The dabase stores have a problem as well, which is yet unclear to me
Concerning (1): Could you send the new exception after the patch was
applied? At least the file name given in the exceptions head followed by
"Can not create resource at " should look different for me to see what
might be be going on.
Concerning (2): Could you describe this a bit more in order to make my
rusty mind understand?
Concerning the Unicode vs. UTF-8 issue: How would you decode a string
before storing into the database? Into what? The JDBC method accepts a
string, so you will have to pass it one. As I said, you can only
decode/encode into/from bytes...
Oliver
Jacob Lund wrote:
> The patch did not make any difference - it still throws the same
exception!
>
> What I meant about converting from UTF-8 to Unicode is that the database
> driver can handle Unicode. In the filestore UTF-8 is converted to local
> character set in order to create the files and this is why the filestore
(I
> think) has a problem. If the database could store the data in Unicode then
> there would be no problem. Since java is using Unicode in strings the task
> would simply be to decode the strings before they are stored in the
database
> and then make sure that all text fields in the database are Unicode (or
> widechar or nchar).
>
> Please tell me if I am way off here!
>
> /Jacob
>
> -----Original Message-----
> From: Oliver Zeigermann [mailto:[EMAIL PROTECTED]
> Sent: 29. januar 2004 10:02
> To: Slide Users Mailing List
> Subject: Re: TXFileStore and local filesystem
>
> Jacob Lund wrote:
>
>>No, the filestore works correctly.
>
>
> OK, shall I check in the patch? Did it work for you?
>
>
>>>From what I can see the filestore converts from UTF-8 to local before it
>>stores data. This I why UTF-8 works fine for me when I upload files with
>>Danish letters in the filename, and also why if fails when it stores files
>>with characters not supported by the codepage.
>>
>>Windows XP use Unicode, but in "dos mode" it will use the old codepage
>>types. The only thing that I can imagine is that java will use this
>
> codepage
>
>>when it is doing IO operations towards the filesystem. This problem might
>
> be
>
>>a problem that only appears on windows systems.
>>
>>I do not think that the problem is in the fill data into the database that
>>has a problem. Some place in slide it will convert that data (in this case
>>the uri) to UTF-8 before it is send to the client. The data stored in the
>>database is UTF-8, and I believe that java is using Unicode. So the
>
> solution
>
>>might be to convert data fetched from the database back to Unicode as soon
>>as it arrives to the store class.
>>
>>The correct solution might be to convert from UTF-8 to Unicode before
>>storing the data and then change the database scheme to Unicode char in
>
> all
>
>>fields containing strings.
>
>
> Hmmmm. You might be confusing certain things here. On one side there is
> Unicode having a number for each character. On the other side there is
> the representation in bytes. Now, UTF-8 *is* Unicode, but on the other
> side, i.e. the representation in bytes. Thus it does not make too much
> sense to compare Unicode with UTF-8. Do you agree?
>
>
>>I am guessing here since I do not have any idea of how the stores are
>>structured in slide. I you want I would be happy to do some debugging, but
>
> I
>
>>will need a short introduction to how the datastores are designed in
>
> slide.
>
> I know, proper documentation is a major problem. I will try to prepare
> something like a short introduction and will post it to the list as soon
> as it is done. This may take a while though :(
>
> Oliver
>
>
>>/Jacob
>>
>>-----Original Message-----
>>From: Oliver Zeigermann [mailto:[EMAIL PROTECTED]
>>Sent: 28. januar 2004 16:40
>>To: Slide Users Mailing List
>>Subject: Re: TXFileStore and local filesystem
>>
>>Jacob Lund wrote:
>>
>>
>>
>>>Sorry about that - yes I am talking about the URI!
>>>
>>>If I look in a record in the database, each Danish character is stored as
>>>two "funny looking" characters corresponding to the unescaped UTF-8
>>
>>encoded
>>
>>
>>>version - so this looks correct! However when I do a propfind on the
>>>collection I which I place this file, then I get something like this
>>>/files/%C3%83%C2%B8 - and this should have been representing one Danish
>>>character. If I take the above and convert from UTF8 to my local, then I
>>
>>get
>>
>>
>>>what is store in the database - If I then convert from UTF8 to local
again
>>>the I get the correct Danish letter.
>>
>>
>>I could not find anything that might have converted the URI strings.
>>They are just plainly filled into the SQL like in
>>
>>
>>
>>> "select 1 from OBJECT o, URI u where
>>
>>o.URI_ID=u.URI_ID and u.URI_STRING=?");
>>
>>
>>> statement.setString(1, uri.toString());
>>
>>
>>So, maybe this is a more general problem...
>>
>>
>>
>>>I seem that slide converts the URI's from the db to UTF8, but they are
>>>already stored in unescaped UTF-8!
>>
>>
>>Does this happen with the file store as well?
>>
>>Oliver
>>
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: [EMAIL PROTECTED]
>>For additional commands, e-mail: [EMAIL PROTECTED]
>>
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: [EMAIL PROTECTED]
>>For additional commands, e-mail: [EMAIL PROTECTED]
>>
>>
>>.
>>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
> .
>
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
/**********************************************************************/
/****** DROP SLIDE TABLES ******/
/**********************************************************************/
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[OBJECT]') and
OBJECTPROPERTY(id, N'IsUserTable') = 1)
DROP TABLE [dbo].[OBJECT]
GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[BINDING]') and
OBJECTPROPERTY(id, N'IsUserTable') = 1)
DROP TABLE [dbo].[BINDING]
GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id =
OBJECT_ID(N'[dbo].[PARENT_BINDING]') and OBJECTPROPERTY(id, N'IsUserTable') = 1)
DROP TABLE [dbo].[PARENT_BINDING]
GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[CHILDREN]') and
OBJECTPROPERTY(id, N'IsUserTable') = 1)
DROP TABLE [dbo].[CHILDREN]
GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[LINKS]') and
OBJECTPROPERTY(id, N'IsUserTable') = 1)
DROP TABLE [dbo].[LINKS]
GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[LOCKS]') and
OBJECTPROPERTY(id, N'IsUserTable') = 1)
DROP TABLE [dbo].[LOCKS]
GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id =
OBJECT_ID(N'[dbo].[VERSION_CONTENT]') and OBJECTPROPERTY(id, N'IsUserTable') = 1)
DROP TABLE [dbo].[VERSION_CONTENT]
GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[PROPERTIES]')
and OBJECTPROPERTY(id, N'IsUserTable') = 1)
DROP TABLE [dbo].[PROPERTIES]
GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[PERMISSIONS]')
and OBJECTPROPERTY(id, N'IsUserTable') = 1)
DROP TABLE [dbo].[PERMISSIONS]
GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[VERSION_PREDS]')
and OBJECTPROPERTY(id, N'IsUserTable') = 1)
DROP TABLE [dbo].[VERSION_PREDS]
GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id =
OBJECT_ID(N'[dbo].[VERSION_LABELS]') and OBJECTPROPERTY(id, N'IsUserTable') = 1)
DROP TABLE [dbo].[VERSION_LABELS]
GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id =
OBJECT_ID(N'[dbo].[VERSION_HISTORY]') and OBJECTPROPERTY(id, N'IsUserTable') = 1)
DROP TABLE [dbo].[VERSION_HISTORY]
GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[VERSION]') and
OBJECTPROPERTY(id, N'IsUserTable') = 1)
DROP TABLE [dbo].[VERSION]
GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[BRANCH]') and
OBJECTPROPERTY(id, N'IsUserTable') = 1)
DROP TABLE [dbo].[BRANCH]
GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[LABEL]') and
OBJECTPROPERTY(id, N'IsUserTable') = 1)
DROP TABLE [dbo].[LABEL]
GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[URI]') AND
OBJECTPROPERTY(id, N'IsUserTable') = 1)
DROP TABLE [dbo].[URI]
GO
/**********************************************************************/
/****** DROP EXISTING USER DEFINED DATA TYPES ******/
/**********************************************************************/
sp_droptype id_type
GO
sp_droptype uri_str_type
GO
sp_droptype revision_no_type
GO
sp_droptype hash_type
GO
sp_droptype literal_str_type
GO
sp_droptype value_str_type
GO
/**********************************************************************/
/****** ADD USER DEFINED DATA TYPES ******/
/**********************************************************************/
sp_addtype id_type, bigint
GO
sp_addtype uri_str_type, "nvarchar(800)"
GO
sp_addtype revision_no_type, "nvarchar(20)"
GO
sp_addtype hash_type, bigint
GO
sp_addtype literal_str_type, "nvarchar(3000)"
GO
sp_addtype value_str_type, "nvarchar(255)"
GO
/**********************************************************************/
/****** CREATE SLIDE TABLES ******/
/**********************************************************************/
CREATE TABLE dbo.URI (
URI_ID id_type IDENTITY UNIQUE NOT NULL,
URI_STRING uri_str_type UNIQUE NOT NULL,
-- UNIQUE NONCLUSTERED (URI_ID)
)
GO
CREATE INDEX XUID
ON URI(URI_ID)
GO
CREATE INDEX XUSTRING
ON URI(URI_STRING)
GO
CREATE TABLE dbo.OBJECT (
URI_ID id_type PRIMARY KEY,
CLASS_NAME nvarchar(255) NOT NULL,
CONSTRAINT FK_OBJECT_URI_ID
FOREIGN KEY (URI_ID)
REFERENCES URI (URI_ID)
)
GO
CREATE TABLE dbo.BINDING (
URI_ID id_type NOT NULL
REFERENCES URI (URI_ID),
NAME uri_str_type NOT NULL,
CHILD_UURI_ID id_type NOT NULL
REFERENCES URI (URI_ID),
UNIQUE CLUSTERED (URI_ID, NAME, CHILD_UURI_ID)
)
GO
CREATE TABLE dbo.PARENT_BINDING (
URI_ID id_type NOT NULL
REFERENCES URI (URI_ID),
NAME uri_str_type NOT NULL,
PARENT_UURI_ID id_type NOT NULL
REFERENCES URI (URI_ID),
UNIQUE CLUSTERED (URI_ID, NAME, PARENT_UURI_ID)
)
GO
CREATE TABLE dbo.LINKS (
URI_ID id_type NOT NULL FOREIGN KEY
REFERENCES URI (URI_ID),
LINK_TO_ID id_type NOT NULL FOREIGN KEY
REFERENCES URI (URI_ID),
--UNIQUE CLUSTERED (URI_ID, LINK_TO_ID)
)
GO
CREATE INDEX XURI_ID
ON LINKS(URI_ID)
GO
CREATE INDEX XLINK_TO_ID
ON LINKS(LINK_TO_ID)
GO
CREATE TABLE dbo.LOCKS (
LOCK_ID id_type PRIMARY KEY,
OBJECT_ID id_type NOT NULL FOREIGN KEY
REFERENCES URI (URI_ID),
SUBJECT_ID id_type NOT NULL FOREIGN KEY
REFERENCES URI (URI_ID),
TYPE_ID id_type NOT NULL FOREIGN KEY
REFERENCES URI (URI_ID),
EXPIRATION_DATE numeric(14, 0) NOT NULL,
IS_INHERITABLE bit NOT NULL,
IS_EXCLUSIVE bit NOT NULL,
OWNER nvarchar(255),
CONSTRAINT FK_LOCKS_LOCK_ID
FOREIGN KEY (LOCK_ID)
REFERENCES URI (URI_ID)
)
GO
CREATE TABLE dbo.BRANCH (
BRANCH_ID id_type IDENTITY UNIQUE NOT NULL,
BRANCH_STRING nvarchar(255) UNIQUE NOT NULL,
UNIQUE NONCLUSTERED (BRANCH_ID)
)
GO
CREATE TABLE dbo.LABEL (
LABEL_ID id_type IDENTITY UNIQUE NOT NULL,
LABEL_STRING nvarchar(255) NOT NULL,
UNIQUE NONCLUSTERED (LABEL_ID)
)
GO
CREATE TABLE dbo.VERSION (
URI_ID id_type PRIMARY KEY,
IS_VERSIONED bit NOT NULL,
CONSTRAINT FK_VERSION_URI_ID
FOREIGN KEY (URI_ID)
REFERENCES URI (URI_ID)
)
GO
CREATE TABLE dbo.VERSION_HISTORY (
VERSION_ID id_type IDENTITY UNIQUE NOT NULL,
URI_ID id_type NOT NULL FOREIGN KEY
REFERENCES VERSION (URI_ID),
BRANCH_ID id_type NOT NULL FOREIGN KEY
REFERENCES BRANCH (BRANCH_ID),
REVISION_NO nVARCHAR(20) NOT NULL,
--UNIQUE CLUSTERED (URI_ID, BRANCH_ID, REVISION_NO)
)
GO
CREATE INDEX XVERSION_HISTORY1
ON VERSION_HISTORY(URI_ID, BRANCH_ID, REVISION_NO)
GO
CREATE TABLE dbo.VERSION_PREDS (
VERSION_ID id_type NOT NULL FOREIGN KEY
REFERENCES VERSION_HISTORY (VERSION_ID),
PREDECESSOR_ID id_type NOT NULL FOREIGN KEY
REFERENCES VERSION_HISTORY (VERSION_ID),
UNIQUE CLUSTERED (VERSION_ID, PREDECESSOR_ID)
)
GO
CREATE INDEX XVERSION_PREDS1
ON VERSION_PREDS(VERSION_ID, PREDECESSOR_ID)
GO
CREATE TABLE dbo.VERSION_LABELS (
VERSION_ID id_type NOT NULL FOREIGN KEY
REFERENCES VERSION_HISTORY (VERSION_ID),
LABEL_ID id_type NOT NULL FOREIGN KEY
REFERENCES LABEL (LABEL_ID),
UNIQUE CLUSTERED (VERSION_ID, LABEL_ID)
)
GO
CREATE TABLE dbo.VERSION_CONTENT (
VERSION_ID id_type PRIMARY KEY,
CONTENT image NOT NULL,
CONSTRAINT FK_VC_VERSION_ID
FOREIGN KEY (VERSION_ID)
REFERENCES VERSION_HISTORY (VERSION_ID),
)
GO
CREATE TABLE dbo.PROPERTIES (
VERSION_ID id_type NOT NULL FOREIGN KEY
REFERENCES VERSION_HISTORY (VERSION_ID),
PROPERTY_NAMESPACE nvarchar(50) NOT NULL,
PROPERTY_NAME nvarchar(50) NOT NULL,
PROPERTY_VALUE nvarchar(255) NOT NULL,
PROPERTY_TYPE nvarchar(50) NOT NULL,
IS_PROTECTED bit NOT NULL,
UNIQUE CLUSTERED (VERSION_ID, PROPERTY_NAMESPACE, PROPERTY_NAME)
)
GO
CREATE TABLE dbo.PERMISSIONS (
OBJECT_ID id_type NOT NULL FOREIGN KEY
REFERENCES URI (URI_ID),
SUBJECT_ID id_type NOT NULL FOREIGN KEY
REFERENCES URI (URI_ID),
ACTION_ID id_type NOT NULL FOREIGN KEY
REFERENCES URI (URI_ID),
VERSION_NO nVARCHAR(20) NULL,
IS_INHERITABLE bit NOT NULL,
IS_NEGATIVE bit NOT NULL,
-- Both order and sequence would be more suitable, but can not be used
SUCCESSION int NOT NULL,
UNIQUE CLUSTERED (OBJECT_ID, SUBJECT_ID, ACTION_ID),
UNIQUE (OBJECT_ID, SUCCESSION)
)
GO
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
