Re: [IMAP] Cut a milestone ?
Hi, that's me again, I just posted some questions on openjpa ml. If we want to keep JPAStreamingMessage without intermediary tables, we may use the proprietary @ElementJoinColumn annotation. For other features, such as index creation, we also probably need proprietary annotations. For example, the current findUnseenMessagesInMailboxOrderByUid SELECT membership FROM Membership membership WHERE membership.mailboxId = :idParam AND membership.seen = FALSE ORDER BY membership.uid ASC could benefit from @Index @Id private long uid; @Index @Id private long mailboxId; So the questions is Should we use those openjpa annotations?. I personally don't see why we couldn't. What do you think of? Btw, I think the original question was Cut a milestone ?... :) Tks, Eric PS: Better use the apache-james-imap-jpa-0.1-M2-SNAPSHOT.jar enhanced via maven in eclipse. If you use the project class folder, the enhancement is not always as it should. Many problems during dev comes from eclipse (or whatever) does not enhance, or take the previous enhancement... F5 is also your friend. On 06/17/2010 11:33 PM, Eric Charles wrote: Hi Tim, Sorry, message_id was indeed needed in the patch, otherwise you've got a id column which is pk and fk. I still made some additional tests, and omitting the @JoinColumn also works. DB is created and mail delivery is correctly working when you apply on JPAProperty. As soon as you apply on JPAProperty and JPAHeader, it hangs. Still looking for. Bye, Eric On 06/17/2010 08:26 AM, Eric Charles wrote: Hi Tim, Comments inside. Tks, Eric On 06/17/2010 12:58 AM, Tim-Christian Mundt wrote: Hey Eric, For example, with Derby, the schema is create with GENERATED ALWAYS columns and not GENERATED BY DEFAULT. Why's that? Wouldn't GENERATED BY DEFAULT solve this problem so we could happily use autoincrements? GENERATED BY DEFAULT would help, but openjpa create with GENERATED ALWAYS. Moreover, when you recreate your database, you need to define a START value depending on your last generated key. We could also define a key-generation-table per table, but I find this overkill. I agree, that's too much. OK I would tend to think to leave it like it is. If you consider the issue above really serious, than lets keep it. On the other hand: nobody would use derby in production, right? Do similar problems exist with other databases? I sometimes use derby in production with low-end PCs as servers. Each database has it own way and syntax for managing pk generation, with its goodness and pitfalls. For the direct @OneToMany, we have issues when implemented in James even if this samples show that it should be correct. Strange... On my side something in my dummy project is just odd. Can't get this running at all. JPA seems to be a bit delicate? OK You said JPA 2.0 defines a way to define the association only in the parent (*Message): Can you send me a working sample? As said: I can't really get it working here. I've seen it on this site: http://en.wikibooks.org/wiki/Java_Persistence/OneToMany#Example_of_a_JPA_2.0_unidirectional_OneToMany_relationship_database The @JoinColumn is put on the other side using referencedColumnName. this sample may be a non working one. One more point is the sql generated. The logs show jpql, and not sql. Do you know if it's possible to view the sql in the logs (I didn't find a way). No, idea, sorry. I'm happy with jpql :) OK. Was just curious to see the generated sql that can show if the database queries are efficient or not. If not possible, can you log on the database (mysql I think you use)? MySQL provides logs for all operations, yes. Will try to get these from derby. I noticed a small issue with your patch: You have to use message_id instead of id for the name of the @JoinColumn. Patch was working on my side with tables and fk created Strange, but I think to remember it was working with both (id and message_id) Lets ask on the OpenJPA ML how to make OpenJPA use the relationship. Maybe they can even have a look at our concrete issue with James. Will you ask? Should I? I will post something in the coming days. But it is working as designed with the samples. They may say it is in the way we are using it. Maybe someone will take time to help us solve it. On the other hand, I'm not sure it is an issue to leave those intermediary table and I don't think it should be a blocker for the release. Best, Tim - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail:
Re: [IMAP] Cut a milestone ?
Hi Tim, Sorry, message_id was indeed needed in the patch, otherwise you've got a id column which is pk and fk. I still made some additional tests, and omitting the @JoinColumn also works. DB is created and mail delivery is correctly working when you apply on JPAProperty. As soon as you apply on JPAProperty and JPAHeader, it hangs. Still looking for. Bye, Eric On 06/17/2010 08:26 AM, Eric Charles wrote: Hi Tim, Comments inside. Tks, Eric On 06/17/2010 12:58 AM, Tim-Christian Mundt wrote: Hey Eric, For example, with Derby, the schema is create with GENERATED ALWAYS columns and not GENERATED BY DEFAULT. Why's that? Wouldn't GENERATED BY DEFAULT solve this problem so we could happily use autoincrements? GENERATED BY DEFAULT would help, but openjpa create with GENERATED ALWAYS. Moreover, when you recreate your database, you need to define a START value depending on your last generated key. We could also define a key-generation-table per table, but I find this overkill. I agree, that's too much. OK I would tend to think to leave it like it is. If you consider the issue above really serious, than lets keep it. On the other hand: nobody would use derby in production, right? Do similar problems exist with other databases? I sometimes use derby in production with low-end PCs as servers. Each database has it own way and syntax for managing pk generation, with its goodness and pitfalls. For the direct @OneToMany, we have issues when implemented in James even if this samples show that it should be correct. Strange... On my side something in my dummy project is just odd. Can't get this running at all. JPA seems to be a bit delicate? OK You said JPA 2.0 defines a way to define the association only in the parent (*Message): Can you send me a working sample? As said: I can't really get it working here. I've seen it on this site: http://en.wikibooks.org/wiki/Java_Persistence/OneToMany#Example_of_a_JPA_2.0_unidirectional_OneToMany_relationship_database The @JoinColumn is put on the other side using referencedColumnName. this sample may be a non working one. One more point is the sql generated. The logs show jpql, and not sql. Do you know if it's possible to view the sql in the logs (I didn't find a way). No, idea, sorry. I'm happy with jpql :) OK. Was just curious to see the generated sql that can show if the database queries are efficient or not. If not possible, can you log on the database (mysql I think you use)? MySQL provides logs for all operations, yes. Will try to get these from derby. I noticed a small issue with your patch: You have to use message_id instead of id for the name of the @JoinColumn. Patch was working on my side with tables and fk created Strange, but I think to remember it was working with both (id and message_id) Lets ask on the OpenJPA ML how to make OpenJPA use the relationship. Maybe they can even have a look at our concrete issue with James. Will you ask? Should I? I will post something in the coming days. But it is working as designed with the samples. They may say it is in the way we are using it. Maybe someone will take time to help us solve it. On the other hand, I'm not sure it is an issue to leave those intermediary table and I don't think it should be a blocker for the release. Best, Tim - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: [IMAP] Cut a milestone ?
Hi Eric, You will find attached (removed from ml, so bcc to you): seems like attachments are not stripped via ML, both emails here have the patch attached. I have to define the OneToMany on the concrete class (the JPAMessage). Yes, that's what I tried then. If this is done without changing JPAHeader/JPAProperty it should be ok, right? I spent hours trying to solve this without success. I can see that from the time you sent this mail... oh boy, that's late/early. One more point: for the insert to work, you need to add property name=openjpa.InverseManager value=true/ in persistence.xml (not in the patch) Is this still standard JPA? Just asking because if so, why would we need an extra option? Currently, the patch still gives issues when reading mails (null pointer on the properties list). Apparently, although the tables are created they are not used. Weird. Even if we solve these last points, we will remain with another issue... Indeed, in the sample, the type of the MayToOne needs to be a concrete class, otherwise openjpa complains. This means that you would be obliged in the Property to link to JPAMessage or JPAStreamingMessage. You can't link to AbstractJPAMessage. JPA 2.0 defines a way to define the association only in the parent (*Message). Then that would be ok. Although it's still strange that it works in your tests but not in James (also not for me). About the pk creation, I will reply a bit later. I reconsidered this issue and I think we should switch to Strategy.IDENTITY = autoincrement because that would allow other systems and implementations to work with this database without messing with the sequence table. Moreover, it's faster. Thanks for the files. Will check them out tonight. Best, Tim - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: [IMAP] Cut a milestone ?
Hi Tim, I'm a big fan of autoincrement pk columns per table. Each table has its own responsiblity, and keys are generated sequentially per table. This gives however requirements when you migrate via export/import to fresh database. For example, with Derby, the schema is create with GENERATED ALWAYS columns and not GENERATED BY DEFAULT. You will have issues on fk constraints even if you apply the procedure described in http://db.apache.org/derby/docs/dev/tools/ctoolsimportidentitycol.html. The current solution uses a pool of 100 keys in memory, and access to database is limited. We could also define a key-generation-table per table, but I find this overkill. I would tend to think to leave it like it is. For the direct @OneToMany, we have issues when implemented in James even if this samples show that it should be correct. Strange... You said JPA 2.0 defines a way to define the association only in the parent (*Message): Can you send me a working sample? One more point is the sql generated. The logs show jpql, and not sql. Do you know if it's possible to view the sql in the logs (I didn't find a way). If not possible, can you log on the database (mysql I think you use)? On my side, I could analyse the derby queryplan and see if we would gain something without intermediate table. Tks, Eric On 06/16/2010 08:37 AM, Tim-Christian Mundt wrote: Hi Eric, You will find attached (removed from ml, so bcc to you): seems like attachments are not stripped via ML, both emails here have the patch attached. I have to define the OneToMany on the concrete class (the JPAMessage). Yes, that's what I tried then. If this is done without changing JPAHeader/JPAProperty it should be ok, right? I spent hours trying to solve this without success. I can see that from the time you sent this mail... oh boy, that's late/early. One more point: for the insert to work, you need to add property name=openjpa.InverseManager value=true/ in persistence.xml (not in the patch) Is this still standard JPA? Just asking because if so, why would we need an extra option? Currently, the patch still gives issues when reading mails (null pointer on the properties list). Apparently, although the tables are created they are not used. Weird. Even if we solve these last points, we will remain with another issue... Indeed, in the sample, the type of the MayToOne needs to be a concrete class, otherwise openjpa complains. This means that you would be obliged in the Property to link to JPAMessage or JPAStreamingMessage. You can't link to AbstractJPAMessage. JPA 2.0 defines a way to define the association only in the parent (*Message). Then that would be ok. Although it's still strange that it works in your tests but not in James (also not for me). About the pk creation, I will reply a bit later. I reconsidered this issue and I think we should switch to Strategy.IDENTITY = autoincrement because that would allow other systems and implementations to work with this database without messing with the sequence table. Moreover, it's faster. Thanks for the files. Will check them out tonight. Best, Tim - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: [IMAP] Cut a milestone ?
Hey Eric, For example, with Derby, the schema is create with GENERATED ALWAYS columns and not GENERATED BY DEFAULT. Why's that? Wouldn't GENERATED BY DEFAULT solve this problem so we could happily use autoincrements? We could also define a key-generation-table per table, but I find this overkill. I agree, that's too much. I would tend to think to leave it like it is. If you consider the issue above really serious, than lets keep it. On the other hand: nobody would use derby in production, right? Do similar problems exist with other databases? For the direct @OneToMany, we have issues when implemented in James even if this samples show that it should be correct. Strange... On my side something in my dummy project is just odd. Can't get this running at all. JPA seems to be a bit delicate? You said JPA 2.0 defines a way to define the association only in the parent (*Message): Can you send me a working sample? As said: I can't really get it working here. I've seen it on this site: http://en.wikibooks.org/wiki/Java_Persistence/OneToMany#Example_of_a_JPA_2.0_unidirectional_OneToMany_relationship_database The @JoinColumn is put on the other side using referencedColumnName. One more point is the sql generated. The logs show jpql, and not sql. Do you know if it's possible to view the sql in the logs (I didn't find a way). No, idea, sorry. I'm happy with jpql :) If not possible, can you log on the database (mysql I think you use)? MySQL provides logs for all operations, yes. I noticed a small issue with your patch: You have to use message_id instead of id for the name of the @JoinColumn. Lets ask on the OpenJPA ML how to make OpenJPA use the relationship. Maybe they can even have a look at our concrete issue with James. Will you ask? Should I? Best, Tim - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: [IMAP] Cut a milestone ?
Hi Tim, I implemented the standard @OneToMany (not the proprietary extension) without intermediary table and the schema is created. As you described, it hanged the first, but recompiling and relaunching made it happen. I'm working now on an issue when inserting mail (fk is not set). I think a few unit tests in the apache-james-imap-jpa will help. On my side property name=penjpa.Log value=SQL=TRACE/ shows the SQL (in fact the JPQL). Recommendations found on http://openjpa.apache.org/faq.html allow the fk to be created. I finally think the current default pk strategy is the good one. There is still the decision to take for the MailMembership. Finally, I don't think we should take the coherence between all stores as an argument. For example, storing on file system is completely different than storing in DB, than in JCR, than in NoSQL,... The only reason why we should keep it may be the memory economy when listing mails for databases that does not support the streaming. Tks, Eric On 06/11/2010 08:59 AM, Norman Maurer wrote: Hi Tim, comments inside.. 2010/6/10 Tim-Christian Mundtd...@tim-erwin.de: Hi, Am Donnerstag, den 10.06.2010, 19:56 +0200 schrieb Norman Maurer: After thinking a bit more about it, could it be that the two tables are used for mapping ? I think that would make sense because we have a many relation here.. So one for mapping Message and Properties and one for Message and Header. That's true. However, we don't have Many-to-many mappings here, but one to many (as defined in AbstractJPAMessage). So these tables are basically superfluous and causing (little) performance loss. The reason they are created by OpenJPA is that the mappings are unidirectional. You are right.. One-to-Many should not need the join table. Although we don't need a message field in Headers and Properties I think we should introduce it and thus allow for One-To-Many without mapping table. I already tried doing that, adding @JoinColum and mappedBy according to the specification: http://java.sun.com/javaee/5/docs/api/javax/persistence/OneToMany.html However, when mappedBy=essage is added to the AbstractJPAMessage then OpenJPA hangs without any error output (despiteproperty name=penjpa.Log value=SQL=TRACE/) and the tables won't get created. If put into JPAMessage (overriding the field) the tables get created as usual simply with the addition of a join column (in Header). I guess it has to do with the inheritance. Any idea? Otherwise I'd ask on some OpenJPA mailing list. Askin on the OpenJPA mailing list would be a good idea.. Besides that there is a proprietary solution for OpenJPA: @OneToMany(cascade =ascadeType.ALL, fetchþtchType.LAZY) @ElementJoinColumn(name=essage_id, referencedColumnName=id) private ListJPAHeader headers; This fixes the issue but is non-standard. Other framworks like jpox also have such options if I don't err. If nothing helps we could move the mapping stuff to the OpenJPA* classes, but I think it should be possible with using plain JPA. Tim Bye, Norman - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: [IMAP] Cut a milestone ?
Eric, I implemented the standard @OneToMany (not the proprietary extension) without intermediary table and the schema is created. As you described, it hanged the first, but recompiling and relaunching made it happen. could you please send me a patch for this? I always recompiled and relaunched but I've no idea what I did wrong. On my side property name=penjpa.Log value=SQL=TRACE/ shows the SQL (in fact the JPQL). It does here, too, however, in the situation when it hung creating the tables it didn't. Recommendations found on http://openjpa.apache.org/faq.html allow the fk to be created. I finally think the current default pk strategy is the good one. Foreign keys on a database level are certainly a good idea. For PKs I prefer auto-increments instead of sequence tables for the sake of simplicity and performance. Cheers Tim - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: [IMAP] Cut a milestone ?
Hi Tim, comments inside.. 2010/6/10 Tim-Christian Mundt d...@tim-erwin.de: Hi, Am Donnerstag, den 10.06.2010, 19:56 +0200 schrieb Norman Maurer: After thinking a bit more about it, could it be that the two tables are used for mapping ? I think that would make sense because we have a many relation here.. So one for mapping Message and Properties and one for Message and Header. That's true. However, we don't have Many-to-many mappings here, but one to many (as defined in AbstractJPAMessage). So these tables are basically superfluous and causing (little) performance loss. The reason they are created by OpenJPA is that the mappings are unidirectional. You are right.. One-to-Many should not need the join table. Although we don't need a message field in Headers and Properties I think we should introduce it and thus allow for One-To-Many without mapping table. I already tried doing that, adding @JoinColum and mappedBy according to the specification: http://java.sun.com/javaee/5/docs/api/javax/persistence/OneToMany.html However, when mappedBy=message is added to the AbstractJPAMessage then OpenJPA hangs without any error output (despite property name=openjpa.Log value=SQL=TRACE/) and the tables won't get created. If put into JPAMessage (overriding the field) the tables get created as usual simply with the addition of a join column (in Header). I guess it has to do with the inheritance. Any idea? Otherwise I'd ask on some OpenJPA mailing list. Askin on the OpenJPA mailing list would be a good idea.. Besides that there is a proprietary solution for OpenJPA: �...@onetomany(cascade = CascadeType.ALL, fetch=FetchType.LAZY) �...@elementjoincolumn(name=message_id, referencedColumnName=id) private ListJPAHeader headers; This fixes the issue but is non-standard. Other framworks like jpox also have such options if I don't err. If nothing helps we could move the mapping stuff to the OpenJPA* classes, but I think it should be possible with using plain JPA. Tim Bye, Norman - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: [IMAP] Cut a milestone ?
Hi Eric, comments inside... 2010/6/9 Eric Charles eric.char...@u-mangate.com: Hi, A stable database schema would be great. Upgrading jars is straightforward, but changing the db schema (and potentially the datas) is always a pain. I had a look at the generated tables (see the list after the text). All have primary key and unique constraints, but there are no foreign keys (no constraints integrity). The buildSchema(ForeignKeys=true) does not seem to create something for the @ManyToOne. Or maybe is this specific to derby (I'm using), and other databases such as mysql,... behave differently? I need to check, I'm currently using JCR so no idea atm.. I also didn't find an entity for MESSAGE_PROPERTY nor MESSAGE_HEADER. However the table exists. Maybe somthing from the good old days ? does they get created on a fresh install ? As far I can understand, the primary keys are all generated via the default OPENJPA_SEQUENCE_TABLE. I often let each the table auto_generate the primary keys (strategy=GenerationType.IDENTITY). Are there any plans for this in the future, or will we remain with the unique default OPENJPA_SEQUENCE_TABLE? I have no strong opinion here.. Anyone ? About MailboxMembership, Tim, you said the whole message is not loaded tks to streaming. Right now, openjpa.streaming=false in database.properties but when I look JPAMessage implementation, it seems that the content is loaded via InputStream. So what is the current state? Should the openjpa.streaming property be removed? Let me try to explain it.. When you have a look at JPAMessage you see it use a byte[] object to store the message content. So once you need to read the content, the whole content get loaded in memory. Thats the default and work for every db. If you use openjpa.streaming=true it will stream the content direct from the db. So it never need to load the content into the memory at all. This only works for a few databases (derby is not one of them). Thats why it is false by default. Sorry for my in-vrac comments. Each of them may worth a separate thread. Tks, Eric MEMBERSHIP JAMESUSER MAILBOX HEADER MESSAGE MESSAGE_HEADER MESSAGE_PROPERTY DOMAIN SUBSCRIPTION PROPERTY BAYESIANANALYSIS_SPAM VIRTUALUSERTABLE BAYESIANANALYSIS_MESSAGECOUNTS BAYESIANANALYSIS_HAM OPENJPA_SEQUENCE_TABLE On 06/09/2010 10:18 AM, Norman Maurer wrote: Right, I think we should at least be sure to not change the layout anymore. I don't have a strong opinion on the double storage vs. single storage. If we don't need the single storage we could just merge Document and MailboxMembership interface. Bye, Norman 2010/6/9 Tim-Christian Mundtd...@tim-erwin.de: Hi! There is this one change of the database layout which should be decided upon before releasing, I think. Messing with code after the release is fine, but changing the database would be bad. I'm talking about Normans idea to unite the Message and Membership interfaces/classes = tables. The original idea of separating those was not to pass the whole mime message around when just the flags are needed. Moreover, the Membership part would serve as a reference to the Message so that a copy would just mean another reference to the same mail. The question: does the increased complexity really pay off? I think saving space is not really an argument because it's not really common to have duplicates (at least in my experience). And loading the whole message would not mean the contents but only a stream handler, not a memory killer, right? Regards Tim Eric Charles: Running here without any problem jpa (embedded derby) + jdbc domainlist + spamassassin + forwarding mailet. When do you think to release? Tks, Eric On 06/07/2010 04:51 PM, Norman Maurer wrote: Thx mate.. I deployed fresh trunk and everything seems to work so far without problems :) Bye, Norman Ps: I'm using JCR Mailbox 2010/6/7 Eric Charleseric.char...@u-mangate.com: Hi, I will deploy a fresh trunk this night just to make sure everything is still ok (cfr last commits in protocol,..). The last snapshot I took is 2 weeks-old and is really stable. Tks, Eric On 06/07/2010 04:40 PM, Norman Maurer wrote: Hi all, I think all the stuff in the imap library is now very usable. I think we should cut a milestone and then cut one of james server. Anything you guys want to get refactored before ? Bye, Norman Ps: Even if we discover a bug later we can cut just another one.. Release often, Release early (urgh...) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: [IMAP] Cut a milestone ?
Hi Norman, - I will also try to see what happens with the foreign keys. - MESSAGE_PROPERTY and MESSAGE_HEADER are created today during a fresh install. - Yes, input from community is welcome for the primary key generation strategy. Anyone? - I now better understand JPAMessage streaming. Many tks for the explanation. Will also look for derby (maybe with latest 1.6). Back to MailboxMembership, I was happy to see uid is part of the MailboxMembership. It's good that a mail copy get a new UID. As Tim, I don't often copy mails: the space argument should not be retained. The memory argument is interesting for database that does not support streaming. I'm just wondering when the mail is completely loaded: during mail list when a client consults a server or only during attachment download? We should also stick to a common strategy for all stores (jcr, jpa,...). Is JCR also working with a temporary structure linking Mailbox and Mail? Tks, Eric On 06/10/2010 11:09 AM, Norman Maurer wrote: Hi Eric, comments inside... 2010/6/9 Eric Charleseric.char...@u-mangate.com: Hi, A stable database schema would be great. Upgrading jars is straightforward, but changing the db schema (and potentially the datas) is always a pain. I had a look at the generated tables (see the list after the text). All have primary key and unique constraints, but there are no foreign keys (no constraints integrity). The buildSchema(ForeignKeys=ue) does not seem to create something for the @ManyToOne. Or maybe is this specific to derby (I'm using), and other databases such as mysql,... behave differently? I need to check, I'm currently using JCR so no idea atm.. I also didn't find an entity for MESSAGE_PROPERTY nor MESSAGE_HEADER. However the table exists. Maybe somthing from the good old days ? does they get created on a fresh install ? As far I can understand, the primary keys are all generated via the default OPENJPA_SEQUENCE_TABLE. I often let each the table auto_generate the primary keys (strategy=nerationType.IDENTITY). Are there any plans for this in the future, or will we remain with the unique default OPENJPA_SEQUENCE_TABLE? I have no strong opinion here.. Anyone ? About MailboxMembership, Tim, you said the whole message is not loaded tks to streaming. Right now, openjpa.streamingúlse in database.properties but when I look JPAMessage implementation, it seems that the content is loaded via InputStream. So what is the current state? Should the openjpa.streaming property be removed? Let me try to explain it.. When you have a look at JPAMessage you see it use a byte[] object to store the message content. So once you need to read the content, the whole content get loaded in memory. Thats the default and work for every db. If you use openjpa.streaming=ue it will stream the content direct from the db. So it never need to load the content into the memory at all. This only works for a few databases (derby is not one of them). Thats why it is false by default. Sorry for my in-vrac comments. Each of them may worth a separate thread. Tks, Eric MEMBERSHIP JAMESUSER MAILBOX HEADER MESSAGE MESSAGE_HEADER MESSAGE_PROPERTY DOMAIN SUBSCRIPTION PROPERTY BAYESIANANALYSIS_SPAM VIRTUALUSERTABLE BAYESIANANALYSIS_MESSAGECOUNTS BAYESIANANALYSIS_HAM OPENJPA_SEQUENCE_TABLE On 06/09/2010 10:18 AM, Norman Maurer wrote: Right, I think we should at least be sure to not change the layout anymore. I don't have a strong opinion on the double storage vs. single storage. If we don't need the single storage we could just merge Document and MailboxMembership interface. Bye, Norman 2010/6/9 Tim-Christian Mundtd...@tim-erwin.de: Hi! There is this one change of the database layout which should be decided upon before releasing, I think. Messing with code after the release is fine, but changing the database would be bad. I'm talking about Normans idea to unite the Message and Membership interfaces/classes = tables. The original idea of separating those was not to pass the whole mime message around when just the flags are needed. Moreover, the Membership part would serve as a reference to the Message so that a copy would just mean another reference to the same mail. The question: does the increased complexity really pay off? I think saving space is not really an argument because it's not really common to have duplicates (at least in my experience). And loading the whole message would not mean the contents but only a stream handler, not a memory killer, right? Regards Tim Eric Charles: Running here without any problem jpa (embedded derby) + jdbc domainlist + spamassassin + forwarding mailet. When do you think to release? Tks, Eric On 06/07/2010 04:51 PM, Norman Maurer wrote: Thx mate.. I deployed fresh trunk and everything seems to work so far without problems :) Bye, Norman Ps: I'm using JCR Mailbox 2010/6/7 Eric
Re: [IMAP] Cut a milestone ?
After thinking a bit more about it, could it be that the two tables are used for mapping ? I think that would make sense because we have a many relation here.. So one for mapping Message and Properties and one for Message and Header. Bye, Norman 2010/6/10 Eric Charles eric.char...@u-mangate.com: Hi Norman, - I will also try to see what happens with the foreign keys. - MESSAGE_PROPERTY and MESSAGE_HEADER are created today during a fresh install. - Yes, input from community is welcome for the primary key generation strategy. Anyone? - I now better understand JPAMessage streaming. Many tks for the explanation. Will also look for derby (maybe with latest 1.6). Back to MailboxMembership, I was happy to see uid is part of the MailboxMembership. It's good that a mail copy get a new UID. As Tim, I don't often copy mails: the space argument should not be retained. The memory argument is interesting for database that does not support streaming. I'm just wondering when the mail is completely loaded: during mail list when a client consults a server or only during attachment download? We should also stick to a common strategy for all stores (jcr, jpa,...). Is JCR also working with a temporary structure linking Mailbox and Mail? Tks, Eric On 06/10/2010 11:09 AM, Norman Maurer wrote: Hi Eric, comments inside... 2010/6/9 Eric Charleseric.char...@u-mangate.com: Hi, A stable database schema would be great. Upgrading jars is straightforward, but changing the db schema (and potentially the datas) is always a pain. I had a look at the generated tables (see the list after the text). All have primary key and unique constraints, but there are no foreign keys (no constraints integrity). The buildSchema(ForeignKeys=ue) does not seem to create something for the @ManyToOne. Or maybe is this specific to derby (I'm using), and other databases such as mysql,... behave differently? I need to check, I'm currently using JCR so no idea atm.. I also didn't find an entity for MESSAGE_PROPERTY nor MESSAGE_HEADER. However the table exists. Maybe somthing from the good old days ? does they get created on a fresh install ? As far I can understand, the primary keys are all generated via the default OPENJPA_SEQUENCE_TABLE. I often let each the table auto_generate the primary keys (strategy=nerationType.IDENTITY). Are there any plans for this in the future, or will we remain with the unique default OPENJPA_SEQUENCE_TABLE? I have no strong opinion here.. Anyone ? About MailboxMembership, Tim, you said the whole message is not loaded tks to streaming. Right now, openjpa.streamingúlse in database.properties but when I look JPAMessage implementation, it seems that the content is loaded via InputStream. So what is the current state? Should the openjpa.streaming property be removed? Let me try to explain it.. When you have a look at JPAMessage you see it use a byte[] object to store the message content. So once you need to read the content, the whole content get loaded in memory. Thats the default and work for every db. If you use openjpa.streaming=ue it will stream the content direct from the db. So it never need to load the content into the memory at all. This only works for a few databases (derby is not one of them). Thats why it is false by default. Sorry for my in-vrac comments. Each of them may worth a separate thread. Tks, Eric MEMBERSHIP JAMESUSER MAILBOX HEADER MESSAGE MESSAGE_HEADER MESSAGE_PROPERTY DOMAIN SUBSCRIPTION PROPERTY BAYESIANANALYSIS_SPAM VIRTUALUSERTABLE BAYESIANANALYSIS_MESSAGECOUNTS BAYESIANANALYSIS_HAM OPENJPA_SEQUENCE_TABLE On 06/09/2010 10:18 AM, Norman Maurer wrote: Right, I think we should at least be sure to not change the layout anymore. I don't have a strong opinion on the double storage vs. single storage. If we don't need the single storage we could just merge Document and MailboxMembership interface. Bye, Norman 2010/6/9 Tim-Christian Mundtd...@tim-erwin.de: Hi! There is this one change of the database layout which should be decided upon before releasing, I think. Messing with code after the release is fine, but changing the database would be bad. I'm talking about Normans idea to unite the Message and Membership interfaces/classes = tables. The original idea of separating those was not to pass the whole mime message around when just the flags are needed. Moreover, the Membership part would serve as a reference to the Message so that a copy would just mean another reference to the same mail. The question: does the increased complexity really pay off? I think saving space is not really an argument because it's not really common to have duplicates (at least in my experience). And loading the whole message would not mean the contents but only a stream handler, not a memory killer, right? Regards Tim Eric Charles: Running here without any problem
Re: [IMAP] Cut a milestone ?
Hi, Am Donnerstag, den 10.06.2010, 19:56 +0200 schrieb Norman Maurer: After thinking a bit more about it, could it be that the two tables are used for mapping ? I think that would make sense because we have a many relation here.. So one for mapping Message and Properties and one for Message and Header. That's true. However, we don't have Many-to-many mappings here, but one to many (as defined in AbstractJPAMessage). So these tables are basically superfluous and causing (little) performance loss. The reason they are created by OpenJPA is that the mappings are unidirectional. Although we don't need a message field in Headers and Properties I think we should introduce it and thus allow for One-To-Many without mapping table. I already tried doing that, adding @JoinColum and mappedBy according to the specification: http://java.sun.com/javaee/5/docs/api/javax/persistence/OneToMany.html However, when mappedBy=message is added to the AbstractJPAMessage then OpenJPA hangs without any error output (despite property name=openjpa.Log value=SQL=TRACE/) and the tables won't get created. If put into JPAMessage (overriding the field) the tables get created as usual simply with the addition of a join column (in Header). I guess it has to do with the inheritance. Any idea? Otherwise I'd ask on some OpenJPA mailing list. Besides that there is a proprietary solution for OpenJPA: @OneToMany(cascade = CascadeType.ALL, fetch=FetchType.LAZY) @ElementJoinColumn(name=message_id, referencedColumnName=id) private ListJPAHeader headers; This fixes the issue but is non-standard. Other framworks like jpox also have such options if I don't err. Tim - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: [IMAP] Cut a milestone ?
Hi! There is this one change of the database layout which should be decided upon before releasing, I think. Messing with code after the release is fine, but changing the database would be bad. I'm talking about Normans idea to unite the Message and Membership interfaces/classes = tables. The original idea of separating those was not to pass the whole mime message around when just the flags are needed. Moreover, the Membership part would serve as a reference to the Message so that a copy would just mean another reference to the same mail. The question: does the increased complexity really pay off? I think saving space is not really an argument because it's not really common to have duplicates (at least in my experience). And loading the whole message would not mean the contents but only a stream handler, not a memory killer, right? Regards Tim Eric Charles: Running here without any problem jpa (embedded derby) + jdbc domainlist + spamassassin + forwarding mailet. When do you think to release? Tks, Eric On 06/07/2010 04:51 PM, Norman Maurer wrote: Thx mate.. I deployed fresh trunk and everything seems to work so far without problems :) Bye, Norman Ps: I'm using JCR Mailbox 2010/6/7 Eric Charleseric.char...@u-mangate.com: Hi, I will deploy a fresh trunk this night just to make sure everything is still ok (cfr last commits in protocol,..). The last snapshot I took is 2 weeks-old and is really stable. Tks, Eric On 06/07/2010 04:40 PM, Norman Maurer wrote: Hi all, I think all the stuff in the imap library is now very usable. I think we should cut a milestone and then cut one of james server. Anything you guys want to get refactored before ? Bye, Norman Ps: Even if we discover a bug later we can cut just another one.. Release often, Release early (urgh...) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: [IMAP] Cut a milestone ?
Right, I think we should at least be sure to not change the layout anymore. I don't have a strong opinion on the double storage vs. single storage. If we don't need the single storage we could just merge Document and MailboxMembership interface. Bye, Norman 2010/6/9 Tim-Christian Mundt d...@tim-erwin.de: Hi! There is this one change of the database layout which should be decided upon before releasing, I think. Messing with code after the release is fine, but changing the database would be bad. I'm talking about Normans idea to unite the Message and Membership interfaces/classes = tables. The original idea of separating those was not to pass the whole mime message around when just the flags are needed. Moreover, the Membership part would serve as a reference to the Message so that a copy would just mean another reference to the same mail. The question: does the increased complexity really pay off? I think saving space is not really an argument because it's not really common to have duplicates (at least in my experience). And loading the whole message would not mean the contents but only a stream handler, not a memory killer, right? Regards Tim Eric Charles: Running here without any problem jpa (embedded derby) + jdbc domainlist + spamassassin + forwarding mailet. When do you think to release? Tks, Eric On 06/07/2010 04:51 PM, Norman Maurer wrote: Thx mate.. I deployed fresh trunk and everything seems to work so far without problems :) Bye, Norman Ps: I'm using JCR Mailbox 2010/6/7 Eric Charleseric.char...@u-mangate.com: Hi, I will deploy a fresh trunk this night just to make sure everything is still ok (cfr last commits in protocol,..). The last snapshot I took is 2 weeks-old and is really stable. Tks, Eric On 06/07/2010 04:40 PM, Norman Maurer wrote: Hi all, I think all the stuff in the imap library is now very usable. I think we should cut a milestone and then cut one of james server. Anything you guys want to get refactored before ? Bye, Norman Ps: Even if we discover a bug later we can cut just another one.. Release often, Release early (urgh...) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: [IMAP] Cut a milestone ?
Running here without any problem jpa (embedded derby) + jdbc domainlist + spamassassin + forwarding mailet. When do you think to release? Tks, Eric On 06/07/2010 04:51 PM, Norman Maurer wrote: Thx mate.. I deployed fresh trunk and everything seems to work so far without problems :) Bye, Norman Ps: I'm using JCR Mailbox 2010/6/7 Eric Charleseric.char...@u-mangate.com: Hi, I will deploy a fresh trunk this night just to make sure everything is still ok (cfr last commits in protocol,..). The last snapshot I took is 2 weeks-old and is really stable. Tks, Eric On 06/07/2010 04:40 PM, Norman Maurer wrote: Hi all, I think all the stuff in the imap library is now very usable. I think we should cut a milestone and then cut one of james server. Anything you guys want to get refactored before ? Bye, Norman Ps: Even if we discover a bug later we can cut just another one.. Release often, Release early (urgh...) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[IMAP] Cut a milestone ?
Hi all, I think all the stuff in the imap library is now very usable. I think we should cut a milestone and then cut one of james server. Anything you guys want to get refactored before ? Bye, Norman Ps: Even if we discover a bug later we can cut just another one.. Release often, Release early (urgh...) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: [IMAP] Cut a milestone ?
Hi, I will deploy a fresh trunk this night just to make sure everything is still ok (cfr last commits in protocol,..). The last snapshot I took is 2 weeks-old and is really stable. Tks, Eric On 06/07/2010 04:40 PM, Norman Maurer wrote: Hi all, I think all the stuff in the imap library is now very usable. I think we should cut a milestone and then cut one of james server. Anything you guys want to get refactored before ? Bye, Norman Ps: Even if we discover a bug later we can cut just another one.. Release often, Release early (urgh...) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: [IMAP] Cut a milestone ?
Thx mate.. I deployed fresh trunk and everything seems to work so far without problems :) Bye, Norman Ps: I'm using JCR Mailbox 2010/6/7 Eric Charles eric.char...@u-mangate.com: Hi, I will deploy a fresh trunk this night just to make sure everything is still ok (cfr last commits in protocol,..). The last snapshot I took is 2 weeks-old and is really stable. Tks, Eric On 06/07/2010 04:40 PM, Norman Maurer wrote: Hi all, I think all the stuff in the imap library is now very usable. I think we should cut a milestone and then cut one of james server. Anything you guys want to get refactored before ? Bye, Norman Ps: Even if we discover a bug later we can cut just another one.. Release often, Release early (urgh...) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[IMAP] Cut a milestone ?
Hi all, I would like to cut a milestone of the current imap-trunk once the james-project votes pass.. Anything which you would like to get fixed before a milestone release ? Bye, Norman - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: [IMAP] Cut a milestone ?
Good luck with your exams.. Please cast your vote for james-project before concentrate on your exams please ;) I will take care of the milestone then.. Bye, Norman Am 12.01.2010 um 18:59 schrieb Robert Burrell Donkin: (this is probably the last you'll hear from me until my exams are done) On Tue, Jan 12, 2010 at 2:14 PM, Norman Maurer nor...@apache.org wrote: Hi all, I would like to cut a milestone of the current imap-trunk once the james-project votes pass.. Anything which you would like to get fixed before a milestone release ? release early, release often :-) IMAP isn't ready for a full release but it'd be great to cut a milestone every month or two - robert - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
