Re: [basedb-devel] JobAgents on multi-processor machine

[Pawel Sztromwasser]

Thank you very much.
I thought about altering database schema somehow to set the constraints 
'initially deferred' and check if that works (in small examples it 
worked fine), but if a new version makes no trouble it doesn't make 
sense any more. Thanks once again for your help.

Pawel Sztromwasser


Nicklas Nordborg wrote:
> Nicklas Nordborg wrote:
>   
>> I am taking this back to the mailing list in the hope that there is some 
>> Postgres expert that can shed some light on this.
>>
>> I found this http://www.thescripts.com/forum/thread174537.html
>>
>> It is about a rather old Postgres version (7.4). I am not sure if the 
>> same problem is still around. The problem is that when inserting data 
>> Postgres is locking all rows that are referenced by foreign keys. This 
>> means that if your five imports, for example, inserts data with 
>> references to the same reporter or feature on an array design, only one 
>> can proceed at a time.
>>
>> Actually, if this problem is still around in Postgres 8.2 I think this 
>> makes Postgres more or less unusable in a true multi-user setup. Since 
>> almost any data import or analysis plug-in will insert data and in many 
>> cases the new data will reference a reporter, an experiment or some 
>> other common item there is a high probability that only one job is able 
>> to proceed at a time.
>>
>> There is a tip about a "DEFERRED" mode that can overcome some 
>> limitations, but I am not very familiar with Postgres and don't really 
>> know how to set it. It would be good if there is a global server 
>> configuration option for this.
>>
>> Do you have the possibility to investigate this further?
>> 
>
> I did some investigation on my own. I can verify that the locking 
> problem exists on Postgres 8.0.4. I tested this with only two jobs using 
> the internal job queue and only one could work at a time.
>
> After installing Postgres 8.2.5 (which is the latest version) the 
> locking problem disappeared. Checking the documentation on their website 
> it seems like a new shared row-level lock mode has been introduced some 
> time ago. This means that Postgres no longer has to use a write-lock 
> which can only be held by one transaction at a time.
>
> So, unless some new information about this issue comes up I will not 
> investigate this any more. Postgres users are encouraged to upgrade to 
> Postgres 8.2.5.
>
> /Nicklas
>
> -
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> ___
> basedb-devel mailing list
> basedb-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/basedb-devel
>   


-
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel

Re: [basedb-devel] JobAgents on multi-processor machine

[Pawel Sztromwasser]

We need to wait for the postgres upgrade, so a temporary solution had to 
be made. All the foreign key constraints were dropped and added again 
with INITIALLY DEFERRED flag. Everything seems to be working fine now.
If anyone is interested I can provide files with sql queries I used.

Pawel

Pawel Sztromwasser wrote:
> Thank you very much.
> I thought about altering database schema somehow to set the constraints 
> 'initially deferred' and check if that works (in small examples it 
> worked fine), but if a new version makes no trouble it doesn't make 
> sense any more. Thanks once again for your help.
>
> Pawel Sztromwasser
>
>
> Nicklas Nordborg wrote:
>   
>> Nicklas Nordborg wrote:
>>   
>> 
>>> I am taking this back to the mailing list in the hope that there is some 
>>> Postgres expert that can shed some light on this.
>>>
>>> I found this http://www.thescripts.com/forum/thread174537.html
>>>
>>> It is about a rather old Postgres version (7.4). I am not sure if the 
>>> same problem is still around. The problem is that when inserting data 
>>> Postgres is locking all rows that are referenced by foreign keys. This 
>>> means that if your five imports, for example, inserts data with 
>>> references to the same reporter or feature on an array design, only one 
>>> can proceed at a time.
>>>
>>> Actually, if this problem is still around in Postgres 8.2 I think this 
>>> makes Postgres more or less unusable in a true multi-user setup. Since 
>>> almost any data import or analysis plug-in will insert data and in many 
>>> cases the new data will reference a reporter, an experiment or some 
>>> other common item there is a high probability that only one job is able 
>>> to proceed at a time.
>>>
>>> There is a tip about a "DEFERRED" mode that can overcome some 
>>> limitations, but I am not very familiar with Postgres and don't really 
>>> know how to set it. It would be good if there is a global server 
>>> configuration option for this.
>>>
>>> Do you have the possibility to investigate this further?
>>> 
>>>   
>> I did some investigation on my own. I can verify that the locking 
>> problem exists on Postgres 8.0.4. I tested this with only two jobs using 
>> the internal job queue and only one could work at a time.
>>
>> After installing Postgres 8.2.5 (which is the latest version) the 
>> locking problem disappeared. Checking the documentation on their website 
>> it seems like a new shared row-level lock mode has been introduced some 
>> time ago. This means that Postgres no longer has to use a write-lock 
>> which can only be held by one transaction at a time.
>>
>> So, unless some new information about this issue comes up I will not 
>> investigate this any more. Postgres users are encouraged to upgrade to 
>> Postgres 8.2.5.
>>
>> /Nicklas
>>
>> -
>> This SF.net email is sponsored by: Splunk Inc.
>> Still grepping through log files to find problems?  Stop.
>> Now Search log events and configuration files using AJAX and a browser.
>> Download your FREE copy of Splunk now >> http://get.splunk.com/
>> ___
>> basedb-devel mailing list
>> basedb-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/basedb-devel
>>   
>> 
>
>
> -
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> ___
> basedb-devel mailing list
> basedb-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/basedb-devel
>   


-
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel

[basedb-devel] Index dropping fails with Postgres 7.3.19

[Pawel Sztromwasser]

Hi,

I got an exception while upgrading from Base 2.4.3 to 2.4.5. It was 
during dropping indexes using
"[basedir]/bin/dynamicdb.sh -v -dropindexes" command. The exceptions 
message follows:

// some successful droppings

=
Table   : D21886Filter
  Unique   : D21886Filter_pkey [cube, filter, column, position]
Safe drop  : false
DROP-SQL   : ALTER TABLE "dynamic"."D21886Filter" DROP CONSTRAINT 
"D21886Filter_pkey"
Actions:
  Index: D21886Filtercolumn [column]
Safe drop  : true
DROP-SQL   : DROP INDEX "D21886Filtercolumn"
Actions: DROP
13:22:05,724 ERROR core:1836 - Exception
org.postgresql.util.PSQLException: ERROR:  index "D21886Filtercolumn" 
does not exist
at 
org.postgresql.core.v2.QueryExecutorImpl.receiveErrorMessage(QueryExecutorImpl.java:557)
at 
org.postgresql.core.v2.QueryExecutorImpl.processResults(QueryExecutorImpl.java:480)
at 
org.postgresql.core.v2.QueryExecutorImpl.execute(QueryExecutorImpl.java:364)
at 
org.postgresql.core.v2.QueryExecutorImpl.execute(QueryExecutorImpl.java:258)
at 
org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:452)
at 
org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:340)
at 
org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:286)
at 
com.mchange.v2.c3p0.impl.NewProxyStatement.executeUpdate(NewProxyStatement.java:64)
at 
net.sf.basedb.core.HibernateUtil.dynamicDbIndexes(HibernateUtil.java:1828)
at net.sf.basedb.install.InitDB.main(InitDB.java:105)


net.sf.basedb.core.BaseException: ERROR:  index "D21886Filtercolumn" 
does not exist
at 
net.sf.basedb.core.HibernateUtil.dynamicDbIndexes(HibernateUtil.java:1837)
at net.sf.basedb.install.InitDB.main(InitDB.java:105)
Caused by: org.postgresql.util.PSQLException: ERROR:  index 
"D21886Filtercolumn" does not exist
at 
org.postgresql.core.v2.QueryExecutorImpl.receiveErrorMessage(QueryExecutorImpl.java:557)
at 
org.postgresql.core.v2.QueryExecutorImpl.processResults(QueryExecutorImpl.java:480)
at 
org.postgresql.core.v2.QueryExecutorImpl.execute(QueryExecutorImpl.java:364)
at 
org.postgresql.core.v2.QueryExecutorImpl.execute(QueryExecutorImpl.java:258)
at 
org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:452)
at 
org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:340)
at 
org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:286)
at 
com.mchange.v2.c3p0.impl.NewProxyStatement.executeUpdate(NewProxyStatement.java:64)
at 
net.sf.basedb.core.HibernateUtil.dynamicDbIndexes(HibernateUtil.java:1828)
... 1 more

I investigated a bit and found out that this index in fact exists. I 
managed to drop it manually using psql with command:

DROP INDEX "dynamic"."D21886Filtercolumn"

Command without schema name prefix (DROP INDEX "D21886Filtercolumn") 
doesn't work. I checked the index dropping code and I found out that in 
net.sf.basedb.core.dbengine.PostgresDbEngine.getDropIndexSql() method 
code(line 120) creating 'drop index query' uses only index name without 
schema name prefix. That is why an exception is thrown.
I didn't find anything about adding dynamic schema to users search_path 
in PostgreSql in Base2 requirements. Some of the code in the same 
'problem causing' method adds schema prefix when dropping constraints, 
so I consider this issue as a bug.

We are looking now for some work-around, like postgres search-path 
setting and will post everything useful we find.

Pawel Sztromwasser

-
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel

Re: [basedb-devel] Index dropping fails with Postgres 7.3.19

[Pawel Sztromwasser]

I found this:

ALTER [base_user_name] SET search_path TO 
"$user",public,[base_dynamic_schema_name];

This command executed from psql PostgreSql client adds dynamic schema to 
the default search_path.
After this dynamicdb.sh -dropindexes completes with no problems.

Cheers,
Pawel


Pawel Sztromwasser wrote:
> Hi,
>
> I got an exception while upgrading from Base 2.4.3 to 2.4.5. It was 
> during dropping indexes using
> "[basedir]/bin/dynamicdb.sh -v -dropindexes" command. The exceptions 
> message follows:
>
> // some successful droppings
>
> =
> Table   : D21886Filter
>   Unique   : D21886Filter_pkey [cube, filter, column, position]
> Safe drop  : false
> DROP-SQL   : ALTER TABLE "dynamic"."D21886Filter" DROP CONSTRAINT 
> "D21886Filter_pkey"
> Actions:
>   Index: D21886Filtercolumn [column]
> Safe drop  : true
> DROP-SQL   : DROP INDEX "D21886Filtercolumn"
> Actions: DROP
> 13:22:05,724 ERROR core:1836 - Exception
> org.postgresql.util.PSQLException: ERROR:  index "D21886Filtercolumn" 
> does not exist
> at 
> org.postgresql.core.v2.QueryExecutorImpl.receiveErrorMessage(QueryExecutorImpl.java:557)
> at 
> org.postgresql.core.v2.QueryExecutorImpl.processResults(QueryExecutorImpl.java:480)
> at 
> org.postgresql.core.v2.QueryExecutorImpl.execute(QueryExecutorImpl.java:364)
> at 
> org.postgresql.core.v2.QueryExecutorImpl.execute(QueryExecutorImpl.java:258)
> at 
> org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:452)
> at 
> org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:340)
> at 
> org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:286)
> at 
> com.mchange.v2.c3p0.impl.NewProxyStatement.executeUpdate(NewProxyStatement.java:64)
> at 
> net.sf.basedb.core.HibernateUtil.dynamicDbIndexes(HibernateUtil.java:1828)
> at net.sf.basedb.install.InitDB.main(InitDB.java:105)
>
>
> net.sf.basedb.core.BaseException: ERROR:  index "D21886Filtercolumn" 
> does not exist
> at 
> net.sf.basedb.core.HibernateUtil.dynamicDbIndexes(HibernateUtil.java:1837)
> at net.sf.basedb.install.InitDB.main(InitDB.java:105)
> Caused by: org.postgresql.util.PSQLException: ERROR:  index 
> "D21886Filtercolumn" does not exist
> at 
> org.postgresql.core.v2.QueryExecutorImpl.receiveErrorMessage(QueryExecutorImpl.java:557)
> at 
> org.postgresql.core.v2.QueryExecutorImpl.processResults(QueryExecutorImpl.java:480)
> at 
> org.postgresql.core.v2.QueryExecutorImpl.execute(QueryExecutorImpl.java:364)
> at 
> org.postgresql.core.v2.QueryExecutorImpl.execute(QueryExecutorImpl.java:258)
> at 
> org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:452)
> at 
> org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:340)
> at 
> org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:286)
> at 
> com.mchange.v2.c3p0.impl.NewProxyStatement.executeUpdate(NewProxyStatement.java:64)
> at 
> net.sf.basedb.core.HibernateUtil.dynamicDbIndexes(HibernateUtil.java:1828)
> ... 1 more
>
> I investigated a bit and found out that this index in fact exists. I 
> managed to drop it manually using psql with command:
>
> DROP INDEX "dynamic"."D21886Filtercolumn"
>
> Command without schema name prefix (DROP INDEX "D21886Filtercolumn") 
> doesn't work. I checked the index dropping code and I found out that in 
> net.sf.basedb.core.dbengine.PostgresDbEngine.getDropIndexSql() method 
> code(line 120) creating 'drop index query' uses only index name without 
> schema name prefix. That is why an exception is thrown.
> I didn't find anything about adding dynamic schema to users search_path 
> in PostgreSql in Base2 requirements. Some of the code in the same 
> 'problem causing' method adds schema prefix when dropping constraints, 
> so I consider this issue as a bug.
>
> We are looking now for some work-around, like postgres search-path 
> setting and will post everything useful we find.
>
> Pawel Sztromwasser
>
> -
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> ht

[basedb-devel] isMIAMErequired in AnnotationType

[Pawel Sztromwasser]

Hi,

We are migrating Base1.2.17+ to 2.4.6. The + stands for our local 
changes made to satisfy the needs of MAGE-ML exporter. When writing code 
to move additional annotations I noticed that information whether 
annotation type is required for MIAME is kept in AnnotationType 
class/table. This results in annotation type being MIAME-required either 
for all of the item types it is enabled for, or for none of them. It 
would be very useful to have annotation type available for several types 
of items, for some as MIAME-required and for the others not.
Is it possible to change it ?

All the best,
Pawel


-- 
Pawe? Sztromwasser
Computational Biology Unit
Bergen Center for Computational Science

-
SF.Net email is sponsored by: The Future of Linux Business White Paper
from Novell.  From the desktop to the data center, Linux is going
mainstream.  Let it simplify your IT future.
http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel

[basedb-devel] DynamicResultIterator.hasNext method

[Pawel Sztromwasser]

Hi all,

I have been trying to optimize the performance of LowessNormalization 
plugin a little bit lately. It is extremely slow on our Postgres 
database when using small 'block groups'.

Anyway, I came across something that surprised me. When using 
DynamicResultIterator I noticed that hasNext() does not behave in the 
way I expect. After fetching a row by next() method, invoking hasNext() 
causes iterator to move to the next row. I checked that on the following 
example (it = iterator):

command| result
---
r = it.next() |  -
r.getInt(pos) | 324
it.hasNext()  | true
r.getInt(pos) | 325
---

I looked into the code to check what is going on. 
DynamicResultIterator.hasNext() invokes next() method on 
java.sql.ResultSet object to check if there are rows to read, but it 
does not move the cursor back(ResultSet.previous()). In addition, next() 
method uses hasNext() to move the cursor forward (otherwise executing 
if(it.hasNext) r=it.next() would skip one row).

Is this how it is meant to be? I realize that the usual way of using 
hasNext() is in the context like:

while(it.hasNext)
r=it.next()
// process r

where this issue is not a problem, but sometimes it is useful to check 
if there are next rows of data without actually moving the cursor. One 
could also benefit from previous() method in that DynamicResultIterator 
class.
There are probably reasons behind this kind of implementation I am not 
aware of. Then mentioning in the docs that hasNext moves the pointer to 
next result would be nice.

Best wishes,
Pawel

-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel

Re: [basedb-devel] DynamicResultIterator.hasNext method

[Pawel Sztromwasser]

Nicklas Nordborg wrote:
> Pawel Sztromwasser wrote:
>> Hi all,
>>
>> I have been trying to optimize the performance of LowessNormalization 
>> plugin a little bit lately. It is extremely slow on our Postgres 
>> database when using small 'block groups'.
> 
> It sounds like you are using an older version of BASE. The performance 
> of the Lowess plug-in for small block group sizes was greatly improved 
> in the 2.5.0 release. The tests showed decrease in execution time to 
> about 10-20% of the original. This was on arrays with 48 blocks and 
> block group size=1. Now the execution time is more or less the same for 
> all values of the block group size parameter. More information about 
> this at:
> 

Yes, we are currently using 2.4.5 and 2.5.0, but the tests were run on 
the older one. Upgrading it is a good idea:)
By the way, download links to .gz files on the web page don't work.

The improvement you are writing about is approximately what we achieved 
by decreasing number of db queries to 1 (no mater how big blocks are). I 
though about posting it with a code changes after some tests, but it's 
great that it has been done already.


>> Anyway, I came across something that surprised me. When using 
>> DynamicResultIterator I noticed that hasNext() does not behave in the 
>> way I expect. After fetching a row by next() method, invoking hasNext() 
>> causes iterator to move to the next row. I checked that on the following 
>> example (it = iterator):
> 
> The next() method always returns the same object which is just a "proxy" 
> that fetches data from the underlying ResultSet object.
> 
>> command| result
>> ---
>> r = it.next() |  -
>> r.getInt(pos) | 324
>> it.hasNext()  | true
>> r.getInt(pos) | 325
>> ---
>>
>> I looked into the code to check what is going on. 
>> DynamicResultIterator.hasNext() invokes next() method on 
>> java.sql.ResultSet object to check if there are rows to read, but it 
>> does not move the cursor back(ResultSet.previous()). In addition, next() 
>> method uses hasNext() to move the cursor forward (otherwise executing 
>> if(it.hasNext) r=it.next() would skip one row).
> 
> No rows are skipped. The hasNext() method only moves to the next row if 
> next() has been called in the meantime. It is safe to call hasNext() any 
> number of times. It will just move to the next row the first time.
> 
>> Is this how it is meant to be? 
> 
> Yes.
> 
>  > I realize that the usual way of using
>> hasNext() is in the context like:
>>
>>  while(it.hasNext)
>>  r=it.next()
>>  // process r
>>
>> where this issue is not a problem, but sometimes it is useful to check 
>> if there are next rows of data without actually moving the cursor. One 
>> could also benefit from previous() method in that DynamicResultIterator 
>> class.
>> There are probably reasons behind this kind of implementation I am not 
>> aware of. Then mentioning in the docs that hasNext moves the pointer to 
>> next result would be nice.
> 
> The only way to check if there are more rows in a ResultSet is to try to 
> move to the next row. For performance reasons all queries are executed 
> in read-only, forward-only mode so there is no possibility to move 
> backwards.
> 
> And since the DynamicResultIterator is an java.util.Iterator, we are 
> only interested in moving forwards anyway.
> 
> You are correct that the documentation could be clearer about the issue 
> that it is actually the hasNext() method that moves to the next row, not 
>   the next() method. The next() method at least states that the returned 
> SqlResult is just a pointer to the current row and if results from 
> previous rows are needed then that information needs to be copied by the 
> calling code. This is in itself also a performance optimization since we 
> never make unnecessary copies in the BASE core.
> 

Thanks for the explanation. It is just not what I expected by hasNext() 
method to do. I think also that other users expecting it to behave like 
java.util.Iterator.hasNext could be a bit surprised before reading docs:)


Best wishes,
Pawel

> /Nicklas
> 
> -
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
> ___
> basedb-devel mailing list
> basedb-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/basedb-devel


-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel

[basedb-devel] Jobagents view doesn't show jobs

[Pawel Sztromwasser]

Hello BASE2 developers,

After upgrade from BASE 2.4.6 to 2.6.0 I noticed that jobs currently run 
by jobagent aren't shown in jobagent's page (listing of all jobagents; 
jobs column) any more. All I can see is '-none-'. They appear when I 
move to particular jobagent's page.
I checked .jsp sources for these pages and it seems like list-agents.jsp 
uses deprecated method to get information about jobs (view-agent.jsp 
uses new method introduced in 2.6). I couldn't add a ticket so I am 
posting it on the mailing list.

Best wishes,
Pawel Sztromwasser

-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel

[basedb-devel] Migration transfers deleted items

[Pawel Sztromwasser]

Hi,

While testing migration I found out that items deleted in BASE1 are 
migrated to BASE2. As I understand from documentation, BASE1 doesn't 
delete items from the database unless purgeDatabase.php script is 
executed. They are just flagged as deleted by setting remove value=2 (or 
 > 1 maybe). Those that have removed flag set to 1 are marked for 
deletion (something equivalent to moving to trashcan). Other items have 
removed=0.

Migration code uses ResultSet.getBoolean() method to retrieve 'removed' 
value (which is smallint) and as I checked, it returns true only when 
removed==1. The consequence is that items marked for deletion go to 
trashcan as they suppose to, but those actually deleted get removed flag 
set to false and appear in BASE2 as not deleted.

This could be due to postgres JDBC driver handling type mapping in 
different way than mysql driver, but I couldn't find any information on 
that. The quick solution is to run purgeDatabase.php before migration to 
remove deleted items from the database.

All the best,
Pawel

-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel

Re: [basedb-devel] Migration transfers deleted items

[Pawel Sztromwasser]

We will be still testing migration for some time, because we haven't 
moved BASE1 to the new server yet. Some additional code was needed to 
move our local adaptations. They are supposed to be a part of 1.2.18 
BASE1 release (if happens), so if anyone ever needs to migrate from that 
version, we might have the code.

We are also working now on task of merging two (or more?) BASE1 
installations into one BASE2, using migration package. Do you know of 
any issues that could be problematic in this case? I managed to run one 
migration after another (from different BASE1s), after minor code 
changes, but still need to check if some items do not clash somehow.

Anyway, we are still working with migration so if anything serious pops 
up, I will let you know. Of course, if any of features I mentioned above 
is interesting for you, we will be glad to contribute to the project.

All the best and have a nice Easter,
Pawel

Nicklas Nordborg wrote:
> Pawel Sztromwasser wrote:
>> Hi,
>>
>> While testing migration I found out that items deleted in BASE1 are 
>> migrated to BASE2. As I understand from documentation, BASE1 doesn't 
>> delete items from the database unless purgeDatabase.php script is 
>> executed. They are just flagged as deleted by setting remove value=2 (or 
>>  > 1 maybe). Those that have removed flag set to 1 are marked for 
>> deletion (something equivalent to moving to trashcan). Other items have 
>> removed=0.
>>
>> Migration code uses ResultSet.getBoolean() method to retrieve 'removed' 
>> value (which is smallint) and as I checked, it returns true only when 
>> removed==1. The consequence is that items marked for deletion go to 
>> trashcan as they suppose to, but those actually deleted get removed flag 
>> set to false and appear in BASE2 as not deleted.
>>
>> This could be due to postgres JDBC driver handling type mapping in 
>> different way than mysql driver, but I couldn't find any information on 
>> that. The quick solution is to run purgeDatabase.php before migration to 
>> remove deleted items from the database.
> 
> I didn't know that. I simply assumed that the removed flag only could be 
> 0 or 1. Kind of like in BASE2. In any case, the 2.6 release was the 
> latest release of the migration program. It will still be included in 
> future releases, but we will not test or actively maintain it anymore. 
> The reason for this is that we are moving our servers to BASE 2, and at 
> some time in the future we will not have a BASE 1 server to use for 
> testing the migration.
> 
> Should any remaining critical bugs appear we will of course assist in 
> fixing it.
> 
> /Nicklas
> 
> -
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
> ___
> basedb-devel mailing list
> basedb-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/basedb-devel


-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel

[basedb-devel] Webservices transport error: 411 Error: Length Required

[Pawel Sztromwasser]

Hello BASE developers,

We have been recently trying to enable retrieving some data from BASE2 
installation directly into our 'home made' analysis tool. We use 
webservices for that and have to say that you are doing a great job. We 
are really looking forward to use them more extensively.

Recently, when I started testing the tool not only within the local 
network I noticed that the webservice client does not switch off 
chunking. It results in attached exception cascade. Chunking is on by 
default in axis2 and some servers (i.e. our gateway to BASE which 
apparently uses HTTP1.0 and not 1.1) have problems with it. The remedy 
is to use

options.setProperty(HTTPConstants.CHUNKED, false);

in net.sf.basedb.ws.client.AbstractRPCClient.getService(). I don't know 
what are the drawbacks of it, but that was the only way I could run the 
services in our current setup (BASE2.6 accessible by proxypass from 
exposed webserver). Could this option be set off by default or 
configurable somewhere? Or maybe there is some other workaround?

Cheers,
Pawel


org.apache.axis2.transport.http.HTTPSender sendViaPost
INFO: Unable to sendViaPost to 
url[http://api.bioinfo.no/base2/services/Session]
org.apache.axis2.AxisFault: Transport error: 411 Error: Length Required
at 
org.apache.axis2.transport.http.HTTPSender.handleResponse(HTTPSender.java:298)
at 
org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:192)
at org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:77)
at 
org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:327)
at 
org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:206)
at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:396)
at 
org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:374)
at 
org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:211)
at 
org.apache.axis2.client.OperationClient.execute(OperationClient.java:163)
at 
org.apache.axis2.client.ServiceClient.sendReceive(ServiceClient.java:528)
at 
org.apache.axis2.client.ServiceClient.sendReceive(ServiceClient.java:508)
at 
org.apache.axis2.rpc.client.RPCServiceClient.invokeBlocking(RPCServiceClient.java:101)
at 
net.sf.basedb.ws.client.AbstractRPCClient.invokeBlocking(AbstractRPCClient.java:62)
at 
net.sf.basedb.ws.client.AbstractRPCClient.invokeBlocking(AbstractRPCClient.java:68)
at net.sf.basedb.ws.client.SessionClient.(SessionClient.java:54)
at 
no.uib.cbu.basedb.wsclients.WSLogger.createNewSession(WSLogger.java:58)
at no.uib.cbu.basedb.wsclients.WSLogger.(WSLogger.java:15)
at no.uib.cbu.basedb.wsclients.RunningClass.main(RunningClass.java:32)

-
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel

Re: [basedb-devel] Webservices transport error: 411 Error: Length Required

[Pawel Sztromwasser]

Hi Nicklas,

I had some time and implemented the 'ServiceFactory' functionality.
Gzipped java files are attached. To summarize changes:

1) three new classes/interfaces were created (I put them in
net.sf.basedb.ws.client.added package):
	a) RPCServiceClientCreator - interface containing RPCServiceClient 
getService(String serviceUrl) method for creating RPCServiceClient objects
	b) DefaultRPCServiceClientCreator - factory class implementing 
interface above. It creates RPCServiceClient objects in the way they 
were created in AbstractRPCClient.getService() method.
	c) NoChunkRPCServiceClientCreator - the same as above but with CHUNK 
option set to false.


2) AbstractRPCClient class uses own RPCServiceClientCreator object to 
create objects in getService() method. Creator object is initialized in 
constructor (type of creator is an argument of additional constructor). 
The creator type can be chosen between values supported by 
AbstractRPCClient, available as public static constants (currently only 
DEFAULT_RPC_SERVICE_CLIENT and NO_CHUNK_RPC_SERVICE_CLIENT). If no 
creator type is supplied, default one is used.


3) all the subclasses of AbstractRPCClient have additional constructor 
with last argument being type of RPCServiceClientCreator which refers to 
constructor in superclass. This makes possible to choose creator type 
when programming BASE2 webservices clients.


I attached diffs.txt file which contains diff output on
net/sf/basedb/ws/client/ folders for my version and 2.7.1. Just thought, 
it might be helpful.


I hope the code will be useful.

Best wishes,
Pawel


Nicklas Nordborg wrote:

Pawel Sztromwasser wrote:

Hello BASE developers,

We have been recently trying to enable retrieving some data from
BASE2 installation directly into our 'home made' analysis tool. We
use webservices for that and have to say that you are doing a great
job.


Thanks.


We are really looking forward to use them more extensively.

Recently, when I started testing the tool not only within the local 
network I noticed that the webservice client does not switch off 
chunking. It results in attached exception cascade. Chunking is on by
 default in axis2 and some servers (i.e. our gateway to BASE which 
apparently uses HTTP1.0 and not 1.1) have problems with it. The

remedy is to use

options.setProperty(HTTPConstants.CHUNKED, false);

in net.sf.basedb.ws.client.AbstractRPCClient.getService(). I don't
know what are the drawbacks of it, but that was the only way I could
run the services in our current setup (BASE2.6 accessible by
proxypass from exposed webserver). Could this option be set off by
default or configurable somewhere? Or maybe there is some other
workaround?


I am afraid that I am not very familiar with all possible options that 
is available in the Axis API:s. There seems to be a lot of them and 
their Javadoc is not very informative about what many of the options are 
used for. The CHUNKED option seems to be related to the way HTTP 
requests are made, but I have to admit that this is the first time I 
have heard of it. This means that I probably know even less than you 
about this.


I think that, in the future, there may be more requests about being able 
to set other options. So, it may be better to think ahead and try to 
figure out some way to be able to do this as easy as possible. With the 
current BASE API the only workaround that doesn't involve changing the 
BASE code is to subclass all AbstractRPCClient subclasses and override 
the 'getService()' method. This is not very elegent since there are 
several such classes and the new 'getService()' will be added to each 
one of them.


A better approach may be to create a 'ServiceFactory' interface which 
has a 'createService()' method. The default implementation would then 
work as the AbstractRPCClient.getService() does today. The last step is 
to figure out a way to switch to another 'ServiceFactory' implementation 
either by having a configuration file or by being able to specify it 
programmatically.


I have opened a ticket for this issue 
http://base.thep.lu.se/ticket/1057. It falls under the 'contributions 
welcome' category since right now we are forced to prioritize other things.


/Nicklas



-
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel





new_client_package.tar.gz
Description: GNU Zip compressed data
diff net/sf/basedb/ws/client/AbstractRPCClient.java 
/Home/bccs/pawels/base2/base-2.7.1-src/src/webservices/client/java/net/sf/basedb/ws/client/AbstractRPCClien

Re: [basedb-devel] Webservices transport error: 411 Error: Length Required

[Pawel Sztromwasser]

Totally agree. That makes it much more flexible. Please, feel free to 
change the code.

Pawel

Nicklas Nordborg wrote:
> Thanks for the code. It was something like this that I had in mind. The 
> one thing I don't like is to pass the type as an 'int' parameter in the 
> constructor and having a switch statement that creates the actual object 
> to use. This makes it impossible for others to use their own 
> implementations of 'RPCServiceClientCreator'. Wouldn't it be better to 
> pass a 'RPCServiceClientCreator' directly to the constructor?
> 
> Eg.
> 
> protected AbstractRPCClient(
>String url, String service,
>    RPCServiceClientCreator serviceCreator)
> {
> ...
> }
> 
> /Nicklas
> 
> Pawel Sztromwasser wrote:
>> Hi Nicklas,
>>
>> I had some time and implemented the 'ServiceFactory' functionality.
>> Gzipped java files are attached. To summarize changes:
>>
>> 1) three new classes/interfaces were created (I put them in
>> net.sf.basedb.ws.client.added package):
>> a) RPCServiceClientCreator - interface containing RPCServiceClient 
>> getService(String serviceUrl) method for creating RPCServiceClient objects
>> b) DefaultRPCServiceClientCreator - factory class implementing 
>> interface above. It creates RPCServiceClient objects in the way they 
>> were created in AbstractRPCClient.getService() method.
>> c) NoChunkRPCServiceClientCreator - the same as above but with CHUNK 
>> option set to false.
>>
>> 2) AbstractRPCClient class uses own RPCServiceClientCreator object to 
>> create objects in getService() method. Creator object is initialized in 
>> constructor (type of creator is an argument of additional constructor). 
>> The creator type can be chosen between values supported by 
>> AbstractRPCClient, available as public static constants (currently only 
>> DEFAULT_RPC_SERVICE_CLIENT and NO_CHUNK_RPC_SERVICE_CLIENT). If no 
>> creator type is supplied, default one is used.
>>
>> 3) all the subclasses of AbstractRPCClient have additional constructor 
>> with last argument being type of RPCServiceClientCreator which refers to 
>> constructor in superclass. This makes possible to choose creator type 
>> when programming BASE2 webservices clients.
>>
>> I attached diffs.txt file which contains diff output on
>> net/sf/basedb/ws/client/ folders for my version and 2.7.1. Just thought, 
>> it might be helpful.
>>
>> I hope the code will be useful.
>>
>> Best wishes,
>> Pawel
>>
>>
>> Nicklas Nordborg wrote:
>>> Pawel Sztromwasser wrote:
>>>> Hello BASE developers,
>>>>
>>>> We have been recently trying to enable retrieving some data from
>>>> BASE2 installation directly into our 'home made' analysis tool. We
>>>> use webservices for that and have to say that you are doing a great
>>>> job.
>>> Thanks.
>>>
>>>> We are really looking forward to use them more extensively.
>>>>
>>>> Recently, when I started testing the tool not only within the local 
>>>> network I noticed that the webservice client does not switch off 
>>>> chunking. It results in attached exception cascade. Chunking is on by
>>>>  default in axis2 and some servers (i.e. our gateway to BASE which 
>>>> apparently uses HTTP1.0 and not 1.1) have problems with it. The
>>>> remedy is to use
>>>>
>>>> options.setProperty(HTTPConstants.CHUNKED, false);
>>>>
>>>> in net.sf.basedb.ws.client.AbstractRPCClient.getService(). I don't
>>>> know what are the drawbacks of it, but that was the only way I could
>>>> run the services in our current setup (BASE2.6 accessible by
>>>> proxypass from exposed webserver). Could this option be set off by
>>>> default or configurable somewhere? Or maybe there is some other
>>>> workaround?
>>> I am afraid that I am not very familiar with all possible options that 
>>> is available in the Axis API:s. There seems to be a lot of them and 
>>> their Javadoc is not very informative about what many of the options 
>>> are used for. The CHUNKED option seems to be related to the way HTTP 
>>> requests are made, but I have to admit that this is the first time I 
>>> have heard of it. This means that I probably know even less than you 
>>> about this.
>>>
>>> I think that, in the future, there may be more requests about being 
>>> able to set other options. So, it may be b

Re: [basedb-devel] Webservices transport error: 411 Error: Length Required

[Pawel Sztromwasser]

Nicklas Nordborg wrote:
> Pawel Sztromwasser wrote:
>> Totally agree. That makes it much more flexible. Please, feel free to 
>> change the code.
> 
> 
> I have now implemented this feature in the trunk. There are several 
> possibilities to specify which 'ServiceFactory' to use. The easiest
> is to use something like this when the application starts up:
> 
> Options opt = new Options();
> opt.setProperty(HTTPConstants.CHUNKED, false);
> ServiceFactory factory = new ConfigurableServiceFactory(opt);
> Factories.setDefaultServiceFactory(factory);
> 
> The rest of the code can remain unchanged. Another option is to specify 
> the factory when you create the SessionClient object:
> 
> SessionClient session = new SessionClient(url, null, null, factory);
> 
> I have not seen any differences between having the HTTPConstants.CHUNKED 
> parameter true or false, so it would really be nice if you can verify 
> that the configurable factory is working.
> 

Thank you for including the changes into 2.8. I tested both ways of 
using custom ServiceFactory and both work. I like more the first way, 
though.

Thanks once again and all the best,
Pawel


> /Nicklas
> 
> 
>> Pawel
>>
>> Nicklas Nordborg wrote:
>>> Thanks for the code. It was something like this that I had in mind. The 
>>> one thing I don't like is to pass the type as an 'int' parameter in the 
>>> constructor and having a switch statement that creates the actual object 
>>> to use. This makes it impossible for others to use their own 
>>> implementations of 'RPCServiceClientCreator'. Wouldn't it be better to 
>>> pass a 'RPCServiceClientCreator' directly to the constructor?
>>>
>>> Eg.
>>>
>>> protected AbstractRPCClient(
>>>String url, String service,
>>>RPCServiceClientCreator serviceCreator)
>>> {
>>> ...
>>> }
>>>
>>> /Nicklas
>>>
>>> Pawel Sztromwasser wrote:
>>>> Hi Nicklas,
>>>>
>>>> I had some time and implemented the 'ServiceFactory' functionality.
>>>> Gzipped java files are attached. To summarize changes:
>>>>
>>>> 1) three new classes/interfaces were created (I put them in
>>>> net.sf.basedb.ws.client.added package):
>>>> a) RPCServiceClientCreator - interface containing RPCServiceClient 
>>>> getService(String serviceUrl) method for creating RPCServiceClient objects
>>>> b) DefaultRPCServiceClientCreator - factory class implementing 
>>>> interface above. It creates RPCServiceClient objects in the way they 
>>>> were created in AbstractRPCClient.getService() method.
>>>> c) NoChunkRPCServiceClientCreator - the same as above but with CHUNK 
>>>> option set to false.
>>>>
>>>> 2) AbstractRPCClient class uses own RPCServiceClientCreator object to 
>>>> create objects in getService() method. Creator object is initialized in 
>>>> constructor (type of creator is an argument of additional constructor). 
>>>> The creator type can be chosen between values supported by 
>>>> AbstractRPCClient, available as public static constants (currently only 
>>>> DEFAULT_RPC_SERVICE_CLIENT and NO_CHUNK_RPC_SERVICE_CLIENT). If no 
>>>> creator type is supplied, default one is used.
>>>>
>>>> 3) all the subclasses of AbstractRPCClient have additional constructor 
>>>> with last argument being type of RPCServiceClientCreator which refers to 
>>>> constructor in superclass. This makes possible to choose creator type 
>>>> when programming BASE2 webservices clients.
>>>>
>>>> I attached diffs.txt file which contains diff output on
>>>> net/sf/basedb/ws/client/ folders for my version and 2.7.1. Just thought, 
>>>> it might be helpful.
>>>>
>>>> I hope the code will be useful.
>>>>
>>>> Best wishes,
>>>> Pawel
>>>>
>>>>
>>>> Nicklas Nordborg wrote:
>>>>> Pawel Sztromwasser wrote:
>>>>>> Hello BASE developers,
>>>>>>
>>>>>> We have been recently trying to enable retrieving some data from
>>>>>> BASE2 installation directly into our 'home made' analysis tool. We
>>>>>> use webservices for that and have to say that you are doing a great
>>>>>> job.
>>>>> Thanks.
>>>>>
>>>>>> We are real

[basedb-devel] Performance/delete issues on Postgres

[Pawel Sztromwasser]

Hello,

I found something interesting regarding BASE2 performance on Postgres 
(we have 8.3 but I suppose it is valid for previous versions as well). 
Recently I was unable to delete ArrayDesign (Illumina Human 8Ref, no 
items connected to it, features imported), because database query was 
endless. I stopped it after 3 days and started similar on development 
installation (much, much smaller). It took almost 20h to complete.

We started digging and found (among other similar) this: 
http://archives.postgresql.org/pgsql-admin/2006-02/msg00238.php
In short, postgres does not create indexes on foreign keys by default. 
That has huge impact on performance when updating/deleting items from 
table that is referenced by other big table. In case above Features 
(that are deleted with ArrayDesign) are referenced by all RawData* 
tables. I created indexes on feature_id column in all of these tables 
and delete took less than a minute.

That reminds me of another case where performance for postgres could be 
improved. When testing variuos kinds of jobs I noticed that concurent 
jobs creating root bioassayset took surprisingly long and did not scale 
well. That was due to sequential lookup in RawData* table on 
rawbioassay_id column. Creating index on that column improved 
performance by more than 50% (see: 
http://picasaweb.google.com/pawel.sztromwasser/Other#5257654388791782770).

I know that indexes have drawbacks as well, but in cases above they are 
a huge advantage. Could these indexes be incorporated in 
updateDb/indexes scripts? If you know about any other possible places in 
BASE database where similar things may occur, let me know and I will 
test them.

Best regards,
Pawel

-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel

Re: [basedb-devel] Illumina Bead Summary Files

[Pawel Sztromwasser]

We experienced that as well. I believe that I have checked these probes 
and they were controls, so they can be ignored.

Pawel

Nicklas Nordborg wrote:
> David Waring wrote:
>> I have recently installed the Illumina Plugins and have imported  
>> features for several Array Designs. We modified our config to  
>> generate the Bead Summary files and I have imported the data. Oddly  
>> have found that the Bead Summary files contain some rows with  
>> illumina codes that are not in the BGX files. For example the  
>> datasets from HumanRef-8 arrays contain 324 probes that are not found  
>> in any of the various versions of BGX files are available from the  
>> Illumina web site. If I run these same datasets through BeadStudio  
>> these these probes are not found in the BeadStudio output.
>>
>> Has anyone else run into this situation?
> 
> Yes, we have seen this also. Just select 'Probe not found=skip' when
> importing the data to BASE and everything should work.
> 
> /Nicklas
> 
> --
> Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
> -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
> -Strategies to boost innovation and cut costs with open source participation
> -Receive a $600 discount off the registration fee with the source code: SFAD
> http://p.sf.net/sfu/XcvMzF8H
> ___
> basedb-devel mailing list
> basedb-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/basedb-devel


--
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel

[basedb-devel] Batch scan importer

[Pawel Sztromwasser]

Hello BASE team,

We really enjoy the set of batch uploading plugins and were doing a 
little training for our users in Bergen. Everything seemed great except 
that they couldn't link freshly created scans with image files. I read 
in the example spreadsheet that it is not supported and was very 
surprised since similar 'file attaching' works very nice for raw 
bioassays. I sat down, did some copy pasting from RawBioAssayImporter 
and after little edits it worked (source attached).


I hope that you can make some use of it.

All the best,
Pawel
/**
	$Id: ScanImporter.java 4551 2008-09-29 07:47:24Z nicklas $

	Copyright (C) 2008 Nicklas Nordborg

	This file is part of BASE - BioArray Software Environment.
	Available at http://base.thep.lu.se/

	BASE is free software; you can redistribute it and/or
	modify it under the terms of the GNU General Public License
	as published by the Free Software Foundation; either version 3
	of the License, or (at your option) any later version.

	BASE is distributed in the hope that it will be useful,
	but WITHOUT ANY WARRANTY; without even the implied warranty of
	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
	GNU General Public License for more details.

	You should have received a copy of the GNU General Public License
	along with BASE. If not, see .
*/
package net.sf.basedb.plugins.batchimport;

import java.util.Collections;
import java.util.List;
import java.util.Set;

import net.sf.basedb.core.DbControl;
import net.sf.basedb.core.File;
import net.sf.basedb.core.Hardware;
import net.sf.basedb.core.Hybridization;
import net.sf.basedb.core.Image;
import net.sf.basedb.core.Item;
import net.sf.basedb.core.ItemQuery;
import net.sf.basedb.core.Path;
import net.sf.basedb.core.PathParameterType;
import net.sf.basedb.core.PluginParameter;
import net.sf.basedb.core.Protocol;
import net.sf.basedb.core.ProtocolType;
import net.sf.basedb.core.Scan;
import net.sf.basedb.core.SystemItems;
import net.sf.basedb.core.Version;
import net.sf.basedb.core.plugin.About;
import net.sf.basedb.core.plugin.AboutImpl;
import net.sf.basedb.core.plugin.GuiContext;
import net.sf.basedb.util.parser.FlatFileParser;
import net.sf.basedb.util.parser.Mapper;

/**
	Plug-in for importing scan items in a batch. The plug-in can create new 
	items and updated existing items.

	@author nicklas
	@version 2.8
	@base.modified $Date: 2008-09-29 09:47:24 +0200 (Mon, 29 Sep 2008) $
*/
public class ScanImporterNMC
	extends AbstractItemImporter
{

	private static final Set guiContexts =  
		Collections.singleton(new GuiContext(Item.SCAN, GuiContext.Type.LIST));

	private static final About about =
		new AboutImpl
	(
			"Scan importer with images",
			"Imports and updates scans in a batch (linking with images).",
			Version.getMajor() + "." + Version.getMinor() + "." + Version.getMaintenance(),
			"2008, Base 2 development team",
			null,
			null,
			"http://base.thep.lu.se";
	);

	
	protected static final PluginParameter hardwareColumnMapping = new PluginParameter(
			"hardwareColumnMapping",
			"Scanner",
			"Mapping that picks the name or ID of the array slide scanner from the data columns. " +
			"The plug-in will first try to find a scanner with the given name. If none is found and " +
			"the value is numeric it will try to load by internal ID. " +
			"Example: \\Scanner\\",
			optionalColumnMapping
			);

	protected static final PluginParameter hybridizationColumnMapping = new PluginParameter(
			"hybridizationColumnMapping",
			"Hybridization",
			"Mapping that picks the name or ID of the hybridization from the data columns. " +
			"The plug-in will first try to find a hybridization with the given name. If none is found and " +
			"the value is numeric it will try to load by internal ID. " +
			"Example: \\Hybridization\\",
			optionalColumnMapping
			);
	
	// extra parameter needed for linking with image files
	protected static final PluginParameter fileColumnMapping = new PluginParameter(
			"fileColumnMapping",
			"File",
			"Mapping that picks the path of a image file from the columns. " +
			"The path can be an absolute path (starting with /) or relative " +
			"to the 'Data directory' parameter." +
			"Example: \\File\\",
			optionalColumnMapping
			);
	
	public ScanImporterNMC()
	{}

	/*
		From the Plugin interface
		
	*/
	@Override
	public About getAbout()
	{
		return about;
	}
	// 

	/*
		From the InteractivePlugin interface
		
	*/
	@Override
	public Set getGuiContexts()
	{
		return guiContexts;
	}
	// 

	/*
		From the AbstractItemImporter class
		
	*/
	
	private Mapper nameMapper;
	private Mapper descriptionMapper;
	private Mapper protocolMapper;
	private Mapper hardwareMapper;
	private Mapper hybridizationMapper;
	//extra mappers for linking with image file
	private Mapper fileMap

Re: [basedb-devel] Batch scan importer

[Pawel Sztromwasser]

I have just noticed that the filename I sent has different name then the 
class inside. I used ScanImporterNMC name to distinguish from the 
original plugin and then renamed the file before sending. Just rename it 
back and all will work.

Pawel

Pawel Sztromwasser wrote:
> Hello BASE team,
> 
> We really enjoy the set of batch uploading plugins and were doing a 
> little training for our users in Bergen. Everything seemed great except 
> that they couldn't link freshly created scans with image files. I read 
> in the example spreadsheet that it is not supported and was very 
> surprised since similar 'file attaching' works very nice for raw 
> bioassays. I sat down, did some copy pasting from RawBioAssayImporter 
> and after little edits it worked (source attached).
> 
> I hope that you can make some use of it.
> 
> All the best,
> Pawel
> 
> 
> 
> 
> --
> Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
> powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
> easily build your RIAs with Flex Builder, the Eclipse(TM)based development
> software that enables intelligent coding and step-through debugging.
> Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
> 
> 
> 
> 
> ___
> basedb-devel mailing list
> basedb-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/basedb-devel


--
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel

Re: [basedb-devel] Batch scan importer

[Pawel Sztromwasser]

I am not sure, I didn't consider image as a 'real type' of item. More 
like attribute of a scan. Maybe because there is no Image submenu:)

I agree that looking for already created images with specific file is 
much better idea, then what I have implemented. Allowing users to 
specify image format, description and preview flag is great, but I would 
prefer to have it in the scan spreadsheet as additional columns (image 
property columns, as with file in rba import). Mainly because image has 
no life on its own and also because there is actually no image list 
where one could run batch import. OK, there is a list of images for 
every scan, but image imports for each and every scan would be laborious.

What do you think of it?

Pawel


Nicklas Nordborg wrote:
> Thanks for the code. I am not sure that I like that it creates a new image 
> every time.
> I think it should at least check if there is already an image with the 
> specified file.
> 
> Is there a need for setting the other image properties (preview, format, 
> description)?
> If so, it might be better to have a specific importer for images.
> 
> /Nicklas
> 
> Pawel Sztromwasser wrote:
>> Hello BASE team,
>>
>> We really enjoy the set of batch uploading plugins and were doing a
>> little training for our users in Bergen. Everything seemed great except
>> that they couldn't link freshly created scans with image files. I read
>> in the example spreadsheet that it is not supported and was very
>> surprised since similar 'file attaching' works very nice for raw
>> bioassays. I sat down, did some copy pasting from RawBioAssayImporter
>> and after little edits it worked (source attached).
>>
>> I hope that you can make some use of it.
>>
>> All the best,
>> Pawel
>>
>>
>> 
>>
>> --
>> Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
>> powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
>> easily build your RIAs with Flex Builder, the Eclipse(TM)based development
>> software that enables intelligent coding and step-through debugging.
>> Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
>>
>>
>> 
>>
>> ___
>> basedb-devel mailing list
>> basedb-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/basedb-devel
> 
> 
> --
> Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
> powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
> easily build your RIAs with Flex Builder, the Eclipse(TM)based development
> software that enables intelligent coding and step-through debugging.
> Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
> ___
> basedb-devel mailing list
> basedb-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/basedb-devel


--
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel

[basedb-devel] External id for protocol

[Pawel Sztromwasser]

Hello,

We have been exporting some experiments lately to ArrayExpress and 
noticed that protocols need to have their own ArrayExpress IDs. 
Currently we have to add them manually to export file generated in BASE 
(Tab2MageExporter), but it would be fairly easy to have it done 
automatically by the plugin. If there only was a place to store it in 
BASE2...

Would it be possible to have it as externalId (like BioMaterials have)? 
Or maybe it is better to enable annotations for protocols? This way it 
would be possible to add even more "programaticaly accessible" 
information to protocols. How do you think: which of these two is easier 
to implement? And which one makes more sense?

If any of these is straight-forward enough, I will probably go for a 
hack like additional column in Protocols table, using description field 
or similar, but I don't really like hack solutions. Thus, if you are 
also interested in implementing such a functionality, I am willing to 
contribute some code/time.

All the best
Pawel

--
The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
production scanning environment may not be a perfect world - but thanks to
Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700
Series Scanner you'll get full speed at 300 dpi even with all image 
processing features enabled. http://p.sf.net/sfu/kodak-com
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel

Re: [basedb-devel] External id for protocol

[Pawel Sztromwasser]

Nicklas Nordborg wrote:
> Pawel Sztromwasser wrote:
>> Hello,
>>
>> We have been exporting some experiments lately to ArrayExpress and 
>> noticed that protocols need to have their own ArrayExpress IDs. 
>> Currently we have to add them manually to export file generated in BASE 
>> (Tab2MageExporter), but it would be fairly easy to have it done 
>> automatically by the plugin. If there only was a place to store it in 
>> BASE2...
>>
>> Would it be possible to have it as externalId (like BioMaterials have)? 
>> Or maybe it is better to enable annotations for protocols? This way it 
>> would be possible to add even more "programaticaly accessible" 
>> information to protocols. How do you think: which of these two is easier 
>> to implement? And which one makes more sense?
> 
> I don't know if one method is better than the other. Biomaterials have both
> an externalId and annotations. Maybe there is a need for both. The drawback
> with the externalId approach is that it is rather undefined what it should be
> used for. Originally it was some kind of idea that it should point to some
> external sample/clinical database that could possible store sensitive 
> information
> and that BASE can't have access to. I don't know if anyone is using the
> externalId at all.
> 
> I think both approaches should be rather easy to implement. The hardest
> part is probably to update the web interface but since the desired 
> functionality
> exists elsewhere there is a lot of code to look at (and to copy and paste).
> 
>> If any of these is straight-forward enough, I will probably go for a 
>> hack like additional column in Protocols table, using description field 
>> or similar, but I don't really like hack solutions. Thus, if you are 
>> also interested in implementing such a functionality, I am willing to 
>> contribute some code/time.
> 
> I don't see any reason not to have the functionality. We will appreciate any
> contributions. Since both functions require changes to the database you must
> work against the trunk source code and there are a few "bookkeeping" things to
> consider, but we can help you with that. Can you create one ticket for the
> externalId approach and one ticket for the annotation approach to the BASE 
> website?
> 

done!

> When you have code to submit it is simple to generate at patch with the 
> command
> 'ant svn.diff' and upload/attach the generated file to the ticket.
> 
> What timeframe do you have on this? We have not set a date for BASE 2.12 yet, 
> but
> expect to have most things completed within a few weeks. We are also doing 
> some more
> work on plug-ins and extensions that are related so it may be up to a month 
> before
> everything is finished.
> 

Well, it could be difficult with time before July. I am off work for the 
whole of June and only one day in a week dedicated to BASE. I will 
probably start next week, but don't expect anything working before my 
holidays. I guess this makes it patch for 2.13 rather then 2.12...

Pawel

> /Nicklas
> 
> --
> The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
> production scanning environment may not be a perfect world - but thanks to
> Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700
> Series Scanner you'll get full speed at 300 dpi even with all image 
> processing features enabled. http://p.sf.net/sfu/kodak-com
> ___
> basedb-devel mailing list
> basedb-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/basedb-devel


--
The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
production scanning environment may not be a perfect world - but thanks to
Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700
Series Scanner you'll get full speed at 300 dpi even with all image 
processing features enabled. http://p.sf.net/sfu/kodak-com
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel

Re: [basedb-devel] External id for protocol

[Pawel Sztromwasser]

Hi Nicklas,

That was much quicker then I expected. Patch is attached here: 
http://base.thep.lu.se/ticket/1310

I just hope that I haven't missed too much lines while copy-pasting your 
code;)

All the best,
Pawel


Nicklas Nordborg wrote:
> Pawel Sztromwasser wrote:
> 
>>> I don't see any reason not to have the functionality. We will appreciate any
>>> contributions. Since both functions require changes to the database you must
>>> work against the trunk source code and there are a few "bookkeeping" things 
>>> to
>>> consider, but we can help you with that. Can you create one ticket for the
>>> externalId approach and one ticket for the annotation approach to the BASE 
>>> website?
>>>
>> done!
> 
> Great.
> 
>>> When you have code to submit it is simple to generate at patch with the 
>>> command
>>> 'ant svn.diff' and upload/attach the generated file to the ticket.
>>>
>>> What timeframe do you have on this? We have not set a date for BASE 2.12 
>>> yet, but
>>> expect to have most things completed within a few weeks. We are also doing 
>>> some more
>>> work on plug-ins and extensions that are related so it may be up to a month 
>>> before
>>> everything is finished.
>>>
>> Well, it could be difficult with time before July. I am off work for the 
>> whole of June and only one day in a week dedicated to BASE. I will 
>> probably start next week, but don't expect anything working before my 
>> holidays. I guess this makes it patch for 2.13 rather then 2.12...
> 
> Thats fine with us. I was just worried that you needed this sooner.
> 
> /Nicklas
> 
> --
> The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
> production scanning environment may not be a perfect world - but thanks to
> Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700
> Series Scanner you'll get full speed at 300 dpi even with all image 
> processing features enabled. http://p.sf.net/sfu/kodak-com
> ___
> basedb-devel mailing list
> basedb-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/basedb-devel


--
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables 
unlimited royalty-free distribution of the report engine 
for externally facing server and web deployment. 
http://p.sf.net/sfu/businessobjects
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel

Re: [basedb-devel] External id for protocol

[Pawel Sztromwasser]

Nicklas Nordborg wrote:
> Pawel Sztromwasser wrote:
>> Hi Nicklas,
>>
>> That was much quicker then I expected. Patch is attached here: 
>> http://base.thep.lu.se/ticket/1310
>>
>> I just hope that I haven't missed too much lines while copy-pasting your 
>> code;)
> 
> The code is looking good. I have not yet tested it but will do that as soon
> as I have some time left. Thanks a lot for fixing this. Is it ok for us to 
> include
> it in the 2.12 release? We'll take care of fixing the database schema 
> versions and
> other stuff related to upgrading.
> 

Of course it is OK. I'll be happy if it is included.

Pawel

> /Nicklas
> 
> 
> --
> Crystal Reports - New Free Runtime and 30 Day Trial
> Check out the new simplified licensing option that enables 
> unlimited royalty-free distribution of the report engine 
> for externally facing server and web deployment. 
> http://p.sf.net/sfu/businessobjects
> ___
> basedb-devel mailing list
> basedb-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/basedb-devel


--
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables 
unlimited royalty-free distribution of the report engine 
for externally facing server and web deployment. 
http://p.sf.net/sfu/businessobjects
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel

[basedb-devel] External authentication

[Pawel Sztromwasser]

Hi,

I am working on an external authentication plugin for BASE that will use 
an STS Web service to authenticate users. The plugin uses a bunch of 
external jars for WS communication, but with different versions then 
BASE. I was hoping for the same classloading mechanism as for regular 
plugins (separate classloaders), but it looks like external 
authentication plugins uses the default classloader of the webapp. Could 
this be changed? How else could I install my plugin, without messing 
with BASE's jars?

Thanks,
Pawel

--
Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
Finally, a world-class log management solution at an even better price-free!
Download using promo code Free_Logger_4_Dev2Dev. Offer expires 
February 28th, so secure your free ArcSight Logger TODAY! 
http://p.sf.net/sfu/arcsight-sfd2d
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel

Re: [basedb-devel] External authentication

[Pawel Sztromwasser]

Thanks for a quick reply. I really wanted to avoid selecting jars and so 
on since BASE has over 50 of them, and maven downloads over 60 for my 
plugin. Lots of manual work that needs to be done every time BASE is 
upgraded.

I will try to change the Application class as you suggested. I have 
already located the place which needs a fix, but was unsure of playing 
so close to the core of the application. What about using the 
net.sf.basedb.util.JarClassLoader, as regular plugins do?

I will post a patch when ready and tested.
Cheers,
Pawel

On 01/02/11 13:03, Nicklas Nordborg wrote:
> On 2011-02-01 11:29, Pawel Sztromwasser wrote:s
>> Hi,
>>
>> I am working on an external authentication plugin for BASE that will use
>> an STS Web service to authenticate users. The plugin uses a bunch of
>> external jars for WS communication, but with different versions then
>> BASE. I was hoping for the same classloading mechanism as for regular
>> plugins (separate classloaders), but it looks like external
>> authentication plugins uses the default classloader of the webapp. Could
>> this be changed? How else could I install my plugin, without messing
>> with BASE's jars?
> That part of BASE is not very "pluginified". I am not aware of any real
> case that uses external authentication so it would be very interesting
> to see if you succeed.
>
> It's quite common that there are dependencies to different versions of
> 3rd-party JAR files, but it usually works if the latest one is used. But
> of course, it may not work if the API has changed in an incompatible way.
>
> If you can't get it to work by clever selection of JAR files, then you
> might have to modify the Application.getAuthenticator() method so that
> it uses a different class loader. It shouldn't be too difficult. It
> could for example pick up a JAR path from the base.config file and then
> use the net.sf.basedb.util.JarClassLoader.getInstance(path) to create a
> class loader. Dependencies need to be listed in the MANIFEST.MF in the
> same way as for plug-ins. There is a short notice about this at
> http://base.thep.lu.se/chrome/site/latest/html/developerdoc/plugin_developer/plugin_developer.organize.html
> and
> http://base.thep.lu.se/chrome/site/latest/api/net/sf/basedb/util/JarClassLoader.html
>
> As always, patches are welcome :)
>
> /Nicklas
>
> --
> Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
> Finally, a world-class log management solution at an even better price-free!
> Download using promo code Free_Logger_4_Dev2Dev. Offer expires
> February 28th, so secure your free ArcSight Logger TODAY!
> http://p.sf.net/sfu/arcsight-sfd2d
> ___
> basedb-devel mailing list
> basedb-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/basedb-devel


--
Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
Finally, a world-class log management solution at an even better price-free!
Download using promo code Free_Logger_4_Dev2Dev. Offer expires 
February 28th, so secure your free ArcSight Logger TODAY! 
http://p.sf.net/sfu/arcsight-sfd2d
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel

Re: [basedb-devel] External authentication

[Pawel Sztromwasser]

Sorry, just noticed that you mentioned the 
...basedb.util.JarClassLoader, not a regular Java class loader as I 
initially  thought.

On 01/02/11 13:17, Pawel Sztromwasser wrote:
> Thanks for a quick reply. I really wanted to avoid selecting jars and so
> on since BASE has over 50 of them, and maven downloads over 60 for my
> plugin. Lots of manual work that needs to be done every time BASE is
> upgraded.
>
> I will try to change the Application class as you suggested. I have
> already located the place which needs a fix, but was unsure of playing
> so close to the core of the application. What about using the
> net.sf.basedb.util.JarClassLoader, as regular plugins do?
>
> I will post a patch when ready and tested.
> Cheers,
> Pawel
>
> On 01/02/11 13:03, Nicklas Nordborg wrote:
>> On 2011-02-01 11:29, Pawel Sztromwasser wrote:s
>>> Hi,
>>>
>>> I am working on an external authentication plugin for BASE that will use
>>> an STS Web service to authenticate users. The plugin uses a bunch of
>>> external jars for WS communication, but with different versions then
>>> BASE. I was hoping for the same classloading mechanism as for regular
>>> plugins (separate classloaders), but it looks like external
>>> authentication plugins uses the default classloader of the webapp. Could
>>> this be changed? How else could I install my plugin, without messing
>>> with BASE's jars?
>> That part of BASE is not very "pluginified". I am not aware of any real
>> case that uses external authentication so it would be very interesting
>> to see if you succeed.
>>
>> It's quite common that there are dependencies to different versions of
>> 3rd-party JAR files, but it usually works if the latest one is used. But
>> of course, it may not work if the API has changed in an incompatible way.
>>
>> If you can't get it to work by clever selection of JAR files, then you
>> might have to modify the Application.getAuthenticator() method so that
>> it uses a different class loader. It shouldn't be too difficult. It
>> could for example pick up a JAR path from the base.config file and then
>> use the net.sf.basedb.util.JarClassLoader.getInstance(path) to create a
>> class loader. Dependencies need to be listed in the MANIFEST.MF in the
>> same way as for plug-ins. There is a short notice about this at
>> http://base.thep.lu.se/chrome/site/latest/html/developerdoc/plugin_developer/plugin_developer.organize.html
>> and
>> http://base.thep.lu.se/chrome/site/latest/api/net/sf/basedb/util/JarClassLoader.html
>>
>> As always, patches are welcome :)
>>
>> /Nicklas
>>
>> --
>> Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
>> Finally, a world-class log management solution at an even better price-free!
>> Download using promo code Free_Logger_4_Dev2Dev. Offer expires
>> February 28th, so secure your free ArcSight Logger TODAY!
>> http://p.sf.net/sfu/arcsight-sfd2d
>> ___
>> basedb-devel mailing list
>> basedb-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/basedb-devel
>
> --
> Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
> Finally, a world-class log management solution at an even better price-free!
> Download using promo code Free_Logger_4_Dev2Dev. Offer expires
> February 28th, so secure your free ArcSight Logger TODAY!
> http://p.sf.net/sfu/arcsight-sfd2d
> ___
> basedb-devel mailing list
> basedb-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/basedb-devel


--
Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
Finally, a world-class log management solution at an even better price-free!
Download using promo code Free_Logger_4_Dev2Dev. Offer expires 
February 28th, so secure your free ArcSight Logger TODAY! 
http://p.sf.net/sfu/arcsight-sfd2d
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel

Re: [basedb-devel] External authentication

[Pawel Sztromwasser]

Hi Nicklas,

I changed the Application.getAuthenticator() to use a separate 
classloader for external login plugin (attached a diff to the ticket 
#1575) and I was able to verify user's credentials using an STS service.

I got a bit unexpected behavior of BASE that tried to create a new 
account for the user after it failed finding it by an externalId. Of 
course, I had not set the externalId in advance. The logins of the 
external STS service and BASE account happened to be the same, so the 
whole process failed violating the unique-key constraint on login 
column. This is sth I will need to consider when integrating two sets of 
users from different applications.

But there is another problem I was struggling with. And apparently not 
only me [1]. Tomcat/axis/java has a problem when two 
axis2-kernel-x.x.jar jars are in the classpath. Even if the versions are 
the same. I tried multiple combinations with axis2 1.4 and 1.5.1. I 
downgraded the plugin to use axis2-1.3 and rampart1.3. I stripped-down 
the set of jars that my plugin uses, trying to use as many libraries 
shipped with BASE as possible. In principle it should work as long as 
all the dependencies were satisfied (by parent classloader of webapp), 
but when axis2-kernel was not loaded by the same classloader as Apache 
Rampart jars, I couldn't get rampart to engage.

It seems like it is impossible to have axis2 in more than one location 
in the classpath. The only way it works is when only one 
axis2-kernel.jar is loaded by tomcat. So I either need to include all 
dependencies of my plugin into the /WEB-INF/lib directory of BASE, and 
use axis2 shipped with BASE; or I remove the axis2-kernel-1.3.jar from 
WEB-INF/lib of BASE and load it together with my plugin and rest of its 
dependencies. The latter one will of course break BASE's Web Services. I 
haven't had problems with other jars, despite I use different version of 
(for example) axiom libraries. No conflicts with other axis2 libraries, 
only axis2-kernel.

I am happy because it works now, although with BASE's axis2 1.3. But I 
can imagine that in some time I might need to use newer axis2, and then 
I will have to try to upgrade it in BASE as well. If it is backwards 
compatible, it should go smooth, and the BASE Web Services will work. 
But it would be still good to test. Is there a test suite available for 
BASE Web Services that I could try? Or maybe you were considering 
upgrade yourself?

Pawel

[1] https://issues.apache.org/jira/browse/AXIS2-2972



Nicklas Nordborg wrote:
> I added a ticket:http://base.thep.lu.se/ticket/1575
>
> /Nicklas
>
> --
> Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
> Finally, a world-class log management solution at an even better price-free!
> Download using promo code Free_Logger_4_Dev2Dev. Offer expires
> February 28th, so secure your free ArcSight Logger TODAY!
> http://p.sf.net/sfu/arcsight-sfd2d
> ___
> basedb-devel mailing list
> basedb-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/basedb-devel
>


--
Free Software Download: Index, Search & Analyze Logs and other IT data in 
Real-Time with Splunk. Collect, index and harness all the fast moving IT data 
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business 
insights. http://p.sf.net/sfu/splunk-dev2dev 
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel

[basedb-devel] Programmatic login using a token

[Pawel Sztromwasser]

Hello BASE team!

Extending a little on our recent developments regarding external 
authentication in BASE [1], I am working now on a Web service interface 
to BASE that will authenticate users based on secure tokens issued by a 
trusted STS. The STS authenticates users with username and password, and 
issues a crypted token confirming their credentials. The token is valid 
for a limited time and can be used for authentication in applications 
and services that trust the STS.


The STS token does not contain a password, only a user id, so with 
current implementation of SessionControl I was unable to log in. I added 
one method to the SessionControl class (see attached diff), included a 
TokenAuthenticator interface (attached), and everything worked nice. The 
extra operation allows to log in using a single token object and a 
validator that can verify the token. The operation contains substantial 
parts of SessionControl.verifyUserExternal(), which can be extracted to 
a separate method. The TokenAuthenticator interface and original 
Authenticator interface are also quite similar, so they could maybe 
share a common ancestor.


I know that the change is in the heart of BASE, but it is little and 
would provide additional way of programmatic interaction with BASE. If 
they pass the tests, could you consider including the changes to the 
BASE codebase?


Best wishes,
Pawel Sztromwasser

[1] 
http://www.mail-archive.com/basedb-devel@lists.sourceforge.net/msg00433.html)
25a26,44
> import net.sf.basedb.core.authentication.TokenAuthenticator;	
2240a2243,2352
> 	
> 	
> 	/**
> 	 * Log in to BASE using a trusted authentication token. 
> 	 * The method delegates validation of the token to tokenAuthenticator.
> 	 * @param token
> 	 *authentication token
> 	 * @param tokenAuthenticator
> 	 * 	  authenticator object that is able to verify the token and authenticate the user
> 	 * @throws ItemNotFoundException
> 	 * If a user identified by the token is not found
> 	 * @throws InvalidPasswordException
> 	 * If the token is invalid
> 	 * @throws PermissionDeniedException
> 	 * If the user doesn't have {@link Permission#USE} permission
> 	 * for the current client application or if a user is already
> 	 * logged in
> 	 * @throws BaseException
> 	 * If there is any other error
> 	 * @see #logout()
> 	 * @see #isLoggedIn()
> 	 * @see #getLoggedInUserId()
> 	 */
> 	public synchronized  void login(T token, TokenAuthenticator tokenAuthenticator)
> 	throws ItemNotFoundException, PermissionDeniedException, InvalidPasswordException, BaseException {
> 		
> 		updateLastAccess();
> 		// Are we already logged in?
> 		if (isLoggedIn()) {
> 			throw new AlreadyLoggedInException(loginInfo.userLogin);
> 		}
> 		
> 		AuthenticationInformation authInfo = tokenAuthenticator.authenticate(token);
> 		String login = authInfo.login;
> 		
> 		if (login == null)
> 			throw new InvalidUseOfNullException("login");
> 		
> 		org.hibernate.Session session = null;
> 		org.hibernate.Transaction tx = null;
> 		try {
> 			session = HibernateUtil.newSession();
> 			tx = HibernateUtil.newTransaction(session);
> 			
> 			org.hibernate.Query query = HibernateUtil.getPredefinedQuery(session, "GET_USER_FOR_EXTERNAL_ID");
> 			/*
> SELECT usr 
> FROM UserData usr
> WHERE usr.externalId = :externalId
> 			*/
> 			query.setParameter("externalId", authInfo.id);
> 			UserData userData = HibernateUtil.loadData(UserData.class, query);
> 			if (userData == null) 
> 			{
> userData = new UserData();
> userData.setExternalId(authInfo.id);
> userData.setLogin(authInfo.login);
> userData.setName(authInfo.name == null ? authInfo.login : authInfo.name);
> userData.getPassword().setMd5Password("");
> userData.setQuota(HibernateUtil.loadData(session, QuotaData.class, SystemItems.getId(Quota.DEFAULT)));
> User.addDefultRolesAndGroups(session, userData);
> 			}
> 			if (tokenAuthenticator.supportsExtraInformation() &&
> (Config.getBoolean("auth.synchronize") || userData.getId() == 0))
> 			{
> userData.setName(authInfo.name == null ? authInfo.login : authInfo.name);
> userData.setOrganisation(authInfo.organisation);
> userData.setAddress(authInfo.address);
> userData.setPhone(authInfo.phone);
> userData.setFax(authInfo.fax);
> userData.setEmail(authInfo.email);
> userData.setUrl(authInfo.url);
> userData.setDescription(authInfo.description);
> 			}
> 			if (userData.getId() == 0)
> 			{
> HibernateUtil.saveData(session, userData);	
> 			}
> 			HibernateUtil.flush(session);
> 
> 			LoginInfo li = cr

Re: [basedb-devel] Programmatic login using a token

[Pawel Sztromwasser]

Hi Nicklas,

Thanks for the suggestions. Maybe I am missing something but wouldn't it 
be possible to send the username and password to the STS as part of the 
BASE login? Or, use an empty password string?


I should have explained the use case better. It does not relate to the 
manual login to BASE using the web interface where one types in a 
username and password. For this we have already implemented and set up 
an authentication plugin that sends the username and password typed in 
by a user to the STS service for verification (just like you suggest). 
Now I am planning to enable programmatic access to BASE (via Web 
services) from another system that shares identity management (STS) with 
BASE. The idea is that the user that is logged in to any of these, has 
obtained a token and is able to access his/her resources in the other 
system without any further authentication, using only the token (single 
sign-on). In this federated authentication system one application will 
be able to automatically integrate resources available to a user in 
several distinct systems.


Since the common authentication mechanism will be based on tokens, I 
don't expect an application to provide a username/password pair anytime 
when it needs to access resources hosted in a different system. 
Currently all the login methods in BASE are username/password oriented, 
but with the minor change I suggest, it could be easily extended to 
cover token-like objects as well. At least in programmatic access 
scenarios, but I can't see a need for any other.


I am not sure if I got your suggestion with empty password right, so 
correct me if I am wrong. The token is not a simple string (it is 
actually a little XML document), and passing it as a username (with 
empty password) to the authenticator plugin would not be a nice solution.


In any case, I am not sure that supplying a TokenAuthenticator object as 
a parameter to the login method is a good idea, since it would be very 
easy to to provide an implementation that just accept anything.



I was thinking similarly, but how does it differ from supplying a login 
method in an authenticator plugin? That method could accept anything as 
well. Access to SessionControl.login(Token, TokenAuthenticator) 
operation in only from the code deployed together with BASE server (in 
my case in implementation of Web services deployed within BASE), so an 
environment that a BASE administrator should have full control over. The 
same as external authentication plugins.


Pawel

Nicklas Nordborg wrote:

On 2011-07-26 22:47, Pawel Sztromwasser wrote:
  

Hello BASE team!

Extending a little on our recent developments regarding external
authentication in BASE [1], I am working now on a Web service interface
to BASE that will authenticate users based on secure tokens issued by a
trusted STS. The STS authenticates users with username and password, and
issues a crypted token confirming their credentials. The token is valid
for a limited time and can be used for authentication in applications
and services that trust the STS.

The STS token does not contain a password, only a user id, so with
current implementation of SessionControl I was unable to log in. I added
one method to the SessionControl class (see attached diff), included a
TokenAuthenticator interface (attached), and everything worked nice. The
extra operation allows to log in using a single token object and a
validator that can verify the token. The operation contains substantial
parts of SessionControl.verifyUserExternal(), which can be extracted to
a separate method. The TokenAuthenticator interface and original
Authenticator interface are also quite similar, so they could maybe
share a common ancestor.

I know that the change is in the heart of BASE, but it is little and
would provide additional way of programmatic interaction with BASE. If
they pass the tests, could you consider including the changes to the
BASE codebase?




Thanks for the suggestions. Maybe I am missing something but wouldn't it 
be possible to send the username and password to the STS as part of the 
BASE login? Or, use an empty password string?


In any case, I am not sure that supplying a TokenAuthenticator object as 
a parameter to the login method is a good idea, since it would be very 
easy to to provide an implementation that just accept anything.


There are also some plans to change the authentication as part of BASE 3 
development (http://base.thep.lu.se/ticket/1599). There is not much 
information since we haven't been thinking very much about this and at 
the moment I am not sure if we will have time to do it before releasing 
BASE 3. It depends a bit on when some of the other BASE 3 features are 
needed in production. I'll keep a link to this thread so that we don't 
forget it.


/Nicklas

--
Got Input?   Slashdot Needs You.
Take our quick su

Re: [basedb-devel] Programmatic login using a token

[Pawel Sztromwasser]

See my short replies inline.

Nicklas Nordborg wrote:

On 2011-08-01 10:51, Pawel Sztromwasser wrote:
  

Hi Nicklas,



Thanks for the suggestions. Maybe I am missing something but wouldn't
it be possible to send the username and password to the STS as part of
the BASE login? Or, use an empty password string?
  

I should have explained the use case better. It does not relate to the
manual login to BASE using the web interface where one types in a
username and password. For this we have already implemented and set up
an authentication plugin that sends the username and password typed in
by a user to the STS service for verification (just like you suggest).
Now I am planning to enable programmatic access to BASE (via Web
services) from another system that shares identity management (STS) with
BASE. The idea is that the user that is logged in to any of these, has
obtained a token and is able to access his/her resources in the other
system without any further authentication, using only the token (single
sign-on). In this federated authentication system one application will
be able to automatically integrate resources available to a user in
several distinct systems.

Since the common authentication mechanism will be based on tokens, I
don't expect an application to provide a username/password pair anytime
when it needs to access resources hosted in a different system.
Currently all the login methods in BASE are username/password oriented,
but with the minor change I suggest, it could be easily extended to
cover token-like objects as well. At least in programmatic access
scenarios, but I can't see a need for any other.



Ok, I understand that this is a bit different than the usual login 
procedure. But I think there is also already another possible way to 
solve this. It could be done in a similar way that we are doing with the 
job agents. They use a "master" user account with a known 
username+password that is used to login to BASE. The account should have 
the "Act as another user" permissions assigned to a role (eg. just as 
the "Job agent" role that is pre-installed in BASE).


Then the SessionControl.impersonateLogin() can be used to login as any 
user without having to provide a password. You'll have to find the 
internal id of the user but that is more or less the same thing that is 
done in the diff for SessionControl that you posted in the original post.
  

Ahhh, clever. And no change to BASE required. It looks promising, I will give 
it a try.

  

In any case, I am not sure that supplying a TokenAuthenticator object
as a parameter to the login method is a good idea, since it would be
very easy to to provide an implementation that just accept anything.
  

I was thinking similarly, but how does it differ from supplying a login
method in an authenticator plugin? That method could accept anything as
well. Access to SessionControl.login(Token, TokenAuthenticator)
operation in only from the code deployed together with BASE server (in
my case in implementation of Web services deployed within BASE), so an
environment that a BASE administrator should have full control over. The
same as external authentication plugins.



The difference is that the administrator sets up the authenticator to 
use in the configuration files and it is not possible for the calling 
code to change this. Ok, some "mean" code could probably get around 
this, so it is good to have control over the server environment. Despite 
this, I think the public BASE api shouldn't provide this kind of 
flexibility. I would prefer a solution where the administrator can setup 
exactly how the authentication should happen.
  

Fair enough.

/Nicklas

--
Got Input?   Slashdot Needs You.
Take our quick survey online.  Come on, we don't ask for help often.
Plus, you'll get a chance to win $100 to spend on ThinkGeek.
http://p.sf.net/sfu/slashdot-survey
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel
  


<>--
Got Input?   Slashdot Needs You.
Take our quick survey online.  Come on, we don't ask for help often.
Plus, you'll get a chance to win $100 to spend on ThinkGeek.
http://p.sf.net/sfu/slashdot-survey
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel

[basedb-devel] Truncated files

[Pawel Sztromwasser]

In BASE it was a Tomcat problem. Hope it helps:
http://www.mail-archive.com/basedb-users@lists.sourceforge.net/msg01838.html

Pawel
<>--
Get a FREE DOWNLOAD! and learn more about uberSVN rich system, 
user administration capabilities and model configuration. Take 
the hassle out of deploying and managing Subversion and the 
tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel

Re: [basedb-devel] Truncated files

[Pawel Sztromwasser]

Sorry, wrong mailing list:)

Pawel Sztromwasser wrote:

In BASE it was a Tomcat problem. Hope it helps:
http://www.mail-archive.com/basedb-users@lists.sourceforge.net/msg01838.html 



Pawel


<>--
Get a FREE DOWNLOAD! and learn more about uberSVN rich system, 
user administration capabilities and model configuration. Take 
the hassle out of deploying and managing Subversion and the 
tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel

[basedb-devel] Mage-Tab export plugin

[Pawel Sztromwasser]

Hello BASE developers,

By the end of the year ArrayExpress is going to remove a possibility of 
importing experiments using the Tab2Mage format. The only supported 
format will be Mage-Tab. We have been extensively using Tab2Mage and the 
BASE plugin that exports experiments in this format. I was wondering if 
anyone else is developing or planning to develop a plugin that would 
support Mage-Tab?


Regards,
Pawel Sztromwasser
<>--
Special Offer -- Download ArcSight Logger for FREE!
Finally, a world-class log management solution at an even better 
price-free! And you'll get a free "Love Thy Logs" t-shirt when you
download Logger. Secure your free ArcSight Logger TODAY!
http://p.sf.net/sfu/arcsisghtdev2dev___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel

Re: [basedb-devel] Mage-Tab export plugin

[Pawel Sztromwasser]

Hallo,

I haven't been to the milestones page for a while, and I didn't know 
that work on BASE 3 was in such an advanced state. Great job guys! 
Support for sequencing data is excellent news for us.
Anyway, before it is released and before we are ready to migrate, we 
will certainly have experiments to export. We could keep using the 
Tab2Mage exporter this year, and the conversion script later, but I 
think I'd prefer to start working on the plugin now when we have time, 
develop against BASE 2 where we would have immediate use for it, and 
port to BASE 3 later after we migrate. It will not be a big change I hope.


Gildas, I have actually started developing the plugin. If you're willing 
to contribute, I could send you the jar/code when it is more-or-less 
ready for testing, so we could debug it on our sites simultanously and 
make it bullet-proof;)


Helen, IMO it is better with one format than two, so I think removing 
support for Tab2Mage (and providing conversion tools) is a move forward. 
One thing I was wondering, if there are any plans for an XML-based 
format for description of MA experiments? MAGE-ML is a bit too complex I 
guess, but an XML Schema equivalent to the current MageTab format would 
be favored by programmers and for programmatic access, in comparison 
with the tab-delimited files.


Best wishes,
Pawel

Nicklas Nordborg wrote:

On 2011-09-01 09:15, Gildas Le Corguillé wrote:
  

Hello,

Our group still interesting in this kind of exporter. It was the main
reason why we decided to use Base. We are in a public lab so we like to
share :P
Last year, I repaired the existing Tab2Mage-exporter module for Base.
So if you need help, I can spend some time on that. But I'm not
necessarily a good developer ^^'



I think that a Mage-Tab exporter has to be developed by the BASE 
community. We can provide space and user accounts on our subversion+trac 
(baseplugins.thep.lu.se) if someone is interested to start on this. 
Unfortunately we have no time to help with the actual development. Of 
course, we'll try to answer questions and provide the information that 
is necessary.


As a starting point, we try to collect and document the major API 
changes in http://base.thep.lu.se/ticket/1603


My guess is that #1153 and #1597 are the major changes that cause 
problems for the current Tab2Mage exporter.


/Nicklas


--
Special Offer -- Download ArcSight Logger for FREE!
Finally, a world-class log management solution at an even better 
price-free! And you'll get a free "Love Thy Logs" t-shirt when you

download Logger. Secure your free ArcSight Logger TODAY!
http://p.sf.net/sfu/arcsisghtdev2dev
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel
  


<>--
Special Offer -- Download ArcSight Logger for FREE!
Finally, a world-class log management solution at an even better 
price-free! And you'll get a free "Love Thy Logs" t-shirt when you
download Logger. Secure your free ArcSight Logger TODAY!
http://p.sf.net/sfu/arcsisghtdev2dev___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel

Re: [basedb-devel] Mage-Tab export plugin

[Pawel Sztromwasser]

Great. Yes, I am planning to implement that as well.

Pawel

On 12/09/11 11:48, Gildas Le Corguillé wrote:

Bonjour,

This is good news if you can develope this plugin and I have enought 
datas on Base2 to test it :P


My contribution on the former plugin Tab2Mage was to add a feature to 
manage pooling of samples (or extracts) and the possibility to have 
several "Files[raw]" per RawBioAssay (ex : one cy3 and one cy5) 
because we used nimblegen multi-color arrays.

Are you plan to manage that feature ?

Gildas


Le 01/09/2011 14:43, Pawel Sztromwasser a écrit :

Hallo,

I haven't been to the milestones page for a while, and I didn't know 
that work on BASE 3 was in such an advanced state. Great job guys! 
Support for sequencing data is excellent news for us.
Anyway, before it is released and before we are ready to migrate, we 
will certainly have experiments to export. We could keep using the 
Tab2Mage exporter this year, and the conversion script later, but I 
think I'd prefer to start working on the plugin now when we have 
time, develop against BASE 2 where we would have immediate use for 
it, and port to BASE 3 later after we migrate. It will not be a big 
change I hope.


Gildas, I have actually started developing the plugin. If you're 
willing to contribute, I could send you the jar/code when it is 
more-or-less ready for testing, so we could debug it on our sites 
simultanously and make it bullet-proof;)


Helen, IMO it is better with one format than two, so I think removing 
support for Tab2Mage (and providing conversion tools) is a move 
forward. One thing I was wondering, if there are any plans for an 
XML-based format for description of MA experiments? MAGE-ML is a bit 
too complex I guess, but an XML Schema equivalent to the current 
MageTab format would be favored by programmers and for programmatic 
access, in comparison with the tab-delimited files.


Best wishes,
Pawel

Nicklas Nordborg wrote:

On 2011-09-01 09:15, Gildas Le Corguillé wrote:

Hello,

Our group still interesting in this kind of exporter. It was the main
reason why we decided to use Base. We are in a public lab so we 
like to

share :P
Last year, I repaired the existing Tab2Mage-exporter module for Base.
So if you need help, I can spend some time on that. But I'm not
necessarily a good developer ^^'


I think that a Mage-Tab exporter has to be developed by the BASE 
community. We can provide space and user accounts on our 
subversion+trac (baseplugins.thep.lu.se) if someone is interested to 
start on this. Unfortunately we have no time to help with the actual 
development. Of course, we'll try to answer questions and provide 
the information that is necessary.


As a starting point, we try to collect and document the major API 
changes in http://base.thep.lu.se/ticket/1603


My guess is that #1153 and #1597 are the major changes that cause 
problems for the current Tab2Mage exporter.


/Nicklas


-- 


Special Offer -- Download ArcSight Logger for FREE!
Finally, a world-class log management solution at an even better 
price-free! And you'll get a free "Love Thy Logs" t-shirt when you

download Logger. Secure your free ArcSight Logger TODAY!
http://p.sf.net/sfu/arcsisghtdev2dev
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel




--
Special Offer -- Download ArcSight Logger for FREE!
Finally, a world-class log management solution at an even better
price-free! And you'll get a free "Love Thy Logs" t-shirt when you
download Logger. Secure your free ArcSight Logger TODAY!
http://p.sf.net/sfu/arcsisghtdev2dev


___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel



--
Doing More with Less: The Next Generation Virtual Desktop
What are the key obstacles that have prevented many mid-market businesses
from deploying virtual desktops?   How do next-generation virtual desktops
provide companies an easier-to-deploy, easier-to-manage and more affordable
virtual desktop model.http://www.accelacomm.com/jaw/sfnl/114/51426474/


___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel


--
Doing More with Less: The Next Generation Virtual Desktop 
What are the key obstacles that have prevented many mid-market businesses
from deploying virtual desktops?   How do next-generation virtual desktops
provide companies an easier

[basedb-devel] MageTab export plugin

[Pawel Sztromwasser]

Hi Nicklas,

I guess the MageTab export plugin is ready for submission. I followed 
instructions for plugin developers 
(http://baseplugins.thep.lu.se/wiki/PluginSubmitReq), so all the 
documentation bits should be in place. How could I send you the bundle? 
Would you provide a repository dir to park the source?

The plugin works with BASE2, but we'll start looking at BASE3 soon, and 
as part of preparations for migration I will port the plugin to BASE3.

Pawel

--
All the data continuously generated in your IT infrastructure 
contains a definitive record of customers, application performance, 
security threats, fraudulent activity, and more. Splunk takes this 
data and makes sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-novd2d
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel

Re: [basedb-devel] MageTab export plugin

[Pawel Sztromwasser]

I think 3 will be the best. That way I could submit changes easily, and 
everyone could get access to versioned code

Pawel

On 22/11/11 14:44, Nicklas Nordborg wrote:
> On 2011-11-22 13:50, Pawel Sztromwasser wrote:
>> Hi Nicklas,
>>
>> I guess the MageTab export plugin is ready for submission. I followed
>> instructions for plugin developers
>> (http://baseplugins.thep.lu.se/wiki/PluginSubmitReq), so all the
>> documentation bits should be in place. How could I send you the bundle?
>> Would you provide a repository dir to park the source?
> There are basically three options:
>
> 1. Submit the packages binary and source files to us (me) off the
> mailing list together with the rest of the information and we will
> create a simple information page (see for example:
> http://baseplugins.thep.lu.se/wiki/net.sf.basedb.reggie) for the plug-in.
>
> 2. We can give you a login and password for our Trac environment and you
> can do everything above yourself.
>
> 3. All in 2 plus that we also setup a subversion repository for the
> source code in case you are interested in that.
>
>
>> The plugin works with BASE2, but we'll start looking at BASE3 soon, and
>> as part of preparations for migration I will port the plugin to BASE3.
> It's great to hear!
>
> /Nicklas
>
> --
> All the data continuously generated in your IT infrastructure
> contains a definitive record of customers, application performance,
> security threats, fraudulent activity, and more. Splunk takes this
> data and makes sense of it. IT sense. And common sense.
> http://p.sf.net/sfu/splunk-novd2d
> ___
> basedb-devel mailing list
> basedb-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/basedb-devel


--
All the data continuously generated in your IT infrastructure 
contains a definitive record of customers, application performance, 
security threats, fraudulent activity, and more. Splunk takes this 
data and makes sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-novd2d
___
basedb-devel mailing list
basedb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/basedb-devel