date:20190101

Re: Solr

2019-01-01 Thread Joan Moreau via dovecot

The real main differecne seems coming from "diffconfig.xml" 


When I put yours, Solr delete (!) schema.xml and create a
"manage-schema" and starts complaining about useless types (tdates,
booleans, etc..) that are not needed for Mail fileds 


When I put mine (from standard distribution of Arch), it keeps things as
they are (yeah !), does not complains about those useless types and
startup properly. 

I attach my diffconfig 


But these are the configurations that one should adjust as per his/her
own use. 


The main problem is : After some time of indexing from Dovecot, Dovecot
returns errors (invalid SID, etc...) and Solr return "out of range
indexes" errors 


On 2019-01-02 07:49, Joan Moreau wrote:

Hi 

Solr is a standard package in ArchLinux. ("pacman -S solr") . the systemd installation script is included (and it is launching /opt/solr/bin/solr.in.sh) 

Instance : sudo -u solr /opt/solr/bin/solr create -c dovecot -> this creates a separate folder with default solrconfig.xml, schema.xml, etc.. 

I made a symlink of the data folder to a second drive (ext4) much bigger 

On 2018-12-31 14:09, Daniel Miller wrote: 
On 12/29/2018 4:49 PM, Joan Moreau wrote: 
Also :


- Java is 10.0.2

Same as me. 
- If i delete schema.xml but create only managed-schema, the solr refuses to start with a java error "schema.xml missing"


Ok...so we need to do some more digging.

How did you install Solr? (I downloaded a "binary" installation and unpacked it)

How did you create the dovecot instance?  (I've provided explicit instructions 
for how I did it - did you follow those exactly or something different)?

How are you starting Solr?  (I use the provided "solr/bin/solr start" command, 
wrapped inside a systemd service).

--
Daniel




  

  
  7.5.0

  

  
  
  

  
  

  
  

  
  
  
  

  
  ${solr.data.dir:}


  
  

  
  

  
  


2000















${solr.lock.type:native}













  


  
  
  
  
  
  

  
  



  ${solr.ulog.dir:}
  ${solr.ulog.numVersionBuckets:65536}




  ${solr.autoCommit.maxTime:15000}
  false





  ${solr.autoSoftCommit.maxTime:-1}




  

  
  

  
  


1024























true





20


200




  

  


  

  



false

  


  
  








  

  
  
  


  explicit
  10
 hdr 
 xml 








  

  
  

  explicit
  json
  true

  


  
  

  explicit

  

  

  _text_

  

  
  

  true
  ignored_
  _text_

  

  

  
  

text_general





  default
  _text_
  solr.DirectSolrSpellChecker
  
  internal
  
  0.5
  
  2
  
  1
  
  5
  
  4
  
  0.01
  




  

  
  

  
  default
  on
  true
  10
  5
  5
  true
  true
  10
  5


  spellcheck

  

  
  

  
  

  true


  tvComponent

  

  

  
  

  
  

  true
  false


  terms

  


  
  

string
  

  
  

  explicit


  elevator

  

  
  

  
  
  

  100

  

  
  

  
  70
  
  0.5
  
  [-\w ,/\n\]{20,200}

  

  
  

  
  

  

  
  

  
  

  
  

  
  

  
  

  

  
  

  
  

  

  

  10
  .,!? 

  

  

  
  WORD
  
  
  en
  US

  

  

  

  
  
  
  
  
[^\w-\.]
_
  
  
  
  
  

  -MM-dd'T'HH:mm:ss.SSSZ
  -MM-dd'T'HH:mm:ss,SSSZ
  -MM-dd'T'HH:mm:ss.SSS
  -MM-dd'T'HH:mm:ss,SSS
  -MM-dd'T'HH:mm:ssZ
  -MM-dd'T'HH:mm:ss
  -MM-dd'T'HH:mmZ
  -MM-dd'T'HH:mm
  -MM-dd HH:mm:ss.SSSZ
  -MM-dd HH:mm:ss,SSSZ
  -MM-dd HH:mm:ss.SSS
  -MM-dd HH:mm:ss,SSS
  -MM-dd HH:mm:ssZ
  -MM-dd HH:mm:ss
  -MM-dd HH:mmZ
  -MM-dd HH:mm
  -MM-dd

  

  

  
  

  
  

  
  

  
  
 
 
 
 
 
 
 
 

  

text/plain; charset=UTF-8
  
 
  
  
${velocity.template.base.dir:}
${velocity.solr.resource.loader.enabled:true}
${velocity.params.resource.loader.enabled:false}
  

  
  
5

Re: Solr

2019-01-01 Thread Joan Moreau via dovecot

Refinement of the schema.xml (below) 


THis however does not solve the "no results" and "Out of range" errors
in Dovecot and Solr 




id

Re: Solr

2019-01-01 Thread Joan Moreau via dovecot

Other use case : 


I type "must" in the search filed-> I have some returns , but very not
all, for instance "solarmust" is not in the results 

If I type "solarmust" -> then I have the solarmust mail 


Honestly, this is highly unstable. Not sure whereas bugs come from Solr
or Dovecot 


Below my adjusted (corrections from the one of Daniel who is definitely
not working) schema.xml 




id





























 


On 2019-01-02 10:04, Joan Moreau wrote:

The first result show "no results" in dovecot for any search by header (I typed an email add in RoundCube search box, using Dovecot as back end, using Solr as own backend) 

So many efforts for crappy results. 

Can't we really revive Squat ? It is 2 lines of config, and no single problems 

On January 2, 2019 08:16:33 Joan Moreau via dovecot  wrote: 

and the first line of the diff is : 

< this file, see http://wiki.apache.org/solr/SolrConfigXml. 
---

this file, see http://wiki.apache.org/solr/SolrConfigXml.

38c38
< 6.4.1
---

7.5.0


So, are you running 6.4.1 or 7.5.0  

On 2019-01-02 08:12, Joan Moreau wrote: 

The real main differecne seems coming from "diffconfig.xml" 

When I put yours, Solr delete (!) schema.xml and create a "manage-schema" and starts complaining about useless types (tdates, booleans, etc..) that are not needed for Mail fileds 

When I put mine (from standard distribution of Arch), it keeps things as they are (yeah !), does not complains about those useless types and startup properly. 

I attach my diffconfig 

But these are the configurations that one should adjust as per his/her own use. 

The main problem is : After some time of indexing from Dovecot, Dovecot returns errors (invalid SID, etc...) and Solr return "out of range indexes" errors 

On 2019-01-02 07:49, Joan Moreau wrote: 

Hi 

Solr is a standard package in ArchLinux. ("pacman -S solr") . the systemd installation script is included (and it is launching /opt/solr/bin/solr.in.sh) 

Instance : sudo -u solr /opt/solr/bin/solr create -c dovecot -> this creates a separate folder with default solrconfig.xml, schema.xml, etc.. 

I made a symlink of the data folder to a second drive (ext4) much bigger 

On 2018-12-31 14:09, Daniel Miller wrote: 
On 12/29/2018 4:49 PM, Joan Moreau wrote: 
Also :


- Java is 10.0.2

Same as me. 
- If i delete schema.xml but create only managed-schema, the solr refuses to start with a java error "schema.xml missing"


Ok...so we need to do some more digging.

How did you install Solr? (I downloaded a "binary" installation and unpacked it)

How did you create the dovecot instance?  (I've provided explicit instructions 
for how I did it - did you follow those exactly or something different)?

How are you starting Solr?  (I use the provided "solr/bin/solr start" command, 
wrapped inside a systemd service).

--
Daniel

Re: Solr

2019-01-01 Thread Joan Moreau via dovecot



The first result show "no results" in dovecot for any search by header (I 
typed an email add in RoundCube search box, using Dovecot as back end, 
using Solr as own backend)


So many efforts for crappy results.

Can't we really revive Squat ? It is 2 lines of config, and no single problems


On January 2, 2019 08:16:33 Joan Moreau via dovecot  
wrote:

and the first line of the diff is :
< this file, see http://wiki.apache.org/solr/SolrConfigXml.
---

this file, see http://wiki.apache.org/solr/SolrConfigXml.

38c38
< 6.4.1
---

7.5.0


So, are you running 6.4.1 or 7.5.0 

On 2019-01-02 08:12, Joan Moreau wrote:

The real main differecne seems coming from "diffconfig.xml"

When I put yours, Solr delete (!) schema.xml and create a "manage-schema" 
and starts complaining about useless types (tdates, booleans, etc..) that 
are not needed for Mail fileds


When I put mine (from standard distribution of Arch), it keeps things as 
they are (yeah !), does not complains about those useless types and startup 
properly.


I attach my diffconfig



But these are the configurations that one should adjust as per his/her own use.

The main problem is : After some time of indexing from Dovecot, Dovecot 
returns errors (invalid SID, etc...) and Solr return "out of range indexes" 
errors







On 2019-01-02 07:49, Joan Moreau wrote:

Hi

Solr is a standard package in ArchLinux. ("pacman -S solr") . the systemd 
installation script is included (and it is launching /opt/solr/bin/solr.in.sh)


Instance : sudo -u solr /opt/solr/bin/solr create -c dovecot -> this 
creates a separate folder with default solrconfig.xml, schema.xml, etc..


I made a symlink of the data folder to a second drive (ext4) much bigger










On 2018-12-31 14:09, Daniel Miller wrote:

On 12/29/2018 4:49 PM, Joan Moreau wrote:

Also :

- Java is 10.0.2

Same as me.

- If i delete schema.xml but create only managed-schema, the solr refuses 
to start with a java error "schema.xml missing"


Ok...so we need to do some more digging.

How did you install Solr? (I downloaded a "binary" installation and 
unpacked it)


How did you create the dovecot instance?  (I've provided explicit 
instructions for how I did it - did you follow those exactly or something 
different)?


How are you starting Solr?  (I use the provided "solr/bin/solr start" 
command, wrapped inside a systemd service).


--
Daniel

Re: Solr

2019-01-01 Thread Joan Moreau via dovecot

and the first line of the diff is :

< this file, see http://wiki.apache.org/solr/SolrConfigXml.
---

this file, see http://wiki.apache.org/solr/SolrConfigXml.

38c38
< 6.4.1
---

7.5.0

So, are you running 6.4.1 or 7.5.0

On 2019-01-02 08:12, Joan Moreau wrote:

The real main differecne seems coming from "diffconfig.xml"

When I put yours, Solr delete (!) schema.xml and create a "manage-schema" and starts complaining about useless types (tdates, booleans, etc..) that are not needed for Mail fileds

When I put mine (from standard distribution of Arch), it keeps things as they are (yeah !), does not complains about those useless types and startup properly.

I attach my diffconfig

But these are the configurations that one should adjust as per his/her own use.

The main problem is : After some time of indexing from Dovecot, Dovecot returns errors (invalid SID, etc...) and Solr return "out of range indexes" errors

On 2019-01-02 07:49, Joan Moreau wrote:

Solr is a standard package in ArchLinux. ("pacman -S solr") . the systemd installation script is included (and it is launching /opt/solr/bin/solr.in.sh)

Instance : sudo -u solr /opt/solr/bin/solr create -c dovecot -> this creates a separate folder with default solrconfig.xml, schema.xml, etc..

I made a symlink of the data folder to a second drive (ext4) much bigger

On 2018-12-31 14:09, Daniel Miller wrote:
On 12/29/2018 4:49 PM, Joan Moreau wrote:
Also :

- Java is 10.0.2

Same as me.
- If i delete schema.xml but create only managed-schema, the solr refuses to start with a java error "schema.xml missing"

Ok...so we need to do some more digging.

How did you install Solr? (I downloaded a "binary" installation and unpacked it)

How did you create the dovecot instance? (I've provided explicit instructions
for how I did it - did you follow those exactly or something different)?

How are you starting Solr? (I use the provided "solr/bin/solr start" command,
wrapped inside a systemd service).

--
Daniel

Re: Solr

2019-01-01 Thread Joan Moreau via dovecot

Hi 


Solr is a standard package in ArchLinux. ("pacman -S solr") . the
systemd installation script is included (and it is launching
/opt/solr/bin/solr.in.sh) 


Instance : sudo -u solr /opt/solr/bin/solr create -c dovecot -> this
creates a separate folder with default solrconfig.xml, schema.xml, etc..


I made a symlink of the data folder to a second drive (ext4) much bigger


On 2018-12-31 14:09, Daniel Miller wrote:

On 12/29/2018 4:49 PM, Joan Moreau wrote: 


Also :

- Java is 10.0.2
Same as me. 


- If i delete schema.xml but create only managed-schema, the solr refuses to start with a 
java error "schema.xml missing"

Ok...so we need to do some more digging.

How did you install Solr? (I downloaded a "binary" installation and unpacked it)

How did you create the dovecot instance?  (I've provided explicit instructions 
for how I did it - did you follow those exactly or something different)?

How are you starting Solr?  (I use the provided "solr/bin/solr start" command, 
wrapped inside a systemd service).

--
Daniel

Re: Several problems on Solaris10

2019-01-01 Thread Sami Ketola

> On 31 Dec 2018, at 19.28, Pierluigi Frullani  
> wrote:
> 
> 
> Answer inline.
> 
> On Sun, Dec 30, 2018 at 12:59 PM James  > wrote:
> On 29/12/2018 13:49, Pierluigi Frullani wrote:
> 
> > My version is 2.2.13 ( it was the last one, at the time of the first
> > server setup ).
> 
> 2.2.13 is from around May 2014.  It worked but I can't see why you 
> wouldn't switch to the latest 2.3.4. (You might be seeing what I can't 
> and your question hasn't explained.)
> 
> That's the date for installation. I was using the courier-imap and switched 
> to dovecot. 
> Not changed since then. 
>  
> > I have seen that ( it seems ) the new solaris don't honour the
> > LD_LIBRARY_PATH.
> 
> I'm sure it does but you shouldn't need it anyway.
> 
> Believe me, it doesn't :( 

Believe me it does. I used to work for Sun Microsystems for 14 years in Solaris 
support and sustaining and I can guarantee you that it does.

You problem is that Solaris has concept of Secure Runtime Linker, and for 
trusted applications most of LD_CONFIG and LD_LIBRARY_PATH is ignored for 
security reasons.

For secure applications LD_LIBRARY_PATH components are ignored for non-secure 
directories.

Your dovecot is probably setuid or setgid and considered as secure application 
and secure runtime linker rules are triggered for it. Then /usr/local is 
completely ignored from LD_LIBRARY_PATH.

Sami

Re: dsync connection issue

2019-01-01 Thread subscription1


Thanks for you suggestion, but I couldn't quite get it to work.

I ended up rsyncing the mail folders (there were only a few email 
accounts) a temporary folder on the new server and then used dsync to 
restore the mailfolders.


Thanks again for your help

Leo


On 28/12/18 10:07 am, Sami Ketola wrote:



On 27 Dec 2018, at 19.13, Subscription  wrote:

but when I try to do a backup with the following command from the old to the 
new site

sudo doveadm -D -o imapc_user=user1@oldserver  -o imapc_password=pw-oldserver 
backup -R -u user1@newserver imapc:



Since both of your servers are running dovecot it would be probably better to 
use native doveadm protocol (over ssh pipe if required) to do the migration 
instead of using imapc.

something like:

doveadm backup -u user1@newserver -R ssh oldserver sudo /usr/bin/doveadm 
dsync-server -u user1@oldserver

allowing sudo and ssh login with keys.

Sami

Re: Several problems on Solaris10

2019-01-01 Thread James

On 31/12/2018 17:28, Pierluigi Frullani wrote:

> My version is 2.2.13 ( it was the last one, at the time of the first
> server setup ).

2.2.13 is from around May 2014.  It worked but I can't see why you
wouldn't switch to the latest 2.3.4. (You might be seeing what I can't
and your question hasn't explained.)

That's the date for installation. I was using the courier-imap and
switched to dovecot.
Not changed since then.

So use the latest version.

> I have seen that ( it seems ) the new solaris don't honour the
> LD_LIBRARY_PATH.

I'm sure it does but you shouldn't need it anyway.

Believe me, it doesn't :(

Test:

$ cat > sub.c << EOF
void sub() {}
EOF
$ cat > main.c << EOF
void sub();
int main() { sub(); }
EOF
$ cc -G -o libsub.so sub.c
-- link with no run path in the executable, fails with no other help
$ cc main.c -L. -lsub
$ ./a.out
ld.so.1: a.out: fatal: libsub.so: open failed: No such file or directory
zsh: killed ./a.out
-- succeeds with LD_LIBRARY_PATH set
$ LD_LIBRARY_PATH=. ./a.out
-- link with a run path, succeeds on its own
$ cc main.c -L. -R. -lsub
$ ./a.out
$ dump -Lv a.out | grep RUNPATH
[5] RUNPATH .

-- to see the logic when finding a library use "ldd -s ..."
$ LD_LIBRARY_PATH=. ldd -s ./a.out

   find object=libsub.so; required by a.out
search path=.  (LD_LIBRARY_PATH)
trying path=./libsub.so
libsub.so =>  ./libsub.so

-- clean up:
$ rm sub.c main.c libsub.so a.out

The problem ( as usual ) arise with the openssl libs that solaris ships,

Build your own openssl.

1. Do not put your files in /use/local.  You will clash with someone
else thinking it is the place to put personal stuff.  man filesystem:
"/opt  Root of a subtree for add-on application packages."

 Being that I'm the only one administering this machine I'm sure it
would not  ;)

You will appreciate the separation of system and add-ons.  Solaris 10 
sparse zones can make /usr read only.  It really is the "right" way.

Would you ( or could ) show me where to replace -R and -L on Makefile(s)
to fix the runpath trouble ?

I've tried the following:
CPPFLAGS="-I/usr/local/clucene/include -I/usr/local/libtextcat/include
-I/usr/local/openssl-1.0.1e/include" LDFLAGS="-L/usr/local/clucene/lib
-L/usr/local/libtextcat/lib
-L/usr/local/openssl-1.0.1e/lib -R/usr/local/openssl-1.0.1e/lib "
./configure --prefix=/usr/local/dovecot --with-ssl=yes --with-stemmer
--with-lucene --with-zlib
and it seems to work:

root@puma dump -Lv ./src/imap-login/.libs/imap-login | grep RUN
[14]RUNPATH

/usr/local/dovecot/lib/dovecot:/usr/local/lib:/usr/local/openssl-1.0.1e/lib:/usr/ccs/lib:/lib:/usr/lib:/usr/sfw/lib

It would be easier if you put openssl and your other software under a 
common prefix, $PREFIX/lib, (not a subdirectory).  The SONAME should 
sort out versions.

If you are doing a simple build you will get away with the standard 
handling of -L and -R.

$ PREFIX=/opt/PREFIX
$ export LDFLAGS="-R/opt/FIRST/lib:${PREFIX}/lib"
$ ./configure --prefix=${PREFIX} ...

Gives RPATH: /opt/PREFIX/lib/dovecot:/opt/FIRST/lib:/opt/PREFIX/lib
which I think will work for you.

Problems with libtool are when we require:
+ /opt/PREFIX/lib/$ISALIST at the front before /opt/PREFIX/lib/
+ a non standard lib is needed before the normal one instead of libtool 
using /opt/PREFIX/lib as the first

In general it is easier to let libtool think it is helping but ignore 
it.  Create a wrapper script around cc to rewrite the args back to what 
they were before configure+libtool rearranged, in this case drop the -L 
and -R given and substitute the values in $LDFLAGS.

More reading:
https://blogs.oracle.com/solaris/ldlibrarypath-just-say-no-v2
https://docs.oracle.com/cd/E86824_01/html/E54763/ld.so.1-1.html
https://docs.oracle.com/cd/E19683-01/816-1386/chapter3-13312/
Avoid crle, you don't need it and it affects the whole system.  Hint, 
you might need this to unlock your machine:

# LD_LIBRARY_PATH=/usr/lib rm /var/ld/ld.config

Re: Solr

Re: Solr

Re: Solr

Re: Solr

Re: Solr

Re: Solr

Re: Several problems on Solaris10

Re: dsync connection issue

Re: Several problems on Solaris10

9 matches

Site Navigation

Mail list logo

Footer information