Re: How to retrieve nested documents (parents and their children together) ?

[TK Solr]

Ah, that's what _root_ is for ! I was wondering.

Thank you!


On 7/25/18 2:36 PM, Mikhail Khludnev wrote:

_root_:parent-id

чт, 26 июля 2018, 1:33 TK Solr :


The child doc transformer worked great. Thank you.

In my experiment, posting 'parent-id' to the
update
end point only deleted the parent doc. Do I insert a complex join query
from id
to _version_ and delete all the docs of the matching _version_ ?


On 7/24/18 9:27 PM, TK Solr wrote:

Thank you. I'll try the child doc transformer.

On a related question, if I delete a parent document, will its children

be

deleted also? Or do I have to have a parent_id field in each child so

that the

child docs can be deleted?


On 7/22/18 10:05 AM, Mikhail Khludnev wrote:

Hello,
Check [child]


https://lucene.apache.org/solr/guide/7_4/transforming-result-documents.html#child-childdoctransformerfactory

or [subquery].
Although, it's worth to put reference to it somewhere in blockjoin
qparsers.
Documentation patches are welcome.


On Sun, Jul 22, 2018 at 10:25 AM TK Solr  wrote:


https://lucene.apache.org/solr/guide/7_4/other-parsers.html#block-join-parent-query-parser


talks about {!parent which=}


child
docs>, which returns parent docs only, and
{!child of=} 
docs>,

which
returns child docs only.

Is there a way to retrieve the matched documents in the original,

nested

form?
Using the sample document, is there way to get:


 1
 Solr has block join support
 parentDocument
 
 2
 SolrCloud supports it too!
 


rather than just the parent or the child docs?

RE: sharding and placement of replicas

[Oakley, Craig (NIH/NLM/NCBI) [C]]

I just now tried it with Solr7.4 and am getting the same symptoms as I describe 
below.

The symptoms I describe are quite different from my impression of Shawn 
Heisey's impression of my symptoms, so I will describe my symptoms again.

Let us assume that we start with a SolrCloud of two nodes: one at 
hostname1: and the other at hostname2:

Let us assume that we have a one-shard collection with two replicas. One of the 
replicas is on the node at hostname1: (with the core col_shard1_replica_n1) 
and the other on the node at hostname2: (with the core 
col_shard1_replica_n3)

Then I run SPLITSHARD

I end up with four cores instead of two, as expected. The problem is that three 
of the four cores (col_shard1_0_replica_n5, col_shard1_0_replica0 and 
col_shard1_1_replica_n6) are *all on hostname1*. Only col_shard1_1_replica0 was 
placed on hostname2.

Prior to the SPLITSHARD, if hostname1 becomes temporarily unavailable, the 
SolrCloud can still be used: hostname2 has all the data.

After the SPLITSHARD, if hostname1 becomes temporarily unavailable, the 
SolrCloud does not have any access to the data in shard1_0

Granted, I could add a replica of shard1_0 onto hostname2, and I could then 
drop one of the extraneous shard1_0 replicas which are on hostname1: but I 
don't see the logic in requiring such additional steps every time.



My question is: How can I tell Solr "avoid putting two replicas of the same 
shard on the same node"?



-Original Message-
From: Shawn Heisey [mailto:apa...@elyograg.org] 
Sent: Tuesday, June 19, 2018 2:20 PM
To: solr-user@lucene.apache.org
Subject: Re: sharding and placement of replicas

On 6/15/2018 11:08 AM, Oakley, Craig (NIH/NLM/NCBI) [C] wrote:
> If I start with a collection X on two nodes with one shard and two replicas 
> (for redundancy, in case a node goes down): a node on host1 has 
> X_shard1_replica1 and a node on host2 has X_shard1_replica2: when I try 
> SPLITSHARD, I generally get X_shard1_0_replica1, X_shard1_1_replica1 and 
> X_shard1_0_replica0 all on the node on host1 with X_shard1_1_replica0 sitting 
> alone on the node on host2. If host1 were to go down at this point, shard1_0 
> would be unavailable.

https://lucene.apache.org/solr/guide/6_6/collections-api.html#CollectionsAPI-splitshard

That documentation says "The new shards will have as many replicas as
the original shard."  That tells me that what you're seeing is not
matching the *intent* of the SPLITSHARD feature.  The fact that you get
*one* of the new shards but not the other is suspicious.  I'm wondering
if maybe Solr tried to create it but had a problem doing so.  Can you
check for errors in the solr logfile on host2?

If there's nothing about your environment that would cause a failure to
create the replica, then it might be a bug.

> Is there a way either of specifying placement or of giving hints that 
> replicas ought to be separated?

It shouldn't be necessary to give Solr any parameters for that.  All
nodes where the shard exists should get copies of the new shards when
you split it.

> I am currently running Solr6.6.0, if that is relevant.

If this is a provable and reproducible bug, and it's still a problem in
the current stable branch (next release from that will be 7.4.0), then
it will definitely be fixed.  If it's only a problem in 6.x, then I
can't guarantee that it will be fixed.  That's because the 6.x line is
in maintenance mode, which means that there's a very high bar for
changes.  In most cases, only changes that meet one of these criteria
are made in maintenance mode:

 * Fixes a security bug.
 * Fixes a MAJOR bug with no workaround.
 * Fix is a very trivial code change and not likely to introduce new bugs.

Of those criteria, generally only the first two are likely to prompt an
actual new software release.  If enough changes of the third type
accumulate, that might prompt a new release.

My personal opinion:  If this is a general problem in 6.x, it should be
fixed there.  Because there is a workaround, it would not be cause for
an immediate new release.

Thanks,
Shawn

Re: How to retrieve nested documents (parents and their children together) ?

[Mikhail Khludnev]

_root_:parent-id

чт, 26 июля 2018, 1:33 TK Solr :

> The child doc transformer worked great. Thank you.
>
> In my experiment, posting 'parent-id' to the
> update
> end point only deleted the parent doc. Do I insert a complex join query
> from id
> to _version_ and delete all the docs of the matching _version_ ?
>
>
> On 7/24/18 9:27 PM, TK Solr wrote:
> > Thank you. I'll try the child doc transformer.
> >
> > On a related question, if I delete a parent document, will its children
> be
> > deleted also? Or do I have to have a parent_id field in each child so
> that the
> > child docs can be deleted?
> >
> >
> > On 7/22/18 10:05 AM, Mikhail Khludnev wrote:
> >> Hello,
> >> Check [child]
> >>
> https://lucene.apache.org/solr/guide/7_4/transforming-result-documents.html#child-childdoctransformerfactory
> >>
> >> or [subquery].
> >> Although, it's worth to put reference to it somewhere in blockjoin
> >> qparsers.
> >> Documentation patches are welcome.
> >>
> >>
> >> On Sun, Jul 22, 2018 at 10:25 AM TK Solr  wrote:
> >>
> >>>
> https://lucene.apache.org/solr/guide/7_4/other-parsers.html#block-join-parent-query-parser
> >>>
> >>>
> >>> talks about {!parent which=}
>  >>> child
> >>> docs>, which returns parent docs only, and
> >>> {!child of=}  docs>,
> >>> which
> >>> returns child docs only.
> >>>
> >>> Is there a way to retrieve the matched documents in the original,
> nested
> >>> form?
> >>> Using the sample document, is there way to get:
> >>>
> >>> 
> >>> 1
> >>> Solr has block join support
> >>> parentDocument
> >>> 
> >>> 2
> >>> SolrCloud supports it too!
> >>> 
> >>> 
> >>>
> >>> rather than just the parent or the child docs?
> >>>
> >>>
> >>>
> >
>
>

Re: How to retrieve nested documents (parents and their children together) ?

[TK Solr]

The child doc transformer worked great. Thank you.

In my experiment, posting 'parent-id' to the update 
end point only deleted the parent doc. Do I insert a complex join query from id 
to _version_ and delete all the docs of the matching _version_ ?



On 7/24/18 9:27 PM, TK Solr wrote:

Thank you. I'll try the child doc transformer.

On a related question, if I delete a parent document, will its children be 
deleted also? Or do I have to have a parent_id field in each child so that the 
child docs can be deleted?



On 7/22/18 10:05 AM, Mikhail Khludnev wrote:

Hello,
Check [child]
https://lucene.apache.org/solr/guide/7_4/transforming-result-documents.html#child-childdoctransformerfactory 


or [subquery].
Although, it's worth to put reference to it somewhere in blockjoin
qparsers.
Documentation patches are welcome.


On Sun, Jul 22, 2018 at 10:25 AM TK Solr  wrote:

https://lucene.apache.org/solr/guide/7_4/other-parsers.html#block-join-parent-query-parser 



talks about {!parent which=} , which returns parent docs only, and
{!child of=} ,
which
returns child docs only.

Is there a way to retrieve the matched documents in the original, nested
form?
Using the sample document, is there way to get:


1
Solr has block join support
parentDocument

2
SolrCloud supports it too!



rather than just the parent or the child docs?

Re: Upgrading SOLR (not clustered)

[Shawn Heisey]

On 7/25/2018 10:38 AM, Staley, Phil R - DCF wrote:
> What are the steps for upgrading a non-clustered SOLR version?  Here's what I 
> thought should work:

That depends on how you did the initial install.

If you used the service installer script that's included with Solr to do
the initial install:  Stop the existing service, then run that script
again from the new version with the same commandline options, adding the
-f option to force the install even though the service already exists.

If you did not use the service installer script, then you would need to
discuss this with whoever installed Solr on your system and ask them
what needs to happen.

Thanks,
Shawn

RE: Upgrading SOLR (not clustered)

[Staley, Phil R - DCF]

Christopher,

I'll test it tomorrow and let you know!  Thanks again!

Phil

-Original Message-
From: Christopher Schultz  
Sent: Wednesday, July 25, 2018 3:40 PM
To: solr-user@lucene.apache.org
Subject: Re: Upgrading SOLR (not clustered)

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Phil,

On 7/25/18 4:38 PM, Staley, Phil R - DCF wrote:
> Christopher,
> 
> Testing an upgrade from version 7.2.1 to 7.4.0 on SUSE Linux 12
> 
> From the /etc/init.d/solr file?
> 
> SOLR_INSTALL_DIR="/opt/solr"
> 
> From the /etc/default/solr.in.sh file? (and these are my data 
> and/indexing core locations
> 
> SOLR_PID_DIR="/var/solr" SOLR_HOME="/var/solr/data" 
> LOG4J_PROPS="/var/solr/log4j.properties" 
> SOLR_LOGS_DIR="/var/solr/logs" SOLR_PORT="8983"

I would expect your process to work. Did it?

- -chris

> -Original Message- From: Christopher Schultz 
>  Sent: Wednesday, July 25, 2018 3:23 PM 
> To: solr-user@lucene.apache.org Subject: Re: Upgrading SOLR (not
> clustered)
> 
> Phil,
> 
> On 7/25/18 12:38 PM, Staley, Phil R - DCF wrote:
>> What are the steps for upgrading a non-clustered SOLR version? 
>> Here's what I thought should work:
> 
> 
> 
>> 1.  Open a bash window and ssh login to desired server with 
>> your Linux admin credentials
> 
>> 2.  Change directories:  cd /opt
> 
>> 3.  Download the latest Linux/OSX version direct to server: 
>> sudo wget
>> http://secure-web.cisco.com/1m3u-zHHzT7PG9DMKzh18vroXutH5_t3ai-gl70-Y
x
>>
>> 
ZzDhjDAlBf5297ajnpoZ0PptxeKUldcLaRREkQF6UwpkpjGJvBhFyMYKEleNgOv2KiAXuZ
>> Qw4HjRFeUCRluU7gPGPLiYF7_aaBeutMU6Kr0LxiOTpUTv2z9qZiIQYU2M-YN1lNy-acH
K
>>
>> 
rY5ZfGuMw0fSBmdRa9PSzP9ZUj1qGEY94PCLQXxVNkYx_u4CXx-TaA0Fo-aKqvl2x9ejFB
>> uVt2jF1e8zf3i9E367USmyBdbEQ/http%3A%2F%2Fapache.claz.org%2Flucene%2Fs
o
>>
>> 
lr%2Fx.x.x%2Fsolr-x.x.x.tgz (replace x.x.x with the latest version
>> number)
> 
>> a.  Additional download mirror servers are available @ 
>> http://secure-web.cisco.com/1Mafx4QIn9BgkDPtPKbw6pF3EugYCWQHwgSifrgOr
_
>>
>> 
5l1VTprI53j3huCKwyUxst3FIbRgyqah-96wu9NC3fcClwiEqV6ww9g796bhMz6OQDxYb17q
2WPVzIhkB8ozsOw6CJoJKu9xvQuPlab4QkH5DAqOfWBFbtBavS2s-eRdGexv327ATH5BZZP0
snS49XnaiUJjYgYPf4ILzXPp5DLQmbLYSxuHlIp0UP3J_4b_gxq9JEB7_E6dcDiq1hrEN_wW
4n8MvuaRQ3PqPgO_ucjaFoYOL5ZoFSM-svWmZcoD1E/http%3A%2F%2Fhttp://secure-web.cisco.com/1_3-KGt2Sn4UmSTkgx1WPIhCp5xgWDNCQY7LeZOVmYKZ6dv3qb9V02gi_FBDAGKjJNdRmwW3BuV-4Tv_Ad2-DoRk_b_ZP3CDKaKs-neB7q5KLoasrHou06_xaeJiFmgJvh1amNPBIRSOaJGpNhowOZdlvFR98HwAEyHpk_suw0TA5rzGiX_tocaNnEBX1_W8dqHy0W6Ub1L5OmvDZ9D5jiOFuvRCyc1XSB_eZMk_LvlOBrOBI-d6B5pRwp97jnCQOZVs0bVhtbLITzrJ1gePg1w/http%3A%2F%2Fwww.apache.org%2
Fdyn%2Fcloser.lua%2Flucene%2Fsolr%2F7.3.1
if the
http://secure-web.cisco.com/1mEnNfQ3nil_pfEFLpG5wMsugkz7vhDU0czyVu2MH7pe
J0aomngulTED-W-zTbK-ywavVjNDYF95PcgmerYe2J4MSIrpaWALkysbyL5rYu4BVb9VZXQg
GuPso0kODrtnA_F4Igw1cE2qjoeoRLk6Pff9Or3lnLbyVCuHjIfECo_JOGvuw91ulYljWU3e
113vxCGB8x9ogaAPR06C1qoqDhu4_b1j2tXqAfJb9iiJKLvOHB-RsxGLu1jxdk4_enK1CVE5
5nj2gyHh2QgAgqVmaBA/http%3A%2F%2Fapache.claz.org
is site is slow.
> 
>> 4.  Login as root user:  sudo -i  and enter you admin
>> password
> 
>> 5.  Unzip the .tgz file:  tar zxf solr-x.x.x.tgz
> 
>> 6.  Change directories:  cd /
> 
>> 7.  Stop SOLR service: service solr stop
> 
>> 8.  Confirm that SOLR is stopped:   service solr status
> 
>> 9.  Change directories to your user home directory:  cd 
>> /home/myadminlogonid
> 
>> 10.  Create new solr symbolic link in your user home folder that 
>> points to new SOLR version:   ln -s /opt/solr-x.x.x  solr
> 
>> 11.  Move/replace current symbolic link:  mv solr /opt
> 
> What version are you going from/to?
> 
> What OS is this?
> 
> Do you have an /etc/init.d/solr file? If so, where does 
> SOLR_INSTALL_DIR point?
> 
> Do you have an /etc/default/solr.in.sh file? If it points to all of 
> your data-locations, then you should be okay.
> 
> -chris
> 
-BEGIN PGP SIGNATURE-
Comment: GPGTools - 
http://secure-web.cisco.com/173FZxjmt9_NPA_d3aQA0sMyBodKy_XM15iXC0a-iO28kcwzZCTTOFyLhMLgC_USkWmNEg4qHsCJnI5i674g1pEn07OmUibRwCXTa_5FTKZXJMJBc_UV0hSpGzjiNbk5N4_km22Fm67thxzCxj5wzpIf1mW5QqCSAjhvQzXKE9DwfUDiFH6JZihWUNUUEWLz1Emw2gAuexkAdpsc08lUeOdFh5L6IsEKfNPONgVwyvDtDvO3dT6ndZor51rkqCBPg_6OKsu1IkyWI4xwA7FFCdg/http%3A%2F%2Fgpgtools.org
Comment: Using GnuPG with Thunderbird - 
http://secure-web.cisco.com/1JdkQBqCwLeRWKZPtEnwgpKNvG30t7ydOnUWSZW0Ec_epZzjMF_CXXzKlqop8BTFccjn01t7BQCiNZBxWygHNfSRcyvqPj_-PrHELWDLCL1HhS2p_wEA7Up5eKmaTH6j56Sk9BTs9pq_ytnY6dZrFM9bg2onnjjM2HDGIpenq6IrI22VwAz19EH2H39hDnjpBf78UsbZffNkyR3DP2j8dIGPRoRc8A1_X3OZwunO7yTIZbCXe2xZRsY_KE4BsfxP8bt2kQ_ub2BR7zXckm8kA3A/http%3A%2F%2Fwww.enigmail.net%2F

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAltY4DkACgkQHPApP6U8
pFgShhAAysteX3nkqj/LwLZ8Q5uiLVo/OCNz0kk/Q4VOFM7sgnkydLTCdfRD6hBD
FtqoJue/ayXvGPoRC46R0+LGBYyRQwPajFPBe0S5Ay4dCIe4de1lMjmsc6zfgMpb
tu15YduBFT2O6vdKUyHISHIANqvnaAZnRvfp6P3rlgN1ADL0Ui8y2Kdrx+iHszi2
mIc3fuJY1t8LVpAjMH5Vu8ZD8LuBkH3DOHPLErPoJPkOF+0CaiLrR7DBStrKdsOF

Re: Upgrading SOLR (not clustered)

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Phil,

On 7/25/18 4:38 PM, Staley, Phil R - DCF wrote:
> Christopher,
> 
> Testing an upgrade from version 7.2.1 to 7.4.0 on SUSE Linux 12
> 
> From the /etc/init.d/solr file?
> 
> SOLR_INSTALL_DIR="/opt/solr"
> 
> From the /etc/default/solr.in.sh file? (and these are my data
> and/indexing core locations
> 
> SOLR_PID_DIR="/var/solr" SOLR_HOME="/var/solr/data" 
> LOG4J_PROPS="/var/solr/log4j.properties" 
> SOLR_LOGS_DIR="/var/solr/logs" SOLR_PORT="8983"

I would expect your process to work. Did it?

- -chris

> -Original Message- From: Christopher Schultz
>  Sent: Wednesday, July 25, 2018 3:23
> PM To: solr-user@lucene.apache.org Subject: Re: Upgrading SOLR (not
> clustered)
> 
> Phil,
> 
> On 7/25/18 12:38 PM, Staley, Phil R - DCF wrote:
>> What are the steps for upgrading a non-clustered SOLR version? 
>> Here's what I thought should work:
> 
> 
> 
>> 1.  Open a bash window and ssh login to desired server with 
>> your Linux admin credentials
> 
>> 2.  Change directories:  cd /opt
> 
>> 3.  Download the latest Linux/OSX version direct to server: 
>> sudo wget 
>> http://secure-web.cisco.com/1m3u-zHHzT7PG9DMKzh18vroXutH5_t3ai-gl70-Y
x
>>
>> 
ZzDhjDAlBf5297ajnpoZ0PptxeKUldcLaRREkQF6UwpkpjGJvBhFyMYKEleNgOv2KiAXuZ
>> Qw4HjRFeUCRluU7gPGPLiYF7_aaBeutMU6Kr0LxiOTpUTv2z9qZiIQYU2M-YN1lNy-acH
K
>>
>> 
rY5ZfGuMw0fSBmdRa9PSzP9ZUj1qGEY94PCLQXxVNkYx_u4CXx-TaA0Fo-aKqvl2x9ejFB
>> uVt2jF1e8zf3i9E367USmyBdbEQ/http%3A%2F%2Fapache.claz.org%2Flucene%2Fs
o
>>
>> 
lr%2Fx.x.x%2Fsolr-x.x.x.tgz (replace x.x.x with the latest version
>> number)
> 
>> a.  Additional download mirror servers are available @ 
>> http://secure-web.cisco.com/1Mafx4QIn9BgkDPtPKbw6pF3EugYCWQHwgSifrgOr
_
>>
>> 
5l1VTprI53j3huCKwyUxst3FIbRgyqah-96wu9NC3fcClwiEqV6ww9g796bhMz6OQDxYb17q
2WPVzIhkB8ozsOw6CJoJKu9xvQuPlab4QkH5DAqOfWBFbtBavS2s-eRdGexv327ATH5BZZP0
snS49XnaiUJjYgYPf4ILzXPp5DLQmbLYSxuHlIp0UP3J_4b_gxq9JEB7_E6dcDiq1hrEN_wW
4n8MvuaRQ3PqPgO_ucjaFoYOL5ZoFSM-svWmZcoD1E/http%3A%2F%2Fwww.apache.org%2
Fdyn%2Fcloser.lua%2Flucene%2Fsolr%2F7.3.1
if the
http://secure-web.cisco.com/1mEnNfQ3nil_pfEFLpG5wMsugkz7vhDU0czyVu2MH7pe
J0aomngulTED-W-zTbK-ywavVjNDYF95PcgmerYe2J4MSIrpaWALkysbyL5rYu4BVb9VZXQg
GuPso0kODrtnA_F4Igw1cE2qjoeoRLk6Pff9Or3lnLbyVCuHjIfECo_JOGvuw91ulYljWU3e
113vxCGB8x9ogaAPR06C1qoqDhu4_b1j2tXqAfJb9iiJKLvOHB-RsxGLu1jxdk4_enK1CVE5
5nj2gyHh2QgAgqVmaBA/http%3A%2F%2Fapache.claz.org
is site is slow.
> 
>> 4.  Login as root user:  sudo -i  and enter you admin
>> password
> 
>> 5.  Unzip the .tgz file:  tar zxf solr-x.x.x.tgz
> 
>> 6.  Change directories:  cd /
> 
>> 7.  Stop SOLR service: service solr stop
> 
>> 8.  Confirm that SOLR is stopped:   service solr status
> 
>> 9.  Change directories to your user home directory:  cd 
>> /home/myadminlogonid
> 
>> 10.  Create new solr symbolic link in your user home folder that 
>> points to new SOLR version:   ln -s /opt/solr-x.x.x  solr
> 
>> 11.  Move/replace current symbolic link:  mv solr /opt
> 
> What version are you going from/to?
> 
> What OS is this?
> 
> Do you have an /etc/init.d/solr file? If so, where does
> SOLR_INSTALL_DIR point?
> 
> Do you have an /etc/default/solr.in.sh file? If it points to all of
> your data-locations, then you should be okay.
> 
> -chris
> 
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAltY4DkACgkQHPApP6U8
pFgShhAAysteX3nkqj/LwLZ8Q5uiLVo/OCNz0kk/Q4VOFM7sgnkydLTCdfRD6hBD
FtqoJue/ayXvGPoRC46R0+LGBYyRQwPajFPBe0S5Ay4dCIe4de1lMjmsc6zfgMpb
tu15YduBFT2O6vdKUyHISHIANqvnaAZnRvfp6P3rlgN1ADL0Ui8y2Kdrx+iHszi2
mIc3fuJY1t8LVpAjMH5Vu8ZD8LuBkH3DOHPLErPoJPkOF+0CaiLrR7DBStrKdsOF
5k5Jlgv/oYueCS0X1SAtc1W7t/vqUHgqnqNqnNaInGDOTblW/FTVOxRt8BF90sgS
UPBy8K2EyhS/rZqEBEp7sLndzNhtGHmhCNOIptHsaixt+zh7bepdXEvSNThDLHs2
Pg+NTyGGsr5JzdzkjZZwV4Re5jPY5vNL9LTOqIr/x3rQiSo04M5u1rCuHKRFP3Dw
ZFxamOXPDSo1Oo32042/yAwgpI+En1YVEEvXwNhudCeG1mEAxW+UejdhEAkxrpFt
+BqDo+XWh9jNyqBFUMtMjjzbF3SWfjeDtMfFPCy6IamUqCWJoXk6uhGF6RH1GHqL
QBJP3NSxMMU5X68fVG/dr2DKiprZmuJuuNup3qZJwGWZZAHV3/Z+3gmTLL/ISqDH
RZnm2lc1aIgIFdD7s1cFxHueD3j7shxbwmFXkn5Jd+RUjKf7nDU=
=enmW
-END PGP SIGNATURE-

RE: Upgrading SOLR (not clustered)

[Staley, Phil R - DCF]

Christopher, 

Testing an upgrade from version 7.2.1 to 7.4.0 on SUSE Linux 12

From the /etc/init.d/solr file? 

SOLR_INSTALL_DIR="/opt/solr"

From the /etc/default/solr.in.sh file? (and these are my data and/indexing core 
locations

SOLR_PID_DIR="/var/solr"
SOLR_HOME="/var/solr/data"
LOG4J_PROPS="/var/solr/log4j.properties"
SOLR_LOGS_DIR="/var/solr/logs"
SOLR_PORT="8983"

Thanks,

Phil

-Original Message-
From: Christopher Schultz  
Sent: Wednesday, July 25, 2018 3:23 PM
To: solr-user@lucene.apache.org
Subject: Re: Upgrading SOLR (not clustered)

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Phil,

On 7/25/18 12:38 PM, Staley, Phil R - DCF wrote:
> What are the steps for upgrading a non-clustered SOLR version?
> Here's what I thought should work:
> 
> 
> 
> 1.  Open a bash window and ssh login to desired server with
> your Linux admin credentials
> 
> 2.  Change directories:  cd /opt
> 
> 3.  Download the latest Linux/OSX version direct to server: 
> sudo wget 
> http://secure-web.cisco.com/1m3u-zHHzT7PG9DMKzh18vroXutH5_t3ai-gl70-Yx
> ZzDhjDAlBf5297ajnpoZ0PptxeKUldcLaRREkQF6UwpkpjGJvBhFyMYKEleNgOv2KiAXuZ
> Qw4HjRFeUCRluU7gPGPLiYF7_aaBeutMU6Kr0LxiOTpUTv2z9qZiIQYU2M-YN1lNy-acHK
> rY5ZfGuMw0fSBmdRa9PSzP9ZUj1qGEY94PCLQXxVNkYx_u4CXx-TaA0Fo-aKqvl2x9ejFB
> uVt2jF1e8zf3i9E367USmyBdbEQ/http%3A%2F%2Fapache.claz.org%2Flucene%2Fso
> lr%2Fx.x.x%2Fsolr-x.x.x.tgz (replace x.x.x with the latest version 
> number)
> 
> a.  Additional download mirror servers are available @
> http://secure-web.cisco.com/1Mafx4QIn9BgkDPtPKbw6pF3EugYCWQHwgSifrgOr_
> 5l1VTprI53j3huCKwyUxst3FIbRgyqah-96wu9NC3fcClwiEqV6ww9g796bhMz6OQDxYb17q2WPVzIhkB8ozsOw6CJoJKu9xvQuPlab4QkH5DAqOfWBFbtBavS2s-eRdGexv327ATH5BZZP0snS49XnaiUJjYgYPf4ILzXPp5DLQmbLYSxuHlIp0UP3J_4b_gxq9JEB7_E6dcDiq1hrEN_wW4n8MvuaRQ3PqPgO_ucjaFoYOL5ZoFSM-svWmZcoD1E/http%3A%2F%2Fwww.apache.org%2Fdyn%2Fcloser.lua%2Flucene%2Fsolr%2F7.3.1
>  if the 
> http://secure-web.cisco.com/1mEnNfQ3nil_pfEFLpG5wMsugkz7vhDU0czyVu2MH7peJ0aomngulTED-W-zTbK-ywavVjNDYF95PcgmerYe2J4MSIrpaWALkysbyL5rYu4BVb9VZXQgGuPso0kODrtnA_F4Igw1cE2qjoeoRLk6Pff9Or3lnLbyVCuHjIfECo_JOGvuw91ulYljWU3e113vxCGB8x9ogaAPR06C1qoqDhu4_b1j2tXqAfJb9iiJKLvOHB-RsxGLu1jxdk4_enK1CVE55nj2gyHh2QgAgqVmaBA/http%3A%2F%2Fapache.claz.org
>  is site is slow.
> 
> 4.  Login as root user:  sudo -i  and enter you admin password
> 
> 5.  Unzip the .tgz file:  tar zxf solr-x.x.x.tgz
> 
> 6.  Change directories:  cd /
> 
> 7.  Stop SOLR service: service solr stop
> 
> 8.  Confirm that SOLR is stopped:   service solr status
> 
> 9.  Change directories to your user home directory:  cd
> /home/myadminlogonid
> 
> 10.  Create new solr symbolic link in your user home folder that
> points to new SOLR version:   ln -s /opt/solr-x.x.x  solr
> 
> 11.  Move/replace current symbolic link:  mv solr /opt

What version are you going from/to?

What OS is this?

Do you have an /etc/init.d/solr file? If so, where does SOLR_INSTALL_DIR point?

Do you have an /etc/default/solr.in.sh file? If it points to all of your 
data-locations, then you should be okay.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - 
http://secure-web.cisco.com/1ak0mJWMZCzmwcyIndMHuJ3k5xajkomTtNWOvHNtOcBmlluzzjTX0Tc7C0tCWeQX-xUlb5Ch0T-qRi82ItzHrVQx7vgDLJgFF51RgDiBjLfVEf_TWTz_4S_RW4OtxmkQkEEfqagSWeiYG39ZuabHXvVvAWdrDwOMOTSNDaC1IK9b5pCjdfDFgLw434CcY1bSP4NxJOyMQy-Fiv5y4_aByTuFrnFem8zB3PUawfJzkYjg2hhGB6PUfniGH9L4ANKXGgQbSQlHyPjbzH4QKUQHWLw/http%3A%2F%2Fgpgtools.org
Comment: Using GnuPG with Thunderbird - 
http://secure-web.cisco.com/1IRbbFHeU2a-UXJDDF-tcE1v1QO2t9s2zAI8TObbDuQRPrQWoJhd40xQJROibQyxx07Yte4ZROVQd8l2T3mcN6KIQNmLiw7jPmK3SgPu97KZet2b9ZLaSkiXX_X7v0ldepb8GpzsPdgltF-Y64g0hvNTIP0jgtEVugFGGAuNJn0lg0avoF4XWRVJedSy-D7K3h4wsULHEVXi95GCGZiCgXzbyWLT7owV3zOPAHBX3vOBr0JNsixKcTvroIPYwCK8NIX3g5X5BXfi3CdAx8uTAGg/http%3A%2F%2Fwww.enigmail.net%2F

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAltY3BgACgkQHPApP6U8
pFgPBRAAvhRGsL2b7WN9ivgerORr8P9Q5+eSrfiCh+DHcBWGUXbLvC5286U2Ieua
F6A5xPbjg6hhNoo7TVN/b+5iPBJbZL/Ea9UnuR7ZdVL+xTVktoN+Y5HWUEHwEFk1
PTzfAw/GyNmN4hQbFLVQbYQn+hzYyj1xXCtwa/RKO82c7CEM5H43aTO90CoZa2Vh
rNBeBiXXKPmlaL+RJdDs2yRZAjpTYO2FMJAZWPrzNq9R956tuZj8rPMrERhpLBuk
Dh/33EZKaanLzEBEfOU5O5Qqm5oOlKqDDOK3hs25ru8o6pZ7wAPsiiBof0dSBkj1
V/DGdUfrSMjzVi7DYC1Ie0m1RI8IvHwUZZV7cT23S73U6+QvP+9ap/m8/P4CZCtH
i06aSfEFHEhcvjM2DQ2+sbn2VRinbiQWggGtlr0lrauOSdJ/NCTb4fgiZ3w/esbC
xdY4O9HwQhkjyKFgagKKIBdx/4klusrM+mx/VdhqQ5RtfiWcO3gqKZPlVHYfyc6m
FWMW4i06QfmZLyLeH6xzBqOVUcUdY7UbwALEOO/Kgm2B9J9t/azDlkM4XcnMLeBT
Ee7WuqREe4JoV9iH+MvReHfA+FbrO5vt0b2LFgI2RmcEgFzp1CDq/vbqkcEESN6C
5tRG1VjOpBTINvAKo2hNNwmiNIDGa6ZpWcxqgknZqQJiZ7AsuUw=
=GP6+
-END PGP SIGNATURE-

Re: Upgrading SOLR (not clustered)

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Phil,

On 7/25/18 12:38 PM, Staley, Phil R - DCF wrote:
> What are the steps for upgrading a non-clustered SOLR version?
> Here's what I thought should work:
> 
> 
> 
> 1.  Open a bash window and ssh login to desired server with
> your Linux admin credentials
> 
> 2.  Change directories:  cd /opt
> 
> 3.  Download the latest Linux/OSX version direct to server: 
> sudo wget http://apache.claz.org/lucene/solr/x.x.x/solr-x.x.x.tgz
> (replace x.x.x with the latest version number)
> 
> a.  Additional download mirror servers are available @
> http://www.apache.org/dyn/closer.lua/lucene/solr/7.3.1 if the
> http://apache.claz.org is site is slow.
> 
> 4.  Login as root user:  sudo -i  and enter you admin password
> 
> 5.  Unzip the .tgz file:  tar zxf solr-x.x.x.tgz
> 
> 6.  Change directories:  cd /
> 
> 7.  Stop SOLR service: service solr stop
> 
> 8.  Confirm that SOLR is stopped:   service solr status
> 
> 9.  Change directories to your user home directory:  cd
> /home/myadminlogonid
> 
> 10.  Create new solr symbolic link in your user home folder that
> points to new SOLR version:   ln -s /opt/solr-x.x.x  solr
> 
> 11.  Move/replace current symbolic link:  mv solr /opt

What version are you going from/to?

What OS is this?

Do you have an /etc/init.d/solr file? If so, where does
SOLR_INSTALL_DIR point?

Do you have an /etc/default/solr.in.sh file? If it points to all of
your data-locations, then you should be okay.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAltY3BgACgkQHPApP6U8
pFgPBRAAvhRGsL2b7WN9ivgerORr8P9Q5+eSrfiCh+DHcBWGUXbLvC5286U2Ieua
F6A5xPbjg6hhNoo7TVN/b+5iPBJbZL/Ea9UnuR7ZdVL+xTVktoN+Y5HWUEHwEFk1
PTzfAw/GyNmN4hQbFLVQbYQn+hzYyj1xXCtwa/RKO82c7CEM5H43aTO90CoZa2Vh
rNBeBiXXKPmlaL+RJdDs2yRZAjpTYO2FMJAZWPrzNq9R956tuZj8rPMrERhpLBuk
Dh/33EZKaanLzEBEfOU5O5Qqm5oOlKqDDOK3hs25ru8o6pZ7wAPsiiBof0dSBkj1
V/DGdUfrSMjzVi7DYC1Ie0m1RI8IvHwUZZV7cT23S73U6+QvP+9ap/m8/P4CZCtH
i06aSfEFHEhcvjM2DQ2+sbn2VRinbiQWggGtlr0lrauOSdJ/NCTb4fgiZ3w/esbC
xdY4O9HwQhkjyKFgagKKIBdx/4klusrM+mx/VdhqQ5RtfiWcO3gqKZPlVHYfyc6m
FWMW4i06QfmZLyLeH6xzBqOVUcUdY7UbwALEOO/Kgm2B9J9t/azDlkM4XcnMLeBT
Ee7WuqREe4JoV9iH+MvReHfA+FbrO5vt0b2LFgI2RmcEgFzp1CDq/vbqkcEESN6C
5tRG1VjOpBTINvAKo2hNNwmiNIDGa6ZpWcxqgknZqQJiZ7AsuUw=
=GP6+
-END PGP SIGNATURE-

Re: how to index GEO JSON

[Alexandre Rafalovitch]

https://lucene.apache.org/solr/guide/7_4/spatial-search.html#indexing-geojson-and-wkt
?

Regards,
   Alex.

On 25 July 2018 at 16:15, SolrUser1543  wrote:
> I have look in reference guide and different wiki articles , but have not
> found anywhere an example of how index geojson .
>
> I have the following field definition :
>  class="solr.SpatialRecursivePrefixTreeFieldType"
>
> spatialContextFactory="org.locationtech.spatial4j.context.jts.JtsSpatialContextFactory"
>autoIndex="true"
>validationRule="repairBuffer0"
>distErrPct="0.025"
>maxDistErr="0.001"
>distanceUnits="kilometers" />
>
> how should post request looks like in order to put geojson in this field ?
>
> I have managed to index WKT , but not geojson .
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

how to index GEO JSON

[SolrUser1543]

I have look in reference guide and different wiki articles , but have not
found anywhere an example of how index geojson .

I have the following field definition :


how should post request looks like in order to put geojson in this field ? 

I have managed to index WKT , but not geojson . 



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Section symbol, ignore in some queries but not others?

[David Hastings]

Ah, so I could index the text including the § character as an alpha, use no
qs value when trying to ignore it, and for users add i a qs value assuming
I use edismax, whic I currently am.

Tested this method and it works as expected.  Thanks, saved me a lot of
time!
-David

On Wed, Jul 25, 2018 at 3:15 PM, Alexandre Rafalovitch 
wrote:

> If you copyField and don't store the copy, then it is only the indexed
> (term) representation for the copy that is much smaller. Just a
> thought.
>
> The other thing is that you seem to be saying that you want to do a
> match phrase but with a token gap, right? Like an eDisMax slop?
> http://lucene.apache.org/solr/guide/7_4/the-extended-dismax-
> query-parser.html
>
> Regards,
>Alex.
>
> On 25 July 2018 at 14:47, David Hastings 
> wrote:
> > Hey all.  have a situation that seems pretty rough.  currently in our
> data
> > we have a lot of sentences like this:
> >
> > elements comprise the "stuff" of the tax. 3 Reg. § 1.901-2(a)(2). 4 Only
> > non-Saudis are subject to the
> >  223%20Regulation%201%22%20OR%20%223%20Regulation%201%22%
> 20OR%20%223%20Reg.%201%22)%20AND%20NOT%20id:hein.
> journals/rcatorbg3.14))=13=hein.journals/
> taxlr53=journals>
> > By default the word delimiter is treating all punctuation as a space.  So
> > when you search for:
> > 3 Reg. 1, your results can include  3 Reg. § 1.901
> >
> > I Have experimented with the WDF and added § => ALPHA and this works, and
> > treats the character as a letter.  however during some queries, I still
> > need searches such as
> >
> > Servitudes 2.10
> >
> > to return results with:
> >
> >
> > Servitudes § 2.10
> >
> >
> > I at the moment can not conceive of a way to to this aside from two
> > separate text fields, and effectively doubling the size of my index.
> > which currently sits at 300 gb optimized, and 500gb if left to its
> > own.
> >
> >
> > Thanks for any help or suggestions
>

Re: Section symbol, ignore in some queries but not others?

[Alexandre Rafalovitch]

If you copyField and don't store the copy, then it is only the indexed
(term) representation for the copy that is much smaller. Just a
thought.

The other thing is that you seem to be saying that you want to do a
match phrase but with a token gap, right? Like an eDisMax slop?
http://lucene.apache.org/solr/guide/7_4/the-extended-dismax-query-parser.html

Regards,
   Alex.

On 25 July 2018 at 14:47, David Hastings  wrote:
> Hey all.  have a situation that seems pretty rough.  currently in our data
> we have a lot of sentences like this:
>
> elements comprise the "stuff" of the tax. 3 Reg. § 1.901-2(a)(2). 4 Only
> non-Saudis are subject to the
> 
> By default the word delimiter is treating all punctuation as a space.  So
> when you search for:
> 3 Reg. 1, your results can include  3 Reg. § 1.901
>
> I Have experimented with the WDF and added § => ALPHA and this works, and
> treats the character as a letter.  however during some queries, I still
> need searches such as
>
> Servitudes 2.10
>
> to return results with:
>
>
> Servitudes § 2.10
>
>
> I at the moment can not conceive of a way to to this aside from two
> separate text fields, and effectively doubling the size of my index.
> which currently sits at 300 gb optimized, and 500gb if left to its
> own.
>
>
> Thanks for any help or suggestions

Section symbol, ignore in some queries but not others?

[David Hastings]

Hey all.  have a situation that seems pretty rough.  currently in our data
we have a lot of sentences like this:

elements comprise the "stuff" of the tax. 3 Reg. § 1.901-2(a)(2). 4 Only
non-Saudis are subject to the

By default the word delimiter is treating all punctuation as a space.  So
when you search for:
3 Reg. 1, your results can include  3 Reg. § 1.901

I Have experimented with the WDF and added § => ALPHA and this works, and
treats the character as a letter.  however during some queries, I still
need searches such as

Servitudes 2.10

to return results with:


Servitudes § 2.10


I at the moment can not conceive of a way to to this aside from two
separate text fields, and effectively doubling the size of my index.
which currently sits at 300 gb optimized, and 500gb if left to its
own.


Thanks for any help or suggestions

Re: FieldValueCache in solr 6.6

[Edward Ribeiro]

FieldValueCache is used by faceting, mostly. So, you would need to execute
warm faceting queries to pre-populate it. More info in this old mailing
list topic:

http://lucene.472066.n3.nabble.com/Loading-data-to-FieldValueCache-tp4175721.html

Cheers,
Edward

Em qua, 11 de jul de 2018 02:09, zhang.mingyue <1206708...@qq.com> escreveu:

> hi:
>I have a question ，how to  load data to FieldValueCache in solr7
>
>thanks
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>

Re: SolrCloud not able to view cloud page - Loading of "/solr/zookeeper?wt=json" failed (HTTP-Status 500)

[aparna]

@shawn Heisey,

My client is facing the same issue. However, I am not sure / have not worked
with the ZkCli script in Zookeeper. Could you please help me with the steps?
If you could send me, where can I find the ZkCli script associated with
Zookeeper and the exact commands to run from each node, it will be great.

Thanks in advance.



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Upgrading SOLR (not clustered)

[Staley, Phil R - DCF]

What are the steps for upgrading a non-clustered SOLR version?  Here's what I 
thought should work:



1.  Open a bash window and ssh login to desired server with your Linux 
admin credentials

2.  Change directories:  cd /opt

3.  Download the latest Linux/OSX version direct to server:
sudo wget http://apache.claz.org/lucene/solr/x.x.x/solr-x.x.x.tgz (replace 
x.x.x with the latest version number)

a.  Additional download mirror servers are available
@ http://www.apache.org/dyn/closer.lua/lucene/solr/7.3.1 if the 
http://apache.claz.org is site is slow.

4.  Login as root user:  sudo -i  and enter you admin password

5.  Unzip the .tgz file:  tar zxf solr-x.x.x.tgz

6.  Change directories:  cd /

7.  Stop SOLR service: service solr stop

8.  Confirm that SOLR is stopped:   service solr status

9.  Change directories to your user home directory:  cd /home/myadminlogonid

10.  Create new solr symbolic link in your user home folder that points to new 
SOLR version:   ln -s /opt/solr-x.x.x  solr

11.  Move/replace current symbolic link:  mv solr /opt



Phil Staley
DCF Webmaster
608 422-6569
phil.sta...@wisconsin.gov

Re: MetricsHistoryHandler getOverseerLeader fails when hostname contains hyphen

[Chris Hostetter]

: Subject: MetricsHistoryHandler getOverseerLeader fails when hostname contains
: hyphen

that's unfortunate.  I filed a jira...

https://issues.apache.org/jira/browse/SOLR-12594

: Can one just ignore this warning and what will happen then?

I think as long as you don't care about the mstrics history reporting 
(which collects long term metrics to rollup and see changes over time) you 
can probably ignore that warning...

https://lucene.apache.org/solr/guide/7_4/metrics-history.html


-Hoss
http://www.lucidworks.com/

MetricsHistoryHandler getOverseerLeader fails when hostname contains hyphen

[Andi S]

Hello!

We encounter a lot of log warning entries from the MetricsHistoryHandler saying

o.a.s.h.a.MetricsHistoryHandler Unknown format of leader id, skipping: 
244550997187166214-server1-b.myhost:8983_solr-n_94


I don't even know what this _MetricsHistoryHandler_ does, but at least there's 
a warning.


Looking at the code you can see that it has to fail if the hostname of the node 
contains a hyphen:

[https://github.com/apache/lucene-solr/blob/d25f62634b95e49df739a5a2612f9b719ef3a77d/solr/core/src/java/org/apache/solr/handler/admin/MetricsHistoryHandler.java#L335]



Can one just ignore this warning and what will happen then?


Thanks in advance!

create collection from existing managed-schema

[Chuming Chen]

Hi All,

From Solr Admin interface, I have created a collection and added field 
definitions. I can get its managed-schema from the Admin interface. 

Can I use this managed-schema to create a new collection? If yes, how?

Thanks,

Chuming

Upgrading SOLR (not clustered)

[Staley, Phil R - DCF]

What are the steps for upgrading a non-clustered SOLR version?  Here's what I 
thought should work:



1.  Open a bash window and ssh login to desired server with your Linux 
admin credentials

2.  Change directories:  cd /opt

3.  Download the latest Linux/OSX version direct to server:
sudo wget http://apache.claz.org/lucene/solr/x.x.x/solr-x.x.x.tgz (replace 
x.x.x with the latest version number)

a.  Additional download mirror servers are available
@ http://www.apache.org/dyn/closer.lua/lucene/solr/7.3.1 if the 
http://apache.claz.org is site is slow.

4.  Login as root user:  sudo -i  and enter you admin password

5.  Unzip the .tgz file:  tar zxf solr-x.x.x.tgz

6.  Change directories:  cd /

7.  Stop SOLR service: service solr stop

8.  Confirm that SOLR is stopped:   service solr status

9.  Change directories to your user home directory:  cd /home/myadminlogonid

10.  Create new solr symbolic link in your user home folder that points to new 
SOLR version:   ln -s /opt/solr-x.x.x  solr

11.  Move/replace current symbolic link:  mv solr /opt



Phil Staley
DCF Webmaster
608 422-6569
phil.sta...@wisconsin.gov

Re: cursorMark and sort order

[Erick Erickson]

When you specify sort order, Solr makes no assumptions at all. So
sort =  asc would sort _only_ by key. Imagine how
puzzled people would be if they specify sort= asc and... the result
was really ordered by score. Yuck.

So yes, you must specify both.

Best,
Erick

On Tue, Jul 24, 2018 at 11:37 PM, Dave Durbin  wrote:
> Hi,
>
> For deep pagination, it is recommended that we use cursorMark and provide a 
> sort order for  as a tiebreaker.
>
> I want my results in relevancy order and so have no sort specified on my 
> query by default.
>
> Do I need to explicitly set :
>
> sort : score desc,  asc
>
> Or can I get away with just :
>
> sort =  asc
>
> and have Solr understand that the sort is only for tie break purposes?
>
>
> Thanks
> Dave
>
>
> --
> *P.S. We've launched a new blog to share the latest ideas and case studies
> from our team. Check it out here: product.canva.com
> . ***
> ** Empowering the world
> to design
> Also, we're hiring. Apply here!
> 
>  
>  
> 
>
>
>
>
>

how to index GEO JSON

[SolrUser1543]

I have look in reference guide and different wiki articles , but have not
found anywhere an example of how index geojson .

I have the following field definition :


how should post request looks like in order to put geojson in this field ? 

I have managed to index WKT , but not geojson . 



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: cursorMark and sort order

[Chris Hostetter]

: For deep pagination, it is recommended that we use cursorMark and 
: provide a sort order for  as a tiebreaker.
: 
: I want my results in relevancy order and so have no sort specified on my 
query by default.
: 
: Do I need to explicitly set :
: 
:   sort : score desc,  asc

Yes.

: Or can I get away with just :
: 
:   sort =  asc
: 
: and have Solr understand that the sort is only for tie break purposes?

No, if you use the later, solr will assume you don't care about scores at 
all.



-Hoss
http://www.lucidworks.com/

Re: Possible to define a field so that substring-search is always used?

[Alexandre Rafalovitch]

> I think n-grams sounds like the only way to get this done.

You don't have to settle for "the only way". You can totally have the
same field(s) copyFielded into multiple locations and then have each
target field use a different indexing pipeline, including ngrams,
phonetic processing, full match with/without "@domain" part, etc.
Then, with eDismax multi-field searches and/or boost queries you can
give higher boost to the copies with least amount of processing and
lower boost to less-precise, more-inclusive matches.

Regards,
   Alex.

On 25 July 2018 at 09:23, Christopher Schultz
 wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Chris,
>
> On 7/24/18 4:46 PM, Chris Hostetter wrote:
>>
>> : We are using Solr as a user index, and users have email
>> addresses. : : Our old search behavior used a SQL substring match
>> for any search : terms entered, and so users are used to being able
>> to search for e.g. : "chr" and finding my email address
>> ("ch...@christopherschultz.net"). : : By default, Solr doesn't
>> perform substring matches, and it might be : difficult to re-train
>> users to use *chr* to find email addresses by : substring.
>>
>> In the past, were you really doing arbitrary substring matching, or
>> just prefix matching?  ie would a search for "sto" match
>> "ch...@christopherschultz.net"
>
> Yes. Searching for "sto" would result in a SQL query with a " WHERE
> ... LIKE '%sto%'" clause. So it was slow as hell, of course.
>
>> Personally, if you know you have an email field, would suggest
>> using a custom tokenizer that splits on "@" and "." (and maybe
>> other punctuation characters like "-") and then take your raw user
>> input and feed it to the prefix parser (instead of requiring your
>> users to add the "*")...
>>
>> q={!prefix f=email v=$user_input}_input=chr
>>
>> ...which would match ch...@gmail.com, f...@chris.com, f...@bar.chr
>> etc.
>>
>> (this wouldn't help you though if you *really* want arbitrary
>> substring matching -- as erick suggested ngrams is pretty much your
>> best bet for something like that)
>>
>> Bear in mind, you can combine that "forced prefix" query against
>> the (otkenized) email field with other queries that could parse
>> your input in other ways...
>>
>> user_input=... q=({!prefix f=email v=$user_input} OR {!dismax
>> qf="first_name last_name" ..etc.. v=$user_input})
>>
>> so if your user input is "chris" you'll get term matches on the
>> first_name field, or the last_name field as well as prefix matches
>> on the email field.
>
> The problem is that our users (admins) sometimes need to locate users
> by their email address, and people often forget the exact spelling. So
> they'll call and say "I can't get in" and we have to search for "chris
> schultz" and then "chris" and then it turns out that their email
> address was actually sexylove...@yahoo.com, so they often have to try
> a bunch of searches before finding the right user record. Having to
> search for "sexylover42", a complete-match word, isn't going to work
> for their use-case. They need to be able to search for "lover" and
> have it work. I think n-grams sounds like the only way to get this
> done. I'll have to play-around with it a little bit to see how it behave
> s.
>
> Thanks,
> - -chris

Re: Possible to define a field so that substring-search is always used?

[Erick Erickson]

If you're not familiar with the edismax query parser, that's often what
people use to fire the query off against more than one field without
having the users be aware of it. That means you could ngram
the e-mail field and when a user types something in the search
box search against both the "all" and "email" fields without them having
to know they exist.

Best,
Erick

On Wed, Jul 25, 2018 at 6:23 AM, Christopher Schultz
 wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Chris,
>
> On 7/24/18 4:46 PM, Chris Hostetter wrote:
>>
>> : We are using Solr as a user index, and users have email
>> addresses. : : Our old search behavior used a SQL substring match
>> for any search : terms entered, and so users are used to being able
>> to search for e.g. : "chr" and finding my email address
>> ("ch...@christopherschultz.net"). : : By default, Solr doesn't
>> perform substring matches, and it might be : difficult to re-train
>> users to use *chr* to find email addresses by : substring.
>>
>> In the past, were you really doing arbitrary substring matching, or
>> just prefix matching?  ie would a search for "sto" match
>> "ch...@christopherschultz.net"
>
> Yes. Searching for "sto" would result in a SQL query with a " WHERE
> ... LIKE '%sto%'" clause. So it was slow as hell, of course.
>
>> Personally, if you know you have an email field, would suggest
>> using a custom tokenizer that splits on "@" and "." (and maybe
>> other punctuation characters like "-") and then take your raw user
>> input and feed it to the prefix parser (instead of requiring your
>> users to add the "*")...
>>
>> q={!prefix f=email v=$user_input}_input=chr
>>
>> ...which would match ch...@gmail.com, f...@chris.com, f...@bar.chr
>> etc.
>>
>> (this wouldn't help you though if you *really* want arbitrary
>> substring matching -- as erick suggested ngrams is pretty much your
>> best bet for something like that)
>>
>> Bear in mind, you can combine that "forced prefix" query against
>> the (otkenized) email field with other queries that could parse
>> your input in other ways...
>>
>> user_input=... q=({!prefix f=email v=$user_input} OR {!dismax
>> qf="first_name last_name" ..etc.. v=$user_input})
>>
>> so if your user input is "chris" you'll get term matches on the
>> first_name field, or the last_name field as well as prefix matches
>> on the email field.
>
> The problem is that our users (admins) sometimes need to locate users
> by their email address, and people often forget the exact spelling. So
> they'll call and say "I can't get in" and we have to search for "chris
> schultz" and then "chris" and then it turns out that their email
> address was actually sexylove...@yahoo.com, so they often have to try
> a bunch of searches before finding the right user record. Having to
> search for "sexylover42", a complete-match word, isn't going to work
> for their use-case. They need to be able to search for "lover" and
> have it work. I think n-grams sounds like the only way to get this
> done. I'll have to play-around with it a little bit to see how it behave
> s.
>
> Thanks,
> - -chris
> -BEGIN PGP SIGNATURE-
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAltYedQACgkQHPApP6U8
> pFjzgQ/9GW7kI9Lefnmj7zH8JsqZfW1Y/PrF4YA1RjbliNWRn2dRPz7Q7C2ITO/n
> Ys73uUII3qPz8M/H6d0LN57Un96BGAjIhf6WZSiIRAQcvenhGaS/lROciq6I8iN8
> hB+1X2GixTG8fbq6Q6Q3jRG22S0GpW+OL2mJcu3wCkQ2dzyBWObWxjF1ag5O4pT+
> AP0lqAgpUTsWAeMPPd6dkuStOhXraJQc+1WwwEw36gohwaZwLMftcOl2ohnys/DM
> pdyqQEQ6fOldJLBHLU8PyNVHxJA5qZjVTwu3S7zv7w+2N+V8bHOl6y5ir3krOEs0
> OIvFX+Do+pbsg+QQ5VY8LDxbPBCjgDiWTpplh3Ym0raaVMoMQ6GfFfsOPF9jYhxS
> gb0eMwVTJFWM0xvMaH4xSXLR/Dh6upT/0do1sTr7kKjhIlwc3pfR/vIwqsVer1HJ
> Qsj6Pc+ZJckOrPGGIYCZEWZwlS8ONinAx4fh23/C1GltU19kHtRvGTQLzRT+9sus
> 2stvkD44Lv7zuc49/Y07NISxcUceTlbZHKC5ebzAtKNDS2p+qYLJlbdTZQIofMsb
> zmncdP+s5cSYgiCZZS19E2GxP7Yw2rmSn2zsSF6yJMgMy9logJi5HS1UQ54IWvn7
> eAzvM+TcV6i+8Hf9kijNcg4/OZPv67DZt6HDcXO2K+a/AMyQElE=
> =4Y/b
> -END PGP SIGNATURE-

Re: Possible to define a field so that substring-search is always used?

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Chris,

On 7/24/18 4:46 PM, Chris Hostetter wrote:
> 
> : We are using Solr as a user index, and users have email
> addresses. : : Our old search behavior used a SQL substring match
> for any search : terms entered, and so users are used to being able
> to search for e.g. : "chr" and finding my email address
> ("ch...@christopherschultz.net"). : : By default, Solr doesn't
> perform substring matches, and it might be : difficult to re-train
> users to use *chr* to find email addresses by : substring.
> 
> In the past, were you really doing arbitrary substring matching, or
> just prefix matching?  ie would a search for "sto" match 
> "ch...@christopherschultz.net"

Yes. Searching for "sto" would result in a SQL query with a " WHERE
... LIKE '%sto%'" clause. So it was slow as hell, of course.

> Personally, if you know you have an email field, would suggest
> using a custom tokenizer that splits on "@" and "." (and maybe
> other punctuation characters like "-") and then take your raw user
> input and feed it to the prefix parser (instead of requiring your
> users to add the "*")...
> 
> q={!prefix f=email v=$user_input}_input=chr
> 
> ...which would match ch...@gmail.com, f...@chris.com, f...@bar.chr
> etc.
> 
> (this wouldn't help you though if you *really* want arbitrary
> substring matching -- as erick suggested ngrams is pretty much your
> best bet for something like that)
> 
> Bear in mind, you can combine that "forced prefix" query against 
> the (otkenized) email field with other queries that could parse
> your input in other ways...
> 
> user_input=... q=({!prefix f=email v=$user_input} OR {!dismax
> qf="first_name last_name" ..etc.. v=$user_input})
> 
> so if your user input is "chris" you'll get term matches on the 
> first_name field, or the last_name field as well as prefix matches
> on the email field.

The problem is that our users (admins) sometimes need to locate users
by their email address, and people often forget the exact spelling. So
they'll call and say "I can't get in" and we have to search for "chris
schultz" and then "chris" and then it turns out that their email
address was actually sexylove...@yahoo.com, so they often have to try
a bunch of searches before finding the right user record. Having to
search for "sexylover42", a complete-match word, isn't going to work
for their use-case. They need to be able to search for "lover" and
have it work. I think n-grams sounds like the only way to get this
done. I'll have to play-around with it a little bit to see how it behave
s.

Thanks,
- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAltYedQACgkQHPApP6U8
pFjzgQ/9GW7kI9Lefnmj7zH8JsqZfW1Y/PrF4YA1RjbliNWRn2dRPz7Q7C2ITO/n
Ys73uUII3qPz8M/H6d0LN57Un96BGAjIhf6WZSiIRAQcvenhGaS/lROciq6I8iN8
hB+1X2GixTG8fbq6Q6Q3jRG22S0GpW+OL2mJcu3wCkQ2dzyBWObWxjF1ag5O4pT+
AP0lqAgpUTsWAeMPPd6dkuStOhXraJQc+1WwwEw36gohwaZwLMftcOl2ohnys/DM
pdyqQEQ6fOldJLBHLU8PyNVHxJA5qZjVTwu3S7zv7w+2N+V8bHOl6y5ir3krOEs0
OIvFX+Do+pbsg+QQ5VY8LDxbPBCjgDiWTpplh3Ym0raaVMoMQ6GfFfsOPF9jYhxS
gb0eMwVTJFWM0xvMaH4xSXLR/Dh6upT/0do1sTr7kKjhIlwc3pfR/vIwqsVer1HJ
Qsj6Pc+ZJckOrPGGIYCZEWZwlS8ONinAx4fh23/C1GltU19kHtRvGTQLzRT+9sus
2stvkD44Lv7zuc49/Y07NISxcUceTlbZHKC5ebzAtKNDS2p+qYLJlbdTZQIofMsb
zmncdP+s5cSYgiCZZS19E2GxP7Yw2rmSn2zsSF6yJMgMy9logJi5HS1UQ54IWvn7
eAzvM+TcV6i+8Hf9kijNcg4/OZPv67DZt6HDcXO2K+a/AMyQElE=
=4Y/b
-END PGP SIGNATURE-

Re: Block Join Faceting issue

[Mikhail Khludnev]

 q=*:*={!parent which="doc_type:parent"}+doc_type:child -child_color:*

Make sure that + isn;t grabbed by url encoding.

On Wed, Jul 25, 2018 at 8:23 AM sagandhi  wrote:

> Hi Mikhail,
>
> Thank you for suggesting to use json facet. I tried json.facet, it works
> great and I am able to make a single query instead of two. Now I am
> planning
> to get rid of the duplicate child fields in parent docs. However I ran into
> problems while forming negative queries with block join.
>
> Here's what I would like to query - Get me parent docs whose children do
> not
> have a particular field.
> I tried these but none worked -
>
> q=*:*={!parent which="doc_type:parent"}*-*child_color:*
> q=*:*={!parent which="doc_type:parent" v=$qq}=(!child_color:*)
>
> Currently I have duplicate entries of child fields in parent docs, so I am
> able to do this -
> =!parent_color:*
>
> Is there a way to form this query using block join?
>
> Thanks,
> Soham
>
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>


-- 
Sincerely yours
Mikhail Khludnev

cursorMark and sort order

[Dave Durbin]

Hi,

For deep pagination, it is recommended that we use cursorMark and provide a 
sort order for  as a tiebreaker.

I want my results in relevancy order and so have no sort specified on my query 
by default.

Do I need to explicitly set :

sort : score desc,  asc

Or can I get away with just :

sort =  asc

and have Solr understand that the sort is only for tie break purposes?


Thanks
Dave


-- 
*P.S. We've launched a new blog to share the latest ideas and case studies 
from our team. Check it out here: product.canva.com 
. ***
** Empowering the world 
to design
Also, we're hiring. Apply here!