Re: [ANNOUNCE] Apache Solr 8.6.0 released

[matthew sporleder]

I hear all of that and agree, obviously, but "curl
solr:8983/collection/dataimport?blah" in cron was *pretty freaking
easy* ;)

Not sure why "pull" is elevated to "anti-pattern"; data is data is data

On Thu, Jul 16, 2020 at 8:49 PM Ishan Chattopadhyaya
 wrote:
>
> Thanks Aroop for your feedback. We shall try to ensure continuity of
> functionality via packages. Your help in those efforts would be greatly
> appreciated as well. Let us take this discussion to SOLR-14660.
>
> > Is there a replacement for DIH?
> DIH is available as a community supported package. However, it is an
> anti-pattern for a search engine to be pulling data from outside. Instead,
> please consider writing separate indexing programs that pull data from the
> database systems and index into Solr. It is not only a good practice, but
> also more efficient in terms of throughput. For more information on this,
> please start another thread in solr-users@ list, and more people can
> suggest best alternatives here.
>
>
> On Fri, Jul 17, 2020 at 5:50 AM matthew sporleder 
> wrote:
>
> > Is there a replacement for DIH?
> >
> > On Wed, Jul 15, 2020 at 10:08 AM Ishan Chattopadhyaya
> >  wrote:
> > >
> > > Dear Solr Users,
> > >
> > > In this release (Solr 8.6), we have deprecated the following:
> > >
> > >   1. Data Import Handler
> > >
> > >   2. HDFS support
> > >
> > >   3. Cross Data Center Replication (CDCR)
> > >
> > >
> > >
> > > All of these are scheduled to be removed in a future 9.x release.
> > >
> > > It was decided that these components did not meet the standards of
> > quality
> > > and support that we wish to ensure for all components we ship. Some of
> > > these also relied on design patterns that we no longer recommend for use
> > in
> > > critical production environments.
> > >
> > > If you rely on these features, you are encouraged to try out community
> > > supported versions of these, where available [0]. Where such community
> > > support is not available, we encourage you to participate in the
> > migration
> > > of these components into community supported packages and help continue
> > the
> > > development. We envision that using packages for these components via
> > > package manager will actually make it easier for users to use such
> > features.
> > >
> > > Regards,
> > >
> > > Ishan Chattopadhyaya
> > >
> > > (On behalf of the Apache Lucene/Solr PMC)
> > >
> > > [0] -
> > >
> > https://cwiki.apache.org/confluence/display/SOLR/Community+supported+packages+for+Solr
> > >
> > > On Wed, Jul 15, 2020 at 2:30 PM Bruno Roustant  > >
> > > wrote:
> > >
> > > > The Lucene PMC is pleased to announce the release of Apache Solr 8.6.0.
> > > >
> > > >
> > > > Solr is the popular, blazing fast, open source NoSQL search platform
> > from
> > > > the Apache Lucene project. Its major features include powerful
> > full-text
> > > > search, hit highlighting, faceted search, dynamic clustering, database
> > > > integration, rich document handling, and geospatial search. Solr is
> > highly
> > > > scalable, providing fault tolerant distributed search and indexing, and
> > > > powers the search and navigation features of many of the world's
> > largest
> > > > internet sites.
> > > >
> > > >
> > > > Solr 8.6.0 is available for immediate download at:
> > > >
> > > >
> > > >   <https://lucene.apache.org/solr/downloads.html>
> > > >
> > > >
> > > > ### Solr 8.6.0 Release Highlights:
> > > >
> > > >
> > > >  * Cross-Collection Join Queries: Join queries can now work
> > > > cross-collection, even when shared or when spanning nodes.
> > > >
> > > >  * Search: Performance improvement for some types of queries when exact
> > > > hit count isn't needed by using BlockMax WAND algorithm.
> > > >
> > > >  * Streaming Expression: Percentiles and standard deviation
> > aggregations
> > > > added to stats, facet and time series.  Streaming expressions added to
> > > > /export handler.  Drill Streaming Expression for efficient and accurate
> > > > high cardinality aggregation.
> > > >
> > > >  * Package manager: Support for cl

Re: [ANNOUNCE] Apache Solr 8.6.0 released

[matthew sporleder]

Is there a replacement for DIH?

On Wed, Jul 15, 2020 at 10:08 AM Ishan Chattopadhyaya
 wrote:
>
> Dear Solr Users,
>
> In this release (Solr 8.6), we have deprecated the following:
>
>   1. Data Import Handler
>
>   2. HDFS support
>
>   3. Cross Data Center Replication (CDCR)
>
>
>
> All of these are scheduled to be removed in a future 9.x release.
>
> It was decided that these components did not meet the standards of quality
> and support that we wish to ensure for all components we ship. Some of
> these also relied on design patterns that we no longer recommend for use in
> critical production environments.
>
> If you rely on these features, you are encouraged to try out community
> supported versions of these, where available [0]. Where such community
> support is not available, we encourage you to participate in the migration
> of these components into community supported packages and help continue the
> development. We envision that using packages for these components via
> package manager will actually make it easier for users to use such features.
>
> Regards,
>
> Ishan Chattopadhyaya
>
> (On behalf of the Apache Lucene/Solr PMC)
>
> [0] -
> https://cwiki.apache.org/confluence/display/SOLR/Community+supported+packages+for+Solr
>
> On Wed, Jul 15, 2020 at 2:30 PM Bruno Roustant 
> wrote:
>
> > The Lucene PMC is pleased to announce the release of Apache Solr 8.6.0.
> >
> >
> > Solr is the popular, blazing fast, open source NoSQL search platform from
> > the Apache Lucene project. Its major features include powerful full-text
> > search, hit highlighting, faceted search, dynamic clustering, database
> > integration, rich document handling, and geospatial search. Solr is highly
> > scalable, providing fault tolerant distributed search and indexing, and
> > powers the search and navigation features of many of the world's largest
> > internet sites.
> >
> >
> > Solr 8.6.0 is available for immediate download at:
> >
> >
> >   
> >
> >
> > ### Solr 8.6.0 Release Highlights:
> >
> >
> >  * Cross-Collection Join Queries: Join queries can now work
> > cross-collection, even when shared or when spanning nodes.
> >
> >  * Search: Performance improvement for some types of queries when exact
> > hit count isn't needed by using BlockMax WAND algorithm.
> >
> >  * Streaming Expression: Percentiles and standard deviation aggregations
> > added to stats, facet and time series.  Streaming expressions added to
> > /export handler.  Drill Streaming Expression for efficient and accurate
> > high cardinality aggregation.
> >
> >  * Package manager: Support for cluster (CoreContainer) level plugins.
> >
> >  * Health Check: HealthCheckHandler can now require that all cores are
> > healthy before returning OK.
> >
> >  * Zookeeper read API: A read API at /api/cluster/zk/* to fetch raw ZK
> > data and view contents of a ZK directory.
> >
> >  * Admin UI: New panel with security info in admin UI's dashboard.
> >
> >  * Query DSL: Support for {param:ref} and {bool: {excludeTags:""}}
> >
> >  * Ref Guide: Major redesign of Solr's documentation.
> >
> >
> > Please read CHANGES.txt for a full list of new features and changes:
> >
> >
> >   
> >
> >
> > Solr 8.6.0 also includes features, optimizations  and bugfixes in the
> > corresponding Apache Lucene release:
> >
> >
> >   
> >
> >
> > Note: The Apache Software Foundation uses an extensive mirroring network
> > for
> >
> > distributing releases. It is possible that the mirror you are using may
> > not have
> >
> > replicated the release yet. If that is the case, please try another mirror.
> >
> > This also applies to Maven access.
> >

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11752) add gzip to jetty

[Matthew Sporleder (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597530#comment-16597530
 ] 

Matthew Sporleder commented on SOLR-11752:
--

why wouldn't it be?

> add gzip to jetty
> -
>
> Key: SOLR-11752
> URL: https://issues.apache.org/jira/browse/SOLR-11752
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server
>Affects Versions: master (8.0)
>    Reporter: Matthew Sporleder
>Priority: Trivial
>  Labels: jetty
> Attachments: SOLR-11752.patch, SOLR-11752.patch
>
>
> with a little bit of typing I am able to add gzip to my solr's jetty, which 
> is a big help for WAN access and completely out-of-band to solr, *and* only 
> happens if the client requests it so I think it is is a good default.
> I will just inline my code to this ticket:
> {code:java}
> #server/etc/jetty-gzip.xml
> #just download it from here: 
> http://grepcode.com/file/repo1.maven.org/maven2/org.eclipse.jetty/jetty-server/9.3.0.v20150612/etc/jetty-gzip.xml?av=f
> {code}
> {code:java}
> #server/modules/gzip.mod
> [depend]
> server
> [xml]
> etc/jetty-gzip.xml
> {code}
> This is where you might want to add an option, but the result should look 
> like this:
> {code:java}
> #bin/solr
> else
>   SOLR_JETTY_CONFIG+=("--module=http,gzip")
> fi
> {code}
> I can now do this:
> {code:java}
> curl -vvv --compressed localhost:8983/solr/ > /dev/null
> {code}
> With:
> {code:java}
> < Content-Encoding: gzip
> < Content-Length: 2890
> {code}
> Without:
> {code:java}
> < Content-Length: 13349
> {code}
> —
> A regular query:
>  With:
> {code:java}
> < Content-Encoding: gzip
> < Content-Length: 2876
> {code}
> Without:
> {code:java}
> < Content-Length: 17761
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11752) add gzip to jetty

[Matthew Sporleder (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479469#comment-16479469
 ] 

Matthew Sporleder commented on SOLR-11752:
--

so what's next for this issue? 

> add gzip to jetty
> -
>
> Key: SOLR-11752
> URL: https://issues.apache.org/jira/browse/SOLR-11752
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server
>Affects Versions: master (8.0)
>    Reporter: Matthew Sporleder
>Priority: Trivial
>  Labels: jetty
> Attachments: SOLR-11752.patch, SOLR-11752.patch
>
>
> with a little bit of typing I am able to add gzip to my solr's jetty, which 
> is a big help for WAN access and completely out-of-band to solr, *and* only 
> happens if the client requests it so I think it is is a good default.
> I will just inline my code to this ticket:
> {code:java}
> #server/etc/jetty-gzip.xml
> #just download it from here: 
> http://grepcode.com/file/repo1.maven.org/maven2/org.eclipse.jetty/jetty-server/9.3.0.v20150612/etc/jetty-gzip.xml?av=f
> {code}
> {code:java}
> #server/modules/gzip.mod
> [depend]
> server
> [xml]
> etc/jetty-gzip.xml
> {code}
> This is where you might want to add an option, but the result should look 
> like this:
> {code:java}
> #bin/solr
> else
>   SOLR_JETTY_CONFIG+=("--module=http,gzip")
> fi
> {code}
> I can now do this:
> {code:java}
> curl -vvv --compressed localhost:8983/solr/ > /dev/null
> {code}
> With:
> {code:java}
> < Content-Encoding: gzip
> < Content-Length: 2890
> {code}
> Without:
> {code:java}
> < Content-Length: 13349
> {code}
> —
> A regular query:
>  With:
> {code:java}
> < Content-Encoding: gzip
> < Content-Length: 2876
> {code}
> Without:
> {code:java}
> < Content-Length: 17761
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11752) add gzip to jetty

[Matthew Sporleder (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matthew Sporleder updated SOLR-11752:
-
Description: 
with a little bit of typing I am able to add gzip to my solr's jetty, which is 
a big help for WAN access and completely out-of-band to solr, *and* only 
happens if the client requests it so I think it is is a good default.

I will just inline my code to this ticket:
{code:java}
#server/etc/jetty-gzip.xml
#just download it from here: 
http://grepcode.com/file/repo1.maven.org/maven2/org.eclipse.jetty/jetty-server/9.3.0.v20150612/etc/jetty-gzip.xml?av=f

{code}
{code:java}
#server/modules/gzip.mod
[depend]
server

[xml]
etc/jetty-gzip.xml
{code}
This is where you might want to add an option, but the result should look like 
this:
{code:java}
#bin/solr
else
  SOLR_JETTY_CONFIG+=("--module=http,gzip")
fi
{code}
I can now do this:
{code:java}
curl -vvv --compressed localhost:8983/solr/ > /dev/null
{code}
With:
{code:java}
< Content-Encoding: gzip
< Content-Length: 2890
{code}
Without:
{code:java}
< Content-Length: 13349
{code}
—

A regular query:
 With:
{code:java}
< Content-Encoding: gzip
< Content-Length: 2876
{code}
Without:
{code:java}
< Content-Length: 17761
{code}

  was:
with a little bit of typing I am able to add gzip to my solr's jetty, which is 
a big help for SAN access and completely out-of-band to solr, *and* only 
happens if the client requests it so I think it is is a good default.

I will just inline my code to this ticket:

{code}
#server/etc/jetty-gzip.xml
#just download it from here: 
http://grepcode.com/file/repo1.maven.org/maven2/org.eclipse.jetty/jetty-server/9.3.0.v20150612/etc/jetty-gzip.xml?av=f

{code}

{code}
#server/modules/gzip.mod
[depend]
server

[xml]
etc/jetty-gzip.xml
{code}

This is where you might want to add an option, but the result should look like 
this:
{code}
#bin/solr
else
  SOLR_JETTY_CONFIG+=("--module=http,gzip")
fi
{code}

I can now do this:
{code}
curl -vvv --compressed localhost:8983/solr/ > /dev/null
{code}

With:
{code}
< Content-Encoding: gzip
< Content-Length: 2890
{code}

Without:
{code}
< Content-Length: 13349
{code}

---

A regular query:
With:
{code}
< Content-Encoding: gzip
< Content-Length: 2876
{code}

Without:
{code}
< Content-Length: 17761
{code}


> add gzip to jetty
> -
>
> Key: SOLR-11752
> URL: https://issues.apache.org/jira/browse/SOLR-11752
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>      Components: Server
>Affects Versions: master (8.0)
>Reporter: Matthew Sporleder
>Priority: Trivial
>  Labels: jetty
> Attachments: SOLR-11752.patch, SOLR-11752.patch
>
>
> with a little bit of typing I am able to add gzip to my solr's jetty, which 
> is a big help for WAN access and completely out-of-band to solr, *and* only 
> happens if the client requests it so I think it is is a good default.
> I will just inline my code to this ticket:
> {code:java}
> #server/etc/jetty-gzip.xml
> #just download it from here: 
> http://grepcode.com/file/repo1.maven.org/maven2/org.eclipse.jetty/jetty-server/9.3.0.v20150612/etc/jetty-gzip.xml?av=f
> {code}
> {code:java}
> #server/modules/gzip.mod
> [depend]
> server
> [xml]
> etc/jetty-gzip.xml
> {code}
> This is where you might want to add an option, but the result should look 
> like this:
> {code:java}
> #bin/solr
> else
>   SOLR_JETTY_CONFIG+=("--module=http,gzip")
> fi
> {code}
> I can now do this:
> {code:java}
> curl -vvv --compressed localhost:8983/solr/ > /dev/null
> {code}
> With:
> {code:java}
> < Content-Encoding: gzip
> < Content-Length: 2890
> {code}
> Without:
> {code:java}
> < Content-Length: 13349
> {code}
> —
> A regular query:
>  With:
> {code:java}
> < Content-Encoding: gzip
> < Content-Length: 2876
> {code}
> Without:
> {code:java}
> < Content-Length: 17761
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-11752) add gzip to jetty

[Matthew Sporleder (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16476098#comment-16476098
 ] 

Matthew Sporleder edited comment on SOLR-11752 at 5/15/18 5:05 PM:
---

the solr web app gui runs faster with compression working as well.  There is no 
reason to limit usage of gzip since it only occurs when a client explicitly 
asks for it.


was (Author: msporleder):
the solr web app gui runs faster with compression working as well.  There is no 
reason to limit usage of gzip on the streams since it only occurs when a client 
explicitly asks for it.

> add gzip to jetty
> -
>
> Key: SOLR-11752
> URL: https://issues.apache.org/jira/browse/SOLR-11752
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server
>Affects Versions: master (8.0)
>    Reporter: Matthew Sporleder
>Priority: Trivial
>  Labels: jetty
> Attachments: SOLR-11752.patch, SOLR-11752.patch
>
>
> with a little bit of typing I am able to add gzip to my solr's jetty, which 
> is a big help for SAN access and completely out-of-band to solr, *and* only 
> happens if the client requests it so I think it is is a good default.
> I will just inline my code to this ticket:
> {code}
> #server/etc/jetty-gzip.xml
> #just download it from here: 
> http://grepcode.com/file/repo1.maven.org/maven2/org.eclipse.jetty/jetty-server/9.3.0.v20150612/etc/jetty-gzip.xml?av=f
> {code}
> {code}
> #server/modules/gzip.mod
> [depend]
> server
> [xml]
> etc/jetty-gzip.xml
> {code}
> This is where you might want to add an option, but the result should look 
> like this:
> {code}
> #bin/solr
> else
>   SOLR_JETTY_CONFIG+=("--module=http,gzip")
> fi
> {code}
> I can now do this:
> {code}
> curl -vvv --compressed localhost:8983/solr/ > /dev/null
> {code}
> With:
> {code}
> < Content-Encoding: gzip
> < Content-Length: 2890
> {code}
> Without:
> {code}
> < Content-Length: 13349
> {code}
> ---
> A regular query:
> With:
> {code}
> < Content-Encoding: gzip
> < Content-Length: 2876
> {code}
> Without:
> {code}
> < Content-Length: 17761
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11752) add gzip to jetty

[Matthew Sporleder (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16476098#comment-16476098
 ] 

Matthew Sporleder commented on SOLR-11752:
--

the solr web app gui runs faster with compression working as well.  There is no 
reason to limit usage of gzip on the streams since it only occurs when a client 
explicitly asks for it.

> add gzip to jetty
> -
>
> Key: SOLR-11752
> URL: https://issues.apache.org/jira/browse/SOLR-11752
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server
>Affects Versions: master (8.0)
>    Reporter: Matthew Sporleder
>Priority: Trivial
>  Labels: jetty
> Attachments: SOLR-11752.patch, SOLR-11752.patch
>
>
> with a little bit of typing I am able to add gzip to my solr's jetty, which 
> is a big help for SAN access and completely out-of-band to solr, *and* only 
> happens if the client requests it so I think it is is a good default.
> I will just inline my code to this ticket:
> {code}
> #server/etc/jetty-gzip.xml
> #just download it from here: 
> http://grepcode.com/file/repo1.maven.org/maven2/org.eclipse.jetty/jetty-server/9.3.0.v20150612/etc/jetty-gzip.xml?av=f
> {code}
> {code}
> #server/modules/gzip.mod
> [depend]
> server
> [xml]
> etc/jetty-gzip.xml
> {code}
> This is where you might want to add an option, but the result should look 
> like this:
> {code}
> #bin/solr
> else
>   SOLR_JETTY_CONFIG+=("--module=http,gzip")
> fi
> {code}
> I can now do this:
> {code}
> curl -vvv --compressed localhost:8983/solr/ > /dev/null
> {code}
> With:
> {code}
> < Content-Encoding: gzip
> < Content-Length: 2890
> {code}
> Without:
> {code}
> < Content-Length: 13349
> {code}
> ---
> A regular query:
> With:
> {code}
> < Content-Encoding: gzip
> < Content-Length: 2876
> {code}
> Without:
> {code}
> < Content-Length: 17761
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11752) add gzip to jetty

[Matthew Sporleder (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16475574#comment-16475574
 ] 

Matthew Sporleder commented on SOLR-11752:
--

the gzip stuff is from jetty directly so if you want to maintain that diff 
going forward and forever, feel free to strip the mime stuff out.

I disagree concerning 10999 and already have Jan H on this ticket agreeing and 
included him in the CHANGES as co-contributor.

> add gzip to jetty
> -
>
> Key: SOLR-11752
> URL: https://issues.apache.org/jira/browse/SOLR-11752
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server
>Affects Versions: master (8.0)
>    Reporter: Matthew Sporleder
>Priority: Trivial
>  Labels: jetty
> Attachments: SOLR-11752.patch, SOLR-11752.patch
>
>
> with a little bit of typing I am able to add gzip to my solr's jetty, which 
> is a big help for SAN access and completely out-of-band to solr, *and* only 
> happens if the client requests it so I think it is is a good default.
> I will just inline my code to this ticket:
> {code}
> #server/etc/jetty-gzip.xml
> #just download it from here: 
> http://grepcode.com/file/repo1.maven.org/maven2/org.eclipse.jetty/jetty-server/9.3.0.v20150612/etc/jetty-gzip.xml?av=f
> {code}
> {code}
> #server/modules/gzip.mod
> [depend]
> server
> [xml]
> etc/jetty-gzip.xml
> {code}
> This is where you might want to add an option, but the result should look 
> like this:
> {code}
> #bin/solr
> else
>   SOLR_JETTY_CONFIG+=("--module=http,gzip")
> fi
> {code}
> I can now do this:
> {code}
> curl -vvv --compressed localhost:8983/solr/ > /dev/null
> {code}
> With:
> {code}
> < Content-Encoding: gzip
> < Content-Length: 2890
> {code}
> Without:
> {code}
> < Content-Length: 13349
> {code}
> ---
> A regular query:
> With:
> {code}
> < Content-Encoding: gzip
> < Content-Length: 2876
> {code}
> Without:
> {code}
> < Content-Length: 17761
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11752) add gzip to jetty

[Matthew Sporleder (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474223#comment-16474223
 ] 

Matthew Sporleder commented on SOLR-11752:
--

one more try

> add gzip to jetty
> -
>
> Key: SOLR-11752
> URL: https://issues.apache.org/jira/browse/SOLR-11752
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server
>Affects Versions: master (8.0)
>    Reporter: Matthew Sporleder
>Priority: Trivial
>  Labels: jetty
> Attachments: SOLR-11752.patch, SOLR-11752.patch
>
>
> with a little bit of typing I am able to add gzip to my solr's jetty, which 
> is a big help for SAN access and completely out-of-band to solr, *and* only 
> happens if the client requests it so I think it is is a good default.
> I will just inline my code to this ticket:
> {code}
> #server/etc/jetty-gzip.xml
> #just download it from here: 
> http://grepcode.com/file/repo1.maven.org/maven2/org.eclipse.jetty/jetty-server/9.3.0.v20150612/etc/jetty-gzip.xml?av=f
> {code}
> {code}
> #server/modules/gzip.mod
> [depend]
> server
> [xml]
> etc/jetty-gzip.xml
> {code}
> This is where you might want to add an option, but the result should look 
> like this:
> {code}
> #bin/solr
> else
>   SOLR_JETTY_CONFIG+=("--module=http,gzip")
> fi
> {code}
> I can now do this:
> {code}
> curl -vvv --compressed localhost:8983/solr/ > /dev/null
> {code}
> With:
> {code}
> < Content-Encoding: gzip
> < Content-Length: 2890
> {code}
> Without:
> {code}
> < Content-Length: 13349
> {code}
> ---
> A regular query:
> With:
> {code}
> < Content-Encoding: gzip
> < Content-Length: 2876
> {code}
> Without:
> {code}
> < Content-Length: 17761
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11752) add gzip to jetty

[Matthew Sporleder (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matthew Sporleder updated SOLR-11752:
-
Attachment: SOLR-11752.patch

> add gzip to jetty
> -
>
> Key: SOLR-11752
> URL: https://issues.apache.org/jira/browse/SOLR-11752
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server
>Affects Versions: master (8.0)
>    Reporter: Matthew Sporleder
>Priority: Trivial
>  Labels: jetty
> Attachments: SOLR-11752.patch, SOLR-11752.patch
>
>
> with a little bit of typing I am able to add gzip to my solr's jetty, which 
> is a big help for SAN access and completely out-of-band to solr, *and* only 
> happens if the client requests it so I think it is is a good default.
> I will just inline my code to this ticket:
> {code}
> #server/etc/jetty-gzip.xml
> #just download it from here: 
> http://grepcode.com/file/repo1.maven.org/maven2/org.eclipse.jetty/jetty-server/9.3.0.v20150612/etc/jetty-gzip.xml?av=f
> {code}
> {code}
> #server/modules/gzip.mod
> [depend]
> server
> [xml]
> etc/jetty-gzip.xml
> {code}
> This is where you might want to add an option, but the result should look 
> like this:
> {code}
> #bin/solr
> else
>   SOLR_JETTY_CONFIG+=("--module=http,gzip")
> fi
> {code}
> I can now do this:
> {code}
> curl -vvv --compressed localhost:8983/solr/ > /dev/null
> {code}
> With:
> {code}
> < Content-Encoding: gzip
> < Content-Length: 2890
> {code}
> Without:
> {code}
> < Content-Length: 13349
> {code}
> ---
> A regular query:
> With:
> {code}
> < Content-Encoding: gzip
> < Content-Length: 2876
> {code}
> Without:
> {code}
> < Content-Length: 17761
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11752) add gzip to jetty

[Matthew Sporleder (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matthew Sporleder updated SOLR-11752:
-
Attachment: (was: SOLR-11752.patch)

> add gzip to jetty
> -
>
> Key: SOLR-11752
> URL: https://issues.apache.org/jira/browse/SOLR-11752
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server
>Affects Versions: master (8.0)
>    Reporter: Matthew Sporleder
>Priority: Trivial
>  Labels: jetty
> Attachments: SOLR-11752.patch, SOLR-11752.patch
>
>
> with a little bit of typing I am able to add gzip to my solr's jetty, which 
> is a big help for SAN access and completely out-of-band to solr, *and* only 
> happens if the client requests it so I think it is is a good default.
> I will just inline my code to this ticket:
> {code}
> #server/etc/jetty-gzip.xml
> #just download it from here: 
> http://grepcode.com/file/repo1.maven.org/maven2/org.eclipse.jetty/jetty-server/9.3.0.v20150612/etc/jetty-gzip.xml?av=f
> {code}
> {code}
> #server/modules/gzip.mod
> [depend]
> server
> [xml]
> etc/jetty-gzip.xml
> {code}
> This is where you might want to add an option, but the result should look 
> like this:
> {code}
> #bin/solr
> else
>   SOLR_JETTY_CONFIG+=("--module=http,gzip")
> fi
> {code}
> I can now do this:
> {code}
> curl -vvv --compressed localhost:8983/solr/ > /dev/null
> {code}
> With:
> {code}
> < Content-Encoding: gzip
> < Content-Length: 2890
> {code}
> Without:
> {code}
> < Content-Length: 13349
> {code}
> ---
> A regular query:
> With:
> {code}
> < Content-Encoding: gzip
> < Content-Length: 2876
> {code}
> Without:
> {code}
> < Content-Length: 17761
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11752) add gzip to jetty

[Matthew Sporleder (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matthew Sporleder updated SOLR-11752:
-
Attachment: (was: SOLR-11752.patch)

> add gzip to jetty
> -
>
> Key: SOLR-11752
> URL: https://issues.apache.org/jira/browse/SOLR-11752
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server
>Affects Versions: master (8.0)
>    Reporter: Matthew Sporleder
>Priority: Trivial
>  Labels: jetty
> Attachments: SOLR-11752.patch, SOLR-11752.patch
>
>
> with a little bit of typing I am able to add gzip to my solr's jetty, which 
> is a big help for SAN access and completely out-of-band to solr, *and* only 
> happens if the client requests it so I think it is is a good default.
> I will just inline my code to this ticket:
> {code}
> #server/etc/jetty-gzip.xml
> #just download it from here: 
> http://grepcode.com/file/repo1.maven.org/maven2/org.eclipse.jetty/jetty-server/9.3.0.v20150612/etc/jetty-gzip.xml?av=f
> {code}
> {code}
> #server/modules/gzip.mod
> [depend]
> server
> [xml]
> etc/jetty-gzip.xml
> {code}
> This is where you might want to add an option, but the result should look 
> like this:
> {code}
> #bin/solr
> else
>   SOLR_JETTY_CONFIG+=("--module=http,gzip")
> fi
> {code}
> I can now do this:
> {code}
> curl -vvv --compressed localhost:8983/solr/ > /dev/null
> {code}
> With:
> {code}
> < Content-Encoding: gzip
> < Content-Length: 2890
> {code}
> Without:
> {code}
> < Content-Length: 13349
> {code}
> ---
> A regular query:
> With:
> {code}
> < Content-Encoding: gzip
> < Content-Length: 2876
> {code}
> Without:
> {code}
> < Content-Length: 17761
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11752) add gzip to jetty

[Matthew Sporleder (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473215#comment-16473215
 ] 

Matthew Sporleder commented on SOLR-11752:
--

I've updated the patch from master and moved the CHANGES entry into 8.0 (I 
don't know if this is accurate; maybe the message should be part of the commit 
or something?)

 

I didn't change the jetty stuff, just re-did the shell script parts.

> add gzip to jetty
> -
>
> Key: SOLR-11752
> URL: https://issues.apache.org/jira/browse/SOLR-11752
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server
>Affects Versions: master (8.0)
>    Reporter: Matthew Sporleder
>Priority: Trivial
>  Labels: jetty
> Attachments: SOLR-11752.patch, SOLR-11752.patch, SOLR-11752.patch
>
>
> with a little bit of typing I am able to add gzip to my solr's jetty, which 
> is a big help for SAN access and completely out-of-band to solr, *and* only 
> happens if the client requests it so I think it is is a good default.
> I will just inline my code to this ticket:
> {code}
> #server/etc/jetty-gzip.xml
> #just download it from here: 
> http://grepcode.com/file/repo1.maven.org/maven2/org.eclipse.jetty/jetty-server/9.3.0.v20150612/etc/jetty-gzip.xml?av=f
> {code}
> {code}
> #server/modules/gzip.mod
> [depend]
> server
> [xml]
> etc/jetty-gzip.xml
> {code}
> This is where you might want to add an option, but the result should look 
> like this:
> {code}
> #bin/solr
> else
>   SOLR_JETTY_CONFIG+=("--module=http,gzip")
> fi
> {code}
> I can now do this:
> {code}
> curl -vvv --compressed localhost:8983/solr/ > /dev/null
> {code}
> With:
> {code}
> < Content-Encoding: gzip
> < Content-Length: 2890
> {code}
> Without:
> {code}
> < Content-Length: 13349
> {code}
> ---
> A regular query:
> With:
> {code}
> < Content-Encoding: gzip
> < Content-Length: 2876
> {code}
> Without:
> {code}
> < Content-Length: 17761
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11752) add gzip to jetty

[Matthew Sporleder (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matthew Sporleder updated SOLR-11752:
-
Attachment: SOLR-11752.patch

> add gzip to jetty
> -
>
> Key: SOLR-11752
> URL: https://issues.apache.org/jira/browse/SOLR-11752
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server
>Affects Versions: master (8.0)
>    Reporter: Matthew Sporleder
>Priority: Trivial
>  Labels: jetty
> Attachments: SOLR-11752.patch, SOLR-11752.patch, SOLR-11752.patch
>
>
> with a little bit of typing I am able to add gzip to my solr's jetty, which 
> is a big help for SAN access and completely out-of-band to solr, *and* only 
> happens if the client requests it so I think it is is a good default.
> I will just inline my code to this ticket:
> {code}
> #server/etc/jetty-gzip.xml
> #just download it from here: 
> http://grepcode.com/file/repo1.maven.org/maven2/org.eclipse.jetty/jetty-server/9.3.0.v20150612/etc/jetty-gzip.xml?av=f
> {code}
> {code}
> #server/modules/gzip.mod
> [depend]
> server
> [xml]
> etc/jetty-gzip.xml
> {code}
> This is where you might want to add an option, but the result should look 
> like this:
> {code}
> #bin/solr
> else
>   SOLR_JETTY_CONFIG+=("--module=http,gzip")
> fi
> {code}
> I can now do this:
> {code}
> curl -vvv --compressed localhost:8983/solr/ > /dev/null
> {code}
> With:
> {code}
> < Content-Encoding: gzip
> < Content-Length: 2890
> {code}
> Without:
> {code}
> < Content-Length: 13349
> {code}
> ---
> A regular query:
> With:
> {code}
> < Content-Encoding: gzip
> < Content-Length: 2876
> {code}
> Without:
> {code}
> < Content-Length: 17761
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-9613) core or collection -> dataimport dangerous default

[Matthew Sporleder (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matthew Sporleder resolved SOLR-9613.
-
   Resolution: Duplicate
Fix Version/s: 7.3

SOLR-11933 fixed this issue

> core or collection -> dataimport dangerous default
> --
>
> Key: SOLR-9613
> URL: https://issues.apache.org/jira/browse/SOLR-9613
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Admin UI
>    Reporter: Matthew Sporleder
>Assignee: Alexandre Rafalovitch
>Priority: Major
> Fix For: 7.3
>
>
> When browsing to dataimport in the web gui and selecting "delta-import" from 
> the drop down, the "full-import" checkbox selections stay checked, including 
> "clean", which is very dangerous for a delta-import, as it deletes most of 
> your data!
> a js event to clear those checkboxes on selection from that dropdown would 
> save a lot of accidental anguish.
> {code}
> var foo = document.getElementById("command")
> function bar() { document.getElementById("clean").checked = false; }
> foo.onchange = function() {
>   if (foo.value == "delta-import")
> {
>   bar();
> }
> };
> {code}
> or whatever



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11752) add gzip to jetty

[Matthew Sporleder (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matthew Sporleder updated SOLR-11752:
-
Attachment: SOLR-11752.patch

> add gzip to jetty
> -
>
> Key: SOLR-11752
> URL: https://issues.apache.org/jira/browse/SOLR-11752
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server
>Affects Versions: master (8.0)
>    Reporter: Matthew Sporleder
>Priority: Trivial
>  Labels: jetty
> Attachments: SOLR-11752.patch
>
>
> with a little bit of typing I am able to add gzip to my solr's jetty, which 
> is a big help for SAN access and completely out-of-band to solr, *and* only 
> happens if the client requests it so I think it is is a good default.
> I will just inline my code to this ticket:
> {code}
> #server/etc/jetty-gzip.xml
> #just download it from here: 
> http://grepcode.com/file/repo1.maven.org/maven2/org.eclipse.jetty/jetty-server/9.3.0.v20150612/etc/jetty-gzip.xml?av=f
> {code}
> {code}
> #server/modules/gzip.mod
> [depend]
> server
> [xml]
> etc/jetty-gzip.xml
> {code}
> This is where you might want to add an option, but the result should look 
> like this:
> {code}
> #bin/solr
> else
>   SOLR_JETTY_CONFIG+=("--module=http,gzip")
> fi
> {code}
> I can now do this:
> {code}
> curl -vvv --compressed localhost:8983/solr/ > /dev/null
> {code}
> With:
> {code}
> < Content-Encoding: gzip
> < Content-Length: 2890
> {code}
> Without:
> {code}
> < Content-Length: 13349
> {code}
> ---
> A regular query:
> With:
> {code}
> < Content-Encoding: gzip
> < Content-Length: 2876
> {code}
> Without:
> {code}
> < Content-Length: 17761
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-11752) add gzip to jetty

[Matthew Sporleder (JIRA)]

Matthew Sporleder created SOLR-11752:


 Summary: add gzip to jetty
 Key: SOLR-11752
 URL: https://issues.apache.org/jira/browse/SOLR-11752
 Project: Solr
  Issue Type: New Feature
  Security Level: Public (Default Security Level. Issues are Public)
  Components: Server
Reporter: Matthew Sporleder
Priority: Trivial


with a little bit of typing I am able to add gzip to my solr's jetty, which is 
a big help for SAN access and completely out-of-band to solr, *and* only 
happens if the client requests it so I think it is is a good default.

I will just inline my code to this ticket:

{code}
#server/etc/jetty-gzip.xml
#just download it from here: 
http://grepcode.com/file/repo1.maven.org/maven2/org.eclipse.jetty/jetty-server/9.3.0.v20150612/etc/jetty-gzip.xml?av=f

{code}

{code}
#server/modules/gzip.mod
[depend]
server

[xml]
etc/jetty-gzip.xml
{code}

This is where you might want to add an option, but the result should look like 
this:
{code}
#bin/solr
else
  SOLR_JETTY_CONFIG+=("--module=http,gzip")
fi
{code}

I can now do this:
{code}
curl -vvv --compressed localhost:8983/solr/ > /dev/null
{code}

With:
{code}
< Content-Encoding: gzip
< Content-Length: 2890
{code}

Without:
{code}
< Content-Length: 13349
{code}

---

A regular query:
With:
{code}
< Content-Encoding: gzip
< Content-Length: 2876
{code}

Without:
{code}
< Content-Length: 17761
{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10130) Serious performance degradation in Solr 6.4.1 due to the new metrics collection

[Matthew Sporleder (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15987240#comment-15987240
 ] 

Matthew Sporleder commented on SOLR-10130:
--

Both are running java version "1.8.0_45" sun jdk

> Serious performance degradation in Solr 6.4.1 due to the new metrics 
> collection
> ---
>
> Key: SOLR-10130
> URL: https://issues.apache.org/jira/browse/SOLR-10130
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Affects Versions: 6.4, 6.4.1
> Environment: Centos 7, OpenJDK 1.8.0 update 111
>Reporter: Ere Maijala
>Assignee: Andrzej Bialecki 
>Priority: Blocker
>  Labels: perfomance
> Fix For: 6.4.2, master (7.0)
>
> Attachments: SOLR-10130.patch, SOLR-10130.patch, 
> solr-8983-console-f1.log
>
>
> We've stumbled on serious performance issues after upgrading to Solr 6.4.1. 
> Looks like the new metrics collection system in MetricsDirectoryFactory is 
> causing a major slowdown. This happens with an index configuration that, as 
> far as I can see, has no metrics specific configuration and uses 
> luceneMatchVersion 5.5.0. In practice a moderate load will completely bog 
> down the server with Solr threads constantly using up all CPU (600% on 6 core 
> machine) capacity with a load that normally  where we normally see an average 
> load of < 50%.
> I took stack traces (I'll attach them) and noticed that the threads are 
> spending time in com.codahale.metrics.Meter.mark. I tested building Solr 
> 6.4.1 with the metrics collection disabled in MetricsDirectoryFactory getByte 
> and getBytes methods and was unable to reproduce the issue.
> As far as I can see there are several issues:
> 1. Collecting metrics on every single byte read is slow.
> 2. Having it enabled by default is not a good idea.
> 3. The comment "enable coarse-grained metrics by default" at 
> https://github.com/apache/lucene-solr/blob/branch_6x/solr/core/src/java/org/apache/solr/update/SolrIndexConfig.java#L104
>  implies that only coarse-grained metrics should be enabled by default, and 
> this contradicts with collecting metrics on every single byte read.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10130) Serious performance degradation in Solr 6.4.1 due to the new metrics collection

[Matthew Sporleder (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15987079#comment-15987079
 ] 

Matthew Sporleder commented on SOLR-10130:
--

Not sure I have the tooling right now for a full drill down, but here are some 
examples of a thread dump:
{code}
"qtp968514068-37953" - Thread t@37953
   java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <2e101d2c> (a sun.nio.ch.Util$2)
- locked <59c1f901> (a java.util.Collections$UnmodifiableSet)
- locked <54c6a926> (a sun.nio.ch.EPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:745)

   Locked ownable synchronizers:
- None

"qtp968514068-37952" - Thread t@37952
   java.lang.Thread.State: TIMED_WAITING
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <2d78e562> (a 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
{code}

Most are in that TIMED_WAITING and most CPU time is spend on 
org.eclipse.jetty.util.BlockingArrayQueue.poll according to visualvm

> Serious performance degradation in Solr 6.4.1 due to the new metrics 
> collection
> ---
>
> Key: SOLR-10130
> URL: https://issues.apache.org/jira/browse/SOLR-10130
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Affects Versions: 6.4, 6.4.1
> Environment: Centos 7, OpenJDK 1.8.0 update 111
>Reporter: Ere Maijala
>Assignee: Andrzej Bialecki 
>Priority: Blocker
>  Labels: perfomance
> Fix For: 6.4.2, master (7.0)
>
> Attachments: SOLR-10130.patch, SOLR-10130.patch, 
> solr-8983-console-f1.log
>
>
> We've stumbled on serious performance issues after upgrading to Solr 6.4.1. 
> Looks like the new metrics collection system in MetricsDirectoryFactory is 
> causing a major slowdown. This happens with an index configuration that, as 
> far as I can see, has no metrics specific configuration and uses 
> luceneMatchVersion 5.5.0. In practice a moderate load will completely bog 
> down the server with Solr threads constantly using up all CPU (600% on 6 core 
> machine) capacity with a load that normally  where we normally see an average 
> load of < 50%.
> I took stack traces (I'll attach them) and noticed that the threads are 
> spending time in com.codahale.metrics.Meter.mark. I tested building Solr 
> 6.4.1 with the metrics collection disabled in MetricsDirectoryFactory getByte 
> and getBytes methods and was unable to reproduce the issue.
> As far as I can see there are several issues:
> 1. Collecting metrics on every single byte read is slow.
> 2. Having it enabled by default is not a good idea.
> 3. The comment "enable coarse-grained metrics by default" at 
> https://github.com/apache/lucene-solr/blob/branch_6x/solr/core/src/java/org/apache/solr/update/SolrIndexConfig.java#L104
>  implies that only coarse-gra

[jira] [Created] (SOLR-9613) core or collection -> dataimport dangerous default

[Matthew Sporleder (JIRA)]

Matthew Sporleder created SOLR-9613:
---

 Summary: core or collection -> dataimport dangerous default
 Key: SOLR-9613
 URL: https://issues.apache.org/jira/browse/SOLR-9613
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
  Components: web gui
Reporter: Matthew Sporleder


When browsing to dataimport in the web gui and selecting "delta-import" from 
the drop down, the "full-import" checkbox selections stay checked, including 
"clean", which is very dangerous for a delta-import, as it deletes most of your 
data!

a js event to clear those checkboxes on selection from that dropdown would save 
a lot of accidental anguish.

{code}
var foo = document.getElementById("command")
function bar() { document.getElementById("clean").checked = false; }
foo.onchange = function() {
  if (foo.value == "delta-import")
{
  bar();
}
};
{code}

or whatever



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-5070) add mbeans for everything in /solr/admin/cores?wt=jsonindexInfo=true

[Matthew Sporleder (JIRA)]

Matthew Sporleder created SOLR-5070:
---

 Summary: add mbeans for everything in 
/solr/admin/cores?wt=jsonindexInfo=true
 Key: SOLR-5070
 URL: https://issues.apache.org/jira/browse/SOLR-5070
 Project: Solr
  Issue Type: Improvement
Reporter: Matthew Sporleder


for solr4,
JMX should have everything in /solr/admin/cores?wt=jsonindexInfo=true

One major omission is: lastModified

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] Created: (SOLR-2344) JMX not reliable - MBeans deleted

[Matthew Sporleder (JIRA)]

JMX not reliable - MBeans deleted
-

 Key: SOLR-2344
 URL: https://issues.apache.org/jira/browse/SOLR-2344
 Project: Solr
  Issue Type: Bug
 Environment: linux 64bit x86_64
java version 1.6.0_13
Reporter: Matthew Sporleder


I am using JMX to monitor my replication status and am finding that my
MBeans are disappearing.  I turned on debugging for JMX and found that
solr seems to be deleting the mbeans.

This appears to be a triggered event, but I am not finding any clues as to why 
this is happening.

Is this a bug?  Some trace info is below..

here's me reading the mbean successfully:
Jan 27, 2011 5:00:02 PM ServerCommunicatorAdmin reqIncoming
FINER: Receive a new request.
Jan 27, 2011 5:00:02 PM DefaultMBeanServerInterceptor getAttribute
FINER: Attribute= indexReplicatedAt, obj=
solr/myapp-core:type=/replication,id=org.apache.solr.handler.ReplicationHandler
Jan 27, 2011 5:00:02 PM Repository retrieve
FINER: 
name=solr/myapp-core:type=/replication,id=org.apache.solr.handler.ReplicationHandler
Jan 27, 2011 5:00:02 PM ServerCommunicatorAdmin reqIncoming
FINER: Finish a request.


a little while later it removes the mbean from the PM Repository
(whatever that is) and then re-adds it:
FINER: Send create notification of object
solr/myapp-core:id=org.apache.solr.handler.component.SearchHandler,type=atlas
Jan 27, 2011 5:16:14 PM DefaultMBeanServerInterceptor sendNotification
FINER: JMX.mbean.registered
solr/myapp-core:type=atlas,id=org.apache.solr.handler.component.SearchHandler
Jan 27, 2011 5:16:14 PM Repository contains
FINER: 
name=solr/myapp-core:type=/replication,id=org.apache.solr.handler.ReplicationHandler
Jan 27, 2011 5:16:14 PM Repository retrieve
FINER: 
name=solr/myapp-core:type=/replication,id=org.apache.solr.handler.ReplicationHandler
Jan 27, 2011 5:16:14 PM Repository remove
FINER: 
name=solr/myapp-core:type=/replication,id=org.apache.solr.handler.ReplicationHandler
Jan 27, 2011 5:16:14 PM DefaultMBeanServerInterceptor unregisterMBean
FINER: Send delete notification of object
solr/myapp-core:id=org.apache.solr.handler.ReplicationHandler,type=/replication
Jan 27, 2011 5:16:14 PM DefaultMBeanServerInterceptor sendNotification
FINER: JMX.mbean.unregistered
solr/myapp-core:type=/replication,id=org.apache.solr.handler.ReplicationHandler
Jan 27, 2011 5:16:14 PM DefaultMBeanServerInterceptor registerMBean
FINER: ObjectName =
solr/myapp-core:type=/replication,id=org.apache.solr.handler.ReplicationHandler
Jan 27, 2011 5:16:14 PM Repository addMBean
FINER: 
name=solr/myapp-core:type=/replication,id=org.apache.solr.handler.ReplicationHandler
Jan 27, 2011 5:16:14 PM DefaultMBeanServerInterceptor addObject
FINER: Send create notification of object
solr/myapp-core:id=org.apache.solr.handler.ReplicationHandler,type=/replication
Jan 27, 2011 5:16:14 PM DefaultMBeanServerInterceptor sendNotification
FINER: JMX.mbean.registered
solr/myapp-core:type=/replication,id=org.apache.solr.handler.ReplicationHandler


And after a tons of messages but still in the same second it does:
Jan 27, 2011 5:16:14 PM Repository contains
FINER: 
name=solr/myapp-core:type=org.apache.solr.handler.ReplicationHandler,id=org.apache.solr.handler.ReplicationHandler
Jan 27, 2011 5:16:14 PM Repository retrieve
FINER: 
name=solr/myapp-core:type=org.apache.solr.handler.ReplicationHandler,id=org.apache.solr.handler.ReplicationHandler
Jan 27, 2011 5:16:14 PM Repository removeFINER:
name=solr/myapp-core:type=org.apache.solr.handler.ReplicationHandler,id=org.apache.solr.handler.ReplicationHandler
Jan 27, 2011 5:16:14 PM DefaultMBeanServerInterceptor unregisterMBean
FINER: Send delete notification of object
solr/myapp-core:id=org.apache.solr.handler.ReplicationHandler,type=org.apache.solr.handler.ReplicationHandler
Jan 27, 2011 5:16:14 PM DefaultMBeanServerInterceptor sendNotification
FINER: JMX.mbean.unregistered
solr/myapp-core:type=org.apache.solr.handler.ReplicationHandler,id=org.apache.solr.handler.ReplicationHandler
Jan 27, 2011 5:16:14 PM DefaultMBeanServerInterceptor registerMBean
FINER: ObjectName =
solr/myapp-core:type=org.apache.solr.handler.ReplicationHandler,id=org.apache.solr.handler.ReplicationHandlerJan
27, 2011 5:16:14 PM Repository addMBean
FINER: 
name=solr/myapp-core:type=org.apache.solr.handler.ReplicationHandler,id=org.apache.solr.handler.ReplicationHandler
Jan 27, 2011 5:16:14 PM DefaultMBeanServerInterceptor addObjectFINER:
Send create notification of object
solr/myapp-core:id=org.apache.solr.handler.ReplicationHandler,type=org.apache.solr.handler.ReplicationHandler
Jan 27, 2011 5:16:14 PM DefaultMBeanServerInterceptor
sendNotificationFINER: JMX.mbean.registered
solr/myapp-core:type=org.apache.solr.handler.ReplicationHandler,id=org.apache.solr.handler.ReplicationHandler


And then I don't know what this is about but it removes the bean again:
Jan 27, 2011 5:16:15 PM Repository contains
FINER: 
name=solr/myapp-core:type