Re: [VOTE] Release Lucene/Solr 8.5.2 RC1

[Andras Salamon]

Hi,

SUCCESS! [1:50:12.408588]

+1 (non-binding)

Andras

On Thu, May 21, 2020 at 2:06 AM Jan Høydahl  wrote:

> +1
>
> SUCCESS! [1:03:16.225214]
> I did no manual tests, just ran smoketester on macOS
>
> Jan
>
> > 20. mai 2020 kl. 19:58 skrev Mike Drob :
> >
> > Devs,
> >
> > Please vote for release candidate 1 for Lucene/Solr 8.5.2
> >
> > The artifacts can be downloaded from:
> >
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.5.2-RC1-rev384dadd9141cec3f848d8c416315dc2384749814
> >
> > You can run the smoke tester directly with this command:
> >
> > python3 -u dev-tools/scripts/smokeTestRelease.py \
> >
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.5.2-RC1-rev384dadd9141cec3f848d8c416315dc2384749814
> >
> > The vote will be open until 2020-05-26 18:00 UTC (extended deadline due
> to multiple holidays in the next 72 hours)
> >
> > [ ] +1  approve
> > [ ] +0  no opinion
> > [ ] -1  disapprove (and reason why)
> >
> > Here is my +1 (non-binding)
> >
> > Mike
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > For additional commands, e-mail: dev-h...@lucene.apache.org
> >
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: Getting warnings out

[Andras Salamon]

Hi,

We have quite a few warnings, it would be difficult to fix them at once.
Checking one directory (or warning type) and handling 10-20 warnings at the
same time seems more reasonable.

There are two umbrella jiras for that:
https://issues.apache.org/jira/browse/SOLR-10778 and
https://issues.apache.org/jira/browse/LUCENE-7907

I have two jiras in patch-available status:

https://issues.apache.org/jira/browse/LUCENE-9323
https://issues.apache.org/jira/browse/SOLR-14266

Andras

On Mon, May 11, 2020 at 4:28 PM Erick Erickson 
wrote:

> Gus:
>
> When it comes to actually removing the necessity of suppresswarnings
> IntelliJ makes a lot of this much easier. The issue is that it’s too much
> work for any one person to have a hope of doing in any reasonable period
> without introducing errors.
>
> There are just too many warnings for one person to have a hope of thinking
> carefully about all of them, so my strategy is to stop adding to the
> problem, raise awareness when it happens etc. I think to remove the
> necessity for SuppressWarnings will require extensive work, best approached
> over time, not in a huge wodge.
>
> Best,
> Erick
>
> > On May 11, 2020, at 9:25 AM, Gus Heck  wrote:
> >
> > I typically battle warnings by conquering one file/directory at a
> time... And letting Intellij take me from warning to warning with F2 key
> really really really speeds things up. This is a wider set than compiler
> warnings, but you can customize it, and many of the additional warnings are
> auto-solvable (things like redundant initializers for variables that are
> already assigned before use), so the extra work is more than paid for by
> the reduction in transition time. The key one to think carefully about is
> the one that wants to minimize access, which is great for new classes,
> dangerous for released classes. Perhaps turn that warning off in
> intellij...
> >
> > On Mon, May 11, 2020 at 8:14 AM Atri Sharma  wrote:
> > +1 to Erick’s proposal.
> >
> > I hate the number of warnings we get — we should still be formulating
> some sort of a strategy to fix them.
> >
> > Atri
> >
> > On Mon, 11 May 2020 at 17:09, Erick Erickson 
> wrote:
> > Just disabling the warning globally nothing to prevent more being added.
> Take raw types. They’re a compromise allowed by the java compiler
> explicitly to be able to continue to use older binaries written before (or
> without) generics. But take a look at SolrQueryResponse for instance. We
> explicitly declare:
> >
> > protected NamedList values = new SimpleOrderedMap<>();
> >
> > but then declare a method:
> >
> > public NamedList getValues() { return values; }
> >
> > This is just bad practice.
> >
> > I don’t mind the grunt work, keeps me from stupid surfing. I’m proposing
> that I fix what’s easy, and suppress the rest.
> >
> > It might have been clearer if I’d said “Then start failing builds on any
> new warnings of these types”.
> >
> > Oh, and I’m also thinking of changing my BadApple report to flag when
> new SuppressWarnings are introduced and then nag people about new ones.
> >
> >
> >
> > > On May 10, 2020, at 11:43 PM, David Smiley 
> wrote:
> > >
> > > Can't we customize the linting to disregard entire categories of
> certain warnings for now?  This makes your task manageable.
> > > https://discuss.gradle.org/t/recompile-with-xlint-parameters/25279
> > >
> > > ~ David
> > >
> > >
> > > On Sun, May 10, 2020 at 10:41 PM Erick Erickson <
> erickerick...@gmail.com> wrote:
> > > I’m really struggling with what to do with compiler warnings,
> particularly all the rawtypes and unchecked warnings.
> > >
> > > On the one hand, the simple mechanical thing to do would be to
> SuppressWarnings on each one that exists presently. Frankly that feels
> pretty useless; that would preserve poor code forever.
> > >
> > > OTOH, actually _fixing_ the issues to not have, say, rawtypes is going
> to be time consuming and error-prone. Especially since I don’t really
> understand all the nuances yet and learning them one by one will introduce
> serious errors without doubt.
> > >
> > > So here’s what I propose. Even though it feels useless, just
> SuppressWarnings on anything that’s not a simple fix. Then start failing
> builds on these warnings to catch any that come in in future. At least that
> way there’ll be some incentive to keep the code from getting _worse_,
> although people will still be able to just add SuppressWarnings to the mix
> I suppose.
> > >
> > > The number of raw NamedList member variables we have is overwhelming
> all by itself….
> > >
> > > Comments?
> > >
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > > For additional commands, e-mail: dev-h...@lucene.apache.org
> > >
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > For additional commands, e-mail: 

Re: [VOTE] Release Lucene/Solr 8.5.0 RC1

[Andras Salamon]

Hi,

+1 (non-binding)

SUCCESS! [1:32:51.320515]

Andras

On Fri, Mar 13, 2020 at 3:31 PM Alan Woodward  wrote:

> Please vote for release candidate 1 for Lucene/Solr 8.5.0
>
> The artifacts can be downloaded from:
>
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.5.0-RC1-rev7ac489bf7b97b61749b19fa2ee0dc46e74b8dc42
>
> You can run the smoke tester directly with this command:
>
> python3 -u dev-tools/scripts/smokeTestRelease.py \
>
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.5.0-RC1-rev7ac489bf7b97b61749b19fa2ee0dc46e74b8dc42
>
> The vote will be open for three working days i.e. until next Tuesday,
> 2020-03-18 14:00 UTC.
>
> [ ] +1  approve
> [ ] +0  no opinion
> [ ] -1  disapprove (and reason why)
>
> Here is my +1
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Solr 8.4.1 Hadoop authentication with Kerberos

[Andras Salamon]

*Hi Solr devs,We are trying to use Hadoop authentication with Kerberos in
Solr 8.4.1 and encountered a problem. We’re using a Hadoop 3.1.1 based
fork. We are using JDK8 so we fall back to HTTP/1.1 but also tested with
JDK11 (HTTP/2) and we got the same error.We have already added a few
upstream changes which are not yet committed
(https://issues.apache.org/jira/browse/SOLR-9840
) or committed only later
(https://issues.apache.org/jira/browse/SOLR-11554
).The important part of
our security.json file is:"authentication": {"class":
"org.apache.solr.security.ConfigurableInternodeAuthHadoopPlugin",
  "sysPropPrefix":
"solr.authentication.","type":
"multi-scheme","clientBuilderFactory":
"org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder",When we try to
add a document using curl we receive 401 error:*

curl -k --negotiate -u : '
https://quasar-mdzaga-1.vpc.cloudera.com:8985/solr/test2/update' -H
'Content-type:application/json' -d ' [ {"id":"book3", "title":"book3title",
"author":"author"}
]'{  "responseHeader":{"rf":2147483647,"status":401,
"QTime":18},  "error":{"metadata":[
"error-class","org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException",
 
"root-error-class","org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException"],
   "msg":"Async
exception during distributed update: Error from server at
https://quasar-mdzaga-3.vpc.cloudera.com:8985/solr/test2_shard2_replica_n6/:
Authentication required\n\n\n\nrequest:
https://quasar-mdzaga-3.vpc.cloudera.com:8985/solr/test2_shard2_replica_n6/
","Code":401}}















*We have debugged the problem and found that curl can send the information
to the node, and the internode TOLEADER request fails, because we don’t
answer to the 401 challenge that is part of the SPNEGO mechanism:HTTP/1.1
401 Unauthorized access...WWW-Authenticate: NegotiateSet-Cookie:
hadoop.auth=; HttpOnlyCache-Control:
must-revalidate,no-cache,no-storeContent-Type:
text/html;charset=iso-8859-1Content-Length: 287Checking the code shows that
ConcurrentUpdateHttp2SolrClient calls Http2SolrClient.initOutStream which
creates an OutputStreamContentProvider where the value of the
isReproducible flag is false and jetty’s AuthenticationProtocolHandler will
not continue the authentication in this caseRelevant code
sections:https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.4.1/solr/solrj/src/java/org/apache/solr/client/solrj/impl/ConcurrentUpdateHttp2SolrClient.java#L212
https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.4.1/solr/solrj/src/java/org/apache/solr/client/solrj/impl/Http2SolrClient.java#L299
https://github.com/eclipse/jetty.project/blob/jetty-9.4.19.v20190610/jetty-client/src/main/java/org/eclipse/jetty/client/AuthenticationProtocolHandler.java#L192
We
have also found a workaround. If we send a simple successfully
authenticated message before Http2SolrClient.initOutStream in
ConcurrentUpdateHttp2SolrClient the authentication works correctly. Not
only for the simple message but also for the upcoming requests. So right
now we send an OPTIONS request here and just ignore the answer.Sending the
OPTIONS request happens before setting up the update stream, so if we send
multiple documents in a single update, only one OPTIONS will be sent to
each leader.Although this workaround works for us, we are not sure that
this is the best place to ensure pre-authentication between the nodes. Does
anybody have a better place to handle it?Is there anybody here who is
successfully using Solr8 with Hadoop Authentication and
Kerberos?Thanks,Andras*