Re: A Last Message to the Solr Users

[Nicolas Paris]

Hi Mark,

Have you shared with the community all the weaknesses of solrcloud  you
have in mind and the advice to overcome that? 
Apparently you wrote most of that code and your feedback would be
helpful for community.

Regards

On Sat, Nov 30, 2019 at 09:31:34PM -0600, Mark Miller wrote:
> I’d also like to say the last 5 years of my life have been spent being paid
> to upgrade Solr systems. I’ve made a lot of doing this.
> 
> As I said from the start - take this for what it’s worst. For his guys it’s
> not worth much. That’s cool.
> 
> And it’s a little inside joke that I’ll be back :) I joke a lot.
> 
> But seriously, you have a second chance here.
> 
> This mostly concerns SolrCloud. That’s why I recommend standalone mode. But
> key people know why to do. I know it will happen - but their lives will be
> easier if you help.
> 
> Lol.
> 
> - Mark
> 
> On Sat, Nov 30, 2019 at 9:25 PM Mark Miller  wrote:
> 
> > I said the key people understand :)
> >
> > I’ve worked in Lucene since 2006 and have an insane amount of the code
> > foot print in Solr and SolrCloud :) Look up the stats. Do you have any
> > contributions?
> >
> > I said the key people know.
> >
> > Solr stand-alone is and has been very capable. People are working around
> > SolrCloud too.All fine and good. Millions are being made and saved.
> > Everyone is comfortable. Some might thinks the sky looks clear and blue.
> > I’ve spent a lot of capital to make sure the wrong people don’t think that
> > anymore ;)
> >
> > Unless you are a Developer, you won’t understand the other issues. But you
> > don’t need too.
> >
> > Mark
> >
> > On Sat, Nov 30, 2019 at 7:05 PM Dave  wrote:
> >
> >> I’m young here I think, not even 40 and only been using solr since like
> >> 2008 or so, so like 1.4 give or take. But I know a really good therapist if
> >> you want to talk about it.
> >>
> >> > On Nov 30, 2019, at 6:56 PM, Mark Miller  wrote:
> >> >
> >> > Now I have sacrificed to give you a new chance. A little for my
> >> community.
> >> > It was my community. But it was mostly for me. The developer I started
> >> as
> >> > would kick my ass today.  Circumstances and luck has brought money to
> >> our
> >> > project. And it has corrupted our process, our community, and our code.
> >> >
> >> > In college i would talk about past Mark screwing future Mark and too bad
> >> > for him. What did he ever do for me? Well, he got me again ;)
> >> >
> >> > I’m out of steam, time and wife patentice.
> >> >
> >> > Enough key people are aware of the scope of the problem now that you
> >> won’t
> >> > need me. I was never actually part of the package. To the many, many
> >> people
> >> > that offered me private notes of encouragement and future help - thank
> >> you
> >> > so much. Your help will be needed.
> >> >
> >> > You will reset. You will fix this. Or I will be back.
> >> >
> >> > Mark
> >> >
> >> >
> >> > --
> >> > - Mark
> >> >
> >> > http://about.me/markrmiller
> >>
> > --
> > - Mark
> >
> > http://about.me/markrmiller
> >
> -- 
> - Mark
> 
> http://about.me/markrmiller

-- 
nicolas

Re: A Last Message to the Solr Users

[Mark Miller]

I’d also like to say the last 5 years of my life have been spent being paid
to upgrade Solr systems. I’ve made a lot of doing this.

As I said from the start - take this for what it’s worst. For his guys it’s
not worth much. That’s cool.

And it’s a little inside joke that I’ll be back :) I joke a lot.

But seriously, you have a second chance here.

This mostly concerns SolrCloud. That’s why I recommend standalone mode. But
key people know why to do. I know it will happen - but their lives will be
easier if you help.

Lol.

- Mark

On Sat, Nov 30, 2019 at 9:25 PM Mark Miller  wrote:

> I said the key people understand :)
>
> I’ve worked in Lucene since 2006 and have an insane amount of the code
> foot print in Solr and SolrCloud :) Look up the stats. Do you have any
> contributions?
>
> I said the key people know.
>
> Solr stand-alone is and has been very capable. People are working around
> SolrCloud too.All fine and good. Millions are being made and saved.
> Everyone is comfortable. Some might thinks the sky looks clear and blue.
> I’ve spent a lot of capital to make sure the wrong people don’t think that
> anymore ;)
>
> Unless you are a Developer, you won’t understand the other issues. But you
> don’t need too.
>
> Mark
>
> On Sat, Nov 30, 2019 at 7:05 PM Dave  wrote:
>
>> I’m young here I think, not even 40 and only been using solr since like
>> 2008 or so, so like 1.4 give or take. But I know a really good therapist if
>> you want to talk about it.
>>
>> > On Nov 30, 2019, at 6:56 PM, Mark Miller  wrote:
>> >
>> > Now I have sacrificed to give you a new chance. A little for my
>> community.
>> > It was my community. But it was mostly for me. The developer I started
>> as
>> > would kick my ass today.  Circumstances and luck has brought money to
>> our
>> > project. And it has corrupted our process, our community, and our code.
>> >
>> > In college i would talk about past Mark screwing future Mark and too bad
>> > for him. What did he ever do for me? Well, he got me again ;)
>> >
>> > I’m out of steam, time and wife patentice.
>> >
>> > Enough key people are aware of the scope of the problem now that you
>> won’t
>> > need me. I was never actually part of the package. To the many, many
>> people
>> > that offered me private notes of encouragement and future help - thank
>> you
>> > so much. Your help will be needed.
>> >
>> > You will reset. You will fix this. Or I will be back.
>> >
>> > Mark
>> >
>> >
>> > --
>> > - Mark
>> >
>> > http://about.me/markrmiller
>>
> --
> - Mark
>
> http://about.me/markrmiller
>
-- 
- Mark

http://about.me/markrmiller

Re: A Last Message to the Solr Users

[Mark Miller]

I said the key people understand :)

I’ve worked in Lucene since 2006 and have an insane amount of the code foot
print in Solr and SolrCloud :) Look up the stats. Do you have any
contributions?

I said the key people know.

Solr stand-alone is and has been very capable. People are working around
SolrCloud too.All fine and good. Millions are being made and saved.
Everyone is comfortable. Some might thinks the sky looks clear and blue.
I’ve spent a lot of capital to make sure the wrong people don’t think that
anymore ;)

Unless you are a Developer, you won’t understand the other issues. But you
don’t need too.

Mark

On Sat, Nov 30, 2019 at 7:05 PM Dave  wrote:

> I’m young here I think, not even 40 and only been using solr since like
> 2008 or so, so like 1.4 give or take. But I know a really good therapist if
> you want to talk about it.
>
> > On Nov 30, 2019, at 6:56 PM, Mark Miller  wrote:
> >
> > Now I have sacrificed to give you a new chance. A little for my
> community.
> > It was my community. But it was mostly for me. The developer I started as
> > would kick my ass today.  Circumstances and luck has brought money to our
> > project. And it has corrupted our process, our community, and our code.
> >
> > In college i would talk about past Mark screwing future Mark and too bad
> > for him. What did he ever do for me? Well, he got me again ;)
> >
> > I’m out of steam, time and wife patentice.
> >
> > Enough key people are aware of the scope of the problem now that you
> won’t
> > need me. I was never actually part of the package. To the many, many
> people
> > that offered me private notes of encouragement and future help - thank
> you
> > so much. Your help will be needed.
> >
> > You will reset. You will fix this. Or I will be back.
> >
> > Mark
> >
> >
> > --
> > - Mark
> >
> > http://about.me/markrmiller
>
-- 
- Mark

http://about.me/markrmiller

Re: A Last Message to the Solr Users

[Dave]

I’m young here I think, not even 40 and only been using solr since like 2008 or 
so, so like 1.4 give or take. But I know a really good therapist if you want to 
talk about it. 

> On Nov 30, 2019, at 6:56 PM, Mark Miller  wrote:
> 
> Now I have sacrificed to give you a new chance. A little for my community.
> It was my community. But it was mostly for me. The developer I started as
> would kick my ass today.  Circumstances and luck has brought money to our
> project. And it has corrupted our process, our community, and our code.
> 
> In college i would talk about past Mark screwing future Mark and too bad
> for him. What did he ever do for me? Well, he got me again ;)
> 
> I’m out of steam, time and wife patentice.
> 
> Enough key people are aware of the scope of the problem now that you won’t
> need me. I was never actually part of the package. To the many, many people
> that offered me private notes of encouragement and future help - thank you
> so much. Your help will be needed.
> 
> You will reset. You will fix this. Or I will be back.
> 
> Mark
> 
> 
> -- 
> - Mark
> 
> http://about.me/markrmiller

Re: A Last Message to the Solr Users

[Mark Miller]

Now I have sacrificed to give you a new chance. A little for my community.
It was my community. But it was mostly for me. The developer I started as
would kick my ass today.  Circumstances and luck has brought money to our
project. And it has corrupted our process, our community, and our code.

In college i would talk about past Mark screwing future Mark and too bad
for him. What did he ever do for me? Well, he got me again ;)

I’m out of steam, time and wife patentice.

Enough key people are aware of the scope of the problem now that you won’t
need me. I was never actually part of the package. To the many, many people
that offered me private notes of encouragement and future help - thank you
so much. Your help will be needed.

You will reset. You will fix this. Or I will be back.

Mark


-- 
- Mark

http://about.me/markrmiller

Re: A Last Message to the Solr Users

[Mark Miller]

It’s going to haunt me if I don’t bring up Hossman. I don’t feel I have to
because who doesn’t know him.

He is a treasure that doesn’t spend much time on SolrCloud and has checked
out of leadership for the large part for reasons I won’t argue with.

Why doesn’t he do much with SolrCloud in a real way? I can only guess. He
will tell you it’s above his pay grade or some dumb shit.

IMO, it’s probably more that super thorough people try to be thorough with
SolrCloud and when you do that, it will poke your eye out with a stick. And
then throw you over a cliff.

Make it something he can work on more than tangentially.

Mark
-- 
- Mark

http://about.me/markrmiller

Re: A Last Message to the Solr Users

[Mark Miller]

I’m including this response to a private email because it’s not something
I’ve brought up and I also think it’s a critical note:

“Yes. That is our biggest advantage. Being Apache. Almost no one seems to
be employed to help other contributors get their work in at the right
level, and all the money has ensured the end of the hobbyist. I hope that
changes too.”

-- 
- Mark

http://about.me/markrmiller

Re: A Last Message to the Solr Users

[Mark Miller]

Yes. That is our biggest advantage. Being Apache. Almost no one seems to be
employed to help other contributors get their work in at the right level,
and all the money has ensured the end of the hobbyist. I hope that changes
too.

Thanks for the note.

Mark

On Thu, Nov 28, 2019 at 1:55 PM Paras Lehana 
wrote:

> Hey Mark,
>
> I was actually expecting (and wanting) this after your LinkedIn post.
>
> At this point, the best way to use Solr is as it’s always been - avoid
>> SolrCloud and setup your own system in standalone mode.
>
>
> That's what I have been telling people who are just getting started with
> Solr and thinking that SolrCloud is actually something superior to the
> standalone mode. That may depend on the use case, but for me, I always
> prefer to achieve things from standalone perspective instead of investing
> my time over switching to Cloud.
>
> I handle Auto-Suggest at IndiaMART. We have over 60 million docs. Single
> server of *standalone* Solr is capable of handling 800 req/sec. In fact,
> on production, we get ~300 req/sec and the single Solr is still able to
> provide responses within 25 ms!
>
> Anyways, I don't think that the project was a failure. All these were the
> small drops of the big Solr Ocean. We, the community and you, tried, we
> tested and we are still here as the open community of one of the most
> powerful search platforms. SolrCloud was also needed to be introduced at
> some time. Notwithstanding, I do think that the project needs to be more
> open with community commits. The community and open-sourceness of Solr is
> what I used to love over those of ElasticSearch's.
>
> Anyways, keep rocking! You have already left your footprints into the
> history of this beast project. 落
>
> On Thu, 28 Nov 2019 at 09:10, Mark Miller  wrote:
>
>> Now one company thinks I’m after them because they were the main source of
>> the jokes.
>>
>> Companies is not a typo.
>>
>> If you are using Solr to make or save tons of money or run your business
>> and you employee developers, please include yourself in this list.
>>
>> You are taking and in my opinion Solr is going down. It’s all against your
>> own interest even.
>>
>> I know of enough people that want to solve this now, that it’s likely only
>> a matter of time before they fix the situation - you ever know though.
>> Things change, people get new jobs, jobs change. It will take at least 3-6
>> months to make things reasonable even with a good group banding together.
>>
>> But if you are extracting value from this project and have Solr developers
>> - id like to think you have enough of a stake in this to think about
>> changing the approach everyone has been taking. It’s not working, and the
>> longer it goes on, the harder it’s getting to fix things.
>>
>>
>> --
>> - Mark
>>
>> http://about.me/markrmiller
>>
>
>
> --
> --
> Regards,
>
> *Paras Lehana* [65871]
> Development Engineer, Auto-Suggest,
> IndiaMART Intermesh Ltd.
>
> 8th Floor, Tower A, Advant-Navis Business Park, Sector 142,
> Noida, UP, IN - 201303
>
> Mob.: +91-9560911996
> Work: 01203916600 | Extn:  *8173*
>
>
> 
>
-- 
- Mark

http://about.me/markrmiller

Re: A Last Message to the Solr Users

[Mark Miller]

The people I have identified that I have the most faith in to lead the
fixing of Solr are Ishan, Noble and David. I encourage you all to look at
and follow and join in their leadership.

You can do this.


Mark
-- 
- Mark

http://about.me/markrmiller

Re: A Last Message to the Solr Users

[Paras Lehana]

Hey Mark,

I was actually expecting (and wanting) this after your LinkedIn post.

At this point, the best way to use Solr is as it’s always been - avoid
> SolrCloud and setup your own system in standalone mode.


That's what I have been telling people who are just getting started with
Solr and thinking that SolrCloud is actually something superior to the
standalone mode. That may depend on the use case, but for me, I always
prefer to achieve things from standalone perspective instead of investing
my time over switching to Cloud.

I handle Auto-Suggest at IndiaMART. We have over 60 million docs. Single
server of *standalone* Solr is capable of handling 800 req/sec. In fact, on
production, we get ~300 req/sec and the single Solr is still able to
provide responses within 25 ms!

Anyways, I don't think that the project was a failure. All these were the
small drops of the big Solr Ocean. We, the community and you, tried, we
tested and we are still here as the open community of one of the most
powerful search platforms. SolrCloud was also needed to be introduced at
some time. Notwithstanding, I do think that the project needs to be more
open with community commits. The community and open-sourceness of Solr is
what I used to love over those of ElasticSearch's.

Anyways, keep rocking! You have already left your footprints into the
history of this beast project. 落

On Thu, 28 Nov 2019 at 09:10, Mark Miller  wrote:

> Now one company thinks I’m after them because they were the main source of
> the jokes.
>
> Companies is not a typo.
>
> If you are using Solr to make or save tons of money or run your business
> and you employee developers, please include yourself in this list.
>
> You are taking and in my opinion Solr is going down. It’s all against your
> own interest even.
>
> I know of enough people that want to solve this now, that it’s likely only
> a matter of time before they fix the situation - you ever know though.
> Things change, people get new jobs, jobs change. It will take at least 3-6
> months to make things reasonable even with a good group banding together.
>
> But if you are extracting value from this project and have Solr developers
> - id like to think you have enough of a stake in this to think about
> changing the approach everyone has been taking. It’s not working, and the
> longer it goes on, the harder it’s getting to fix things.
>
>
> --
> - Mark
>
> http://about.me/markrmiller
>


-- 
-- 
Regards,

*Paras Lehana* [65871]
Development Engineer, Auto-Suggest,
IndiaMART Intermesh Ltd.

8th Floor, Tower A, Advant-Navis Business Park, Sector 142,
Noida, UP, IN - 201303

Mob.: +91-9560911996
Work: 01203916600 | Extn:  *8173*

-- 
*
*

 

Re: A Last Message to the Solr Users

[Mark Miller]

Now one company thinks I’m after them because they were the main source of
the jokes.

Companies is not a typo.

If you are using Solr to make or save tons of money or run your business
and you employee developers, please include yourself in this list.

You are taking and in my opinion Solr is going down. It’s all against your
own interest even.

I know of enough people that want to solve this now, that it’s likely only
a matter of time before they fix the situation - you ever know though.
Things change, people get new jobs, jobs change. It will take at least 3-6
months to make things reasonable even with a good group banding together.

But if you are extracting value from this project and have Solr developers
- id like to think you have enough of a stake in this to think about
changing the approach everyone has been taking. It’s not working, and the
longer it goes on, the harder it’s getting to fix things.


-- 
- Mark

http://about.me/markrmiller

Re: A Last Message to the Solr Users

[Walter Underwood]

I’m a big fan of master/slave Solr. Super robust and trivial to scale-out.

Solr Cloud has been useful for managing sharding and replicas, but less robust 
than I would like. Also less robust than my managers would like. It has gotten 
a bad reputation, only partially undeserved.

I’m also not especially impressed with the Solr community openness to changes 
from outside of the committers. At some point, I’ll do the fourth port of a 
small enhancement to edismax. I’d love to run stock Solr, but I can’t. Fuzzy 
search got 100X faster in 4.x, but it isn’t available in edismax, even though 
it makes a huge difference for misspelled queries.

Want to make a change that radically improves basic search? Implement IDF for 
phrases. Infoseek used that to beat Google in relevance for years. The patent 
has expired, so go for it.

Also, how about a free text parser so we don’t have to rip out all the Lucene 
syntax in queries?

wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)

> On Nov 27, 2019, at 8:37 AM, Mark Miller  wrote:
> 
> If SolrCloud worked well I’d still agree both options are very valid
> depending on your use case. As it is, I’m embarrassed that people give me
> any credit for this. I’m here to try and delight users and I have failed in
> that. I tried to put a lot of my own time to address things outside of
> working on my job of integrating Hadoop and upgrading Solr 4 instances for
> years. But I couldn’t convince anyone of what was necessary to address what
> has been happening, and my paid job has always been doing other things
> since 2012.
> 
> On Wed, Nov 27, 2019 at 6:23 PM David Hastings 
> wrote:
> 
>> Personally I found nothing in solr cloud worth changing from standalone
>> for, and just added more complications, more servers, and required becoming
>> an expert/knowledgeable in zoo keeper, id rather spend my time developing
>> than becoming a systems administrator
>> 
>> On Wed, Nov 27, 2019 at 3:45 AM Mark Miller  wrote:
>> 
>>> This is your queue to come and make your jokes with your name attached.
>>> I’m
>>> sure the Solr users will appreciate them more than I do. I can’t laugh at
>>> this situation because I take production code seriously.
>>> 
>>> --
>>> - Mark
>>> 
>>> http://about.me/markrmiller
>>> 
>> --
> - Mark
> 
> http://about.me/markrmiller

Re: A Last Message to the Solr Users

[Mark Miller]

If SolrCloud worked well I’d still agree both options are very valid
depending on your use case. As it is, I’m embarrassed that people give me
any credit for this. I’m here to try and delight users and I have failed in
that. I tried to put a lot of my own time to address things outside of
working on my job of integrating Hadoop and upgrading Solr 4 instances for
years. But I couldn’t convince anyone of what was necessary to address what
has been happening, and my paid job has always been doing other things
since 2012.

On Wed, Nov 27, 2019 at 6:23 PM David Hastings 
wrote:

> Personally I found nothing in solr cloud worth changing from standalone
> for, and just added more complications, more servers, and required becoming
> an expert/knowledgeable in zoo keeper, id rather spend my time developing
> than becoming a systems administrator
>
> On Wed, Nov 27, 2019 at 3:45 AM Mark Miller  wrote:
>
>> This is your queue to come and make your jokes with your name attached.
>> I’m
>> sure the Solr users will appreciate them more than I do. I can’t laugh at
>> this situation because I take production code seriously.
>>
>> --
>> - Mark
>>
>> http://about.me/markrmiller
>>
> --
- Mark

http://about.me/markrmiller

Re: A Last Message to the Solr Users

[David Hastings]

Personally I found nothing in solr cloud worth changing from standalone
for, and just added more complications, more servers, and required becoming
an expert/knowledgeable in zoo keeper, id rather spend my time developing
than becoming a systems administrator

On Wed, Nov 27, 2019 at 3:45 AM Mark Miller  wrote:

> This is your queue to come and make your jokes with your name attached. I’m
> sure the Solr users will appreciate them more than I do. I can’t laugh at
> this situation because I take production code seriously.
>
> --
> - Mark
>
> http://about.me/markrmiller
>

Re: A Last Message to the Solr Users

[Mark Miller]

This is your queue to come and make your jokes with your name attached. I’m
sure the Solr users will appreciate them more than I do. I can’t laugh at
this situation because I take production code seriously.

-- 
- Mark

http://about.me/markrmiller

Re: A Last Message to the Solr Users

[Mark Miller]

And if you are a developer, enjoy that Gradle build! It was the highlight
of my year.

On Wed, Nov 27, 2019 at 10:00 AM Mark Miller  wrote:

> If you have a SolrCloud installation that is somehow working for you,
> personally I would never upgrade. The software is getting progressively
> more unstable every release.
>
>
> I wrote most of the core of SolrCloud in a prototype fashion many, many
> years ago. Only Yonik’s isolated work is solid and most of my work still
> stands as it was. This situation has me abandoning that project so that
> people understand I won’t stand by garbage work.
>
> Given that no one seems to understand what is happening in SolrCloud under
> the covers or how it was intended to work, their best bet is to start
> rewriting. Until they do this, I recommend you do not upgrade from an
> install that is working for your needs. A new feature will not be worth the
> headaches.
>
>
> Some of the other committers, who certainly do not understand the scope of
> the problem or my code (they would have touched it a bit if they did) would
> prefer to laugh or form a defensive posture than fix the situation. Wait
> them out. The project will collapse or get better. If I ran a production
> instance of SolrCloud, I would wait to see which happens first before
> embracing any update.
>
>
> At this point, the best way to use Solr is as it’s always been - avoid
> SolrCloud and setup your own system in standalone mode. If I had to build a
> new Solr install today, this is what I would do.
>
>
> In my opinion, the companies that have been claiming to back Solr and
> SolrCloud have been negligent, and all of the users are paying the price.
> It hasn’t been my job to work on it in any real fashion since 2012. I’m
> sorry I couldn’t help improve the situation for you.
>
>
> Take it for what it’s worth. To some, not much I’m sure.
>
>
> Mark Miller
> --
> - Mark
>
> http://about.me/markrmiller
>
-- 
- Mark

http://about.me/markrmiller