Closing the loop on vulnerable dependencies, I wound up (for now) deactivating streams-persist-hdfs and streams-persist-elasticsearch, and all of the sub-modules of streams-examples. Suneel I think you are right opensearch is a more appropriate dependency if there is a reason to bring streams-persist-elasticsearch back into the reactor.
Build and unit tests were working for me in JDK8, JDK11, and JDK17. Our github actions CI needs some love and i suspect our jenkins jobs got lost during an infra upgrade so we'll need to bring that back online. Those details aside, I think the project is release-able again. I have not yet typed up anything more than the first message in this thread. I'm planning to spend time over the holiday digging into the current revisions of the Activity Streams 2.0 ontology and ActivityPub spec, and how much of them are currently supported by the standard mastodon back-end. If anyone else wants to help out in this early stage, maybe do the same and we can all compare notes on what the interfaces and deployment plan of a non-throwaway-proof-of-concept/MVP might look like? Just riffing, I suspect we will want to continue trimming down modules and dependencies if they don't have a clear purpose under the new objective of a scalable activitypub backend plus useful plugins (such as the persistence modules, data providers, and data augmentation modules in streams-contrib). I think we could approach a form where our core dependencies are apache projects only - commons, httpcomponents, juneau, lucene/solr, maybe pekko? There will need to be at least one default data repository capable of storing anything in the spec and supporting the queries required by any endpoint. Jena seems like a good candidate for this, at least for small-ish deployments, but there might be another better solution I've not thought of yet. P.S. It's nice to hear from everyone, I hope you are all doing well. On Sun, Dec 17, 2023 at 9:10 PM Brian Hodge <brianpho...@gmail.com> wrote: > Apologies, this got buried in my inbox under other mailing lists. > > Steve, your suggestions make sense to me. I strongly agree that building on > Streams to ship broadly useful software is a worthwhile goal. > > I do have some questions about the realignment you're envisioning. Have you > written (or do you plan to write) anything more longform on design / > architecture of the refactor you have in mind? > > I'm happy to help however I can. Are there near-term tasks that I and > others can pitch in on? > > -Brian Hodge > > On Sun, Dec 17, 2023 at 2:58 PM Suneel Marthi <smar...@apache.org> wrote: > > > Replace ElasticSearch with OpenSearch ??? > > > > On Sun, Dec 17, 2023 at 12:36 PM Steve Blackmon <sblack...@apache.org> > > wrote: > > > > > Thanks Trevor. Silent majority, hope to hear from you as/if the reboot > > > picks up steam. > > > > > > I've gotten started the last few weeks by refactoring to enable builds > > with > > > Java 17 (committed) and a purge of vulnerable dependencies (almost > > ready). > > > > > > It looks like most of of modules will be able to remain active in the > > > reactor to achieve the above, possible exceptions of our hdfs and > > > elasticsearch modules. Elasticsearch is very outdated and even the > > newest > > > available common-hdfs doesn't appear able to have the vulnerable > > > commons-collections:commons-collections removed and still work. > > > > > > Stay tuned > > > > > > On Mon, Dec 4, 2023 at 10:11 AM Trevor Grant <trevor.d.gr...@gmail.com > > > > > wrote: > > > > > > > I agree with most of what you're saying, candidly my bet is on social > > > media > > > > as we know it just sort of silently fading as Millenials get older. > But > > > I'm > > > > still on board to help out- I have cycles to work tickets, but idk > if I > > > > have enough to put in any good faith effort to lead (we're doing a > > > similar > > > > reboot over on Mahout, where I am leading). > > > > > > > > tg > > > > > > > > On Sun, Dec 3, 2023 at 8:21 PM sblack...@apache.org < > > > sblack...@apache.org> > > > > wrote: > > > > > > > > > Hello dev@streams.a.o, > > > > > > > > > > This list has been very quiet for a very long time. I’m hoping to > > > change > > > > > that, and generate some momentum toward a worth-while initiative > > right > > > > > in-line with Streams founding goals. > > > > > > > > > > The project founding proposal [1] includes the following goals: > > > > > > > > > > • Publication of Activities from multiple systems via HTTP > > > > > • Aggregation and syndication of streams > > > > > • Support for security trimming of streams by social graph > > > > > • Noise reduction and intelligent filtering > > > > > • Federation of streams across disparate systems > > > > > • Provide libraries for easy integration in source systems > > > > > > > > > > The code we actually built focused most heavily on goals #2 and #5, > > and > > > > > evolved into a (useful IMO, at least around the time of our TLP > > > > graduation) > > > > > java SDK for building complex social media application back-end > > > > > ’streams’. But, we never shipped software with clear stand-alone > > > utility > > > > > for anyone other than java developers. The internet, and the > social > > > > media > > > > > landscape specifically, has changed a lot since then, and I would > > argue > > > > > that the system envisioned in the proposal is needed now more than > > > > ever. I > > > > > won’t go right now into all the reasons I believe that is the case, > > > > because > > > > > it would be TL;DR and off-topic, but will just say that there > appears > > > to > > > > be > > > > > an unprecedented (at least in this century) interest in hosting > > > > > social-media-like software that can inter-operate across > > installations. > > > > > > > > > > So I’m here asking whether anyone is open to a discussion about > > > > > re-aligning this project to the totality of it’s origin, with a new > > > North > > > > > Star: > > > > > > > > > > • a containerized deployable software stack that provides an > Activity > > > > > Streams 2.0 / Activity Pub interface via rest API > > > > > • Confirmed interoperable with the Mastodon front-end, and other > > > > > ActivityPub client software > > > > > • Built to scale with minimal system admin burden to the needs of > the > > > > > largest existing Mastodon instances: ballpark ~1M total users, 10K > > > > > concurrent > > > > > • Able to scale far beyond that for an experienced team: a > > medium-sized > > > > > Web 2.0 social network: ~100M total users, up to ~1M concurrent > > > > > > > > > > I don’t think it’s hyper-bole to observe that the future of social > > > media > > > > > is in flux, and in early stages of a 10+ year trend toward > > > > > decentralization. I for want really want that decentralized future > > to > > > be > > > > > based on Activity Streams 2.0 and ActivityPub, now official W3C > > > > standards, > > > > > and thus inter-operable when desired, rather than completely > silo’ed > > > like > > > > > the prior generation. Also I think the above is entirely plausible > > on > > > a > > > > > 1-2 year time-scale. > > > > > > > > > > Does anyone else agree? Is anyone willing to support a pivot like > > > this, > > > > > and try to re-build this product and community with me? > > > > > > > > > > • [1] > > > > > > > https://cwiki.apache.org/confluence/display/incubator/StreamsProposal > > > > > > > > > > > > > > > Steve Blackmon > > > > > Apache Streams PMC Chair > > > > > > > > > > > > > > >
