Hi Karl and all, In my use-case, one of the data-sources is an already populated Solr index which is an e-commerce web-site data index (customers, products & services). Apart from the Solr Index, I need to ingest several other heterogeneous data-sources such as PostgresSQL databases, CRM data etc into the federated search index (the output index will either be a Solr, Elastic-search. We haven't yet finalized on the output index, but I know that both of these are supported in MCF as output connectors.).
@Karl based on your comments, I would appreciate your opinion on below ingestion flow. Solr repository/data-source > Solr schema transformations > Solr/Elastic-search search-index For such a scenario, do you think MCF is not the ideal option as the ETL/ingestion tool? Should I go for a lower-level ETL tool such as Apache Nifi ? Or will writing a MCF Solr repository connector be useful to achieve this? WDYT? Thanks a lot. Regards, Dileepa On Mon, Aug 5, 2019 at 3:40 PM Karl Wright <[email protected]> wrote: > If you are trying to extract data from a Solr index, I know of no way to > do that. > Karl > > > On Mon, Aug 5, 2019 at 9:08 AM Dileepa Jayakody <[email protected]> > wrote: > >> Hi All, >> >> Thanks for your replies. >> I'm looking for a repository connector. I've used the Solr output >> connector before. But now what I need is to connect to a solr index as a >> repository and retrieve the documents from there. So I need a Solr >> repository connector. >> >> @Karl >> I will look at the Solr connector, but this is an output connect, isn't >> it? Can use this as a repository connector to retrieve docs? >> >> Thanks, >> Dileepa >> >> On Mon, Aug 5, 2019 at 12:45 PM Cihad Guzel <[email protected]> wrote: >> >>> Hi Dileepa, >>> >>> You can check all MFC Connectors list from >>> https://manifoldcf.apache.org/release/release-2.13/en_US/included-connectors.html >>> >>> MFC have a Solr Output Connector. It is not a repository connector. if >>> you want to use as repository connector, you should write a new repository >>> connector. >>> >>> Regards, >>> Cihad Guzel >>> >>> >>> Dileepa Jayakody <[email protected]>, 5 Ağu 2019 Pzt, 13:18 >>> tarihinde şunu yazdı: >>> >>>> Hi All, >>>> >>>> I'm working on a project which needs to implement a federated search >>>> solution with heterogeneous data repositories. One repository is a Solr >>>> index. I would like to use ManifoldCF as the data ingestion engine in this >>>> project as I have worked with MCF before. >>>> >>>> Does ManifoldCF has a Solr repository connector which I can use here? >>>> Or will I need to implement a new repository connector for Solr? >>>> Any guidance here is much appreciated. >>>> >>>> Thanks, >>>> Dileepa >>>> >>>
