Leonhard,
Glad you got it working and thanks for sharing that code. I saved it
because I'll probably need it some day. I have never used a custom thread
pool before. My app only has a couple of background threads and so I'm just
using @Scheduled annotation of Spring. You must be doing some "heavy
lifting" in your app.
Best regards,
Clay Ferguson
wcl...@gmail.com
On Fri, Aug 7, 2015 at 7:25 AM, lw <leonh...@wachutka.eu> wrote:
> Hi Clay,
>
> thanks for your advice. Indeed I digged quiet deep into the source and got
> it finally running. The trick was to include tika-core. The default
> executor of Oak is quiet hacky, it swallows exceptions thrown by its
> threads without logging. So I didn’t get the class not found exception,
> caused by missing tika dependencies, at least in my tomcat deployment.
>
> Just for the records I post my working config. I also added a custom
> OakExecutorProvider to handle the memory leaks by orphan exectors at oak
> shutdown.
>
> I hope it may help someone facing similar problems.
>
> Best Leonhard Wachutka
>
> In pom.xml:
>
> <dependency>
> <groupId>org.apache.tika</groupId>
> <artifactId>tika-core</artifactId>
> <version>1.9</version>
> </dependency>
> SolrServerProviderFactroy:
>
>
> public class SolrServerProviderFactroy {
>
> @Bean
> public RemoteSolrServerProvider remoteSolrServerProvider()
> {
> RemoteSolrServerConfiguration remoteSolrConfig = new
> RemoteSolrServerConfiguration(null, null, 1, 1, null, "
> http://192.168.2.250:8983/solr/oak";);
> return new RemoteSolrServerProvider(remoteSolrConfig);
>
> }
> @Bean
> public OakSolrConfigurationProvider oakSolrConfigProvider()
> {
> OakSolrConfiguration oakSolrconfiguration = new
> DefaultSolrConfiguration() {
> @Override
> public OakSolrConfiguration.CommitPolicy getCommitPolicy() {
> return OakSolrConfiguration.CommitPolicy.HARD;
> }
> @Override
> public boolean useForPropertyRestrictions() {
> return false;
> }
> @Override
> public boolean useForPrimaryTypes() {
> return false;
> }
> @Override
> public boolean useForPathRestrictions() {
> return false;
> }
> @Override
> public String getFieldNameFor(Type<?> propertyType) {
> return null;
> }
>
> };
>
>
> return new DefaultSolrConfigurationProvider(oakSolrconfiguration);
> }
> }
>
>
>
> protected Repository createRepository() throws Exception {
>
>
> // return JackRabbit repository.
> log.info("Start of Apache Jackrabbit OAK Repository.\n{host:" +
> host +"\ndbName:"+ dbName+"}");
> DB db = new MongoClient(host, 27017).getDB(dbName);
> documentNodeStore = new DocumentMK.Builder().
> setMongoDB(db).getNodeStore();
>
>
> oak = new Oak(documentNodeStore)
> .with(new SolrIndexInitializer(true))
> .with(AggregateIndexProvider.wrap(new
> SolrQueryIndexProvider(solrServerProvider, oakSolrConfigurationProvider)))
> .with(new SolrIndexEditorProvider(solrServerProvider,
> oakSolrConfigurationProvider))
> .with( new NodeStateSolrServersObserver())
> .withAsyncIndexing()
> .with(executorProvider.defaultExecutorService())
> .with(executorProvider.defaultScheduledExecutor());
>
> Repository repo = new Jcr(oak).createRepository();
> return repo;
> }
>
> public class OakExecutorProvider {
> final static Logger logger =
> LoggerFactory.getLogger(OakExecutorProvider.class);
> private ScheduledThreadPoolExecutor scheduledThreadPoolExecutor;
> private ThreadPoolExecutor threadPoolExecutor;
> public ScheduledExecutorService defaultScheduledExecutor() {
> scheduledThreadPoolExecutor = new ScheduledThreadPoolExecutor(32,
> new ThreadFactory() {
> private final AtomicInteger counter = new AtomicInteger();
>
> @Override
> public Thread newThread(@Nonnull Runnable r) {
> Thread thread = new Thread(r, createName());
> thread.setDaemon(true);
> return thread;
> }
>
> private String createName() {
> return "oak-scheduled-executor-" +
> counter.getAndIncrement();
> }
> });
> scheduledThreadPoolExecutor.setKeepAliveTime(1, TimeUnit.MINUTES);
> scheduledThreadPoolExecutor.allowCoreThreadTimeOut(true);
> return scheduledThreadPoolExecutor;
> }
>
> /**
> * Default {@code ExecutorService} used for scheduling concurrent
> tasks.
> * This default spawns as many threads as required with a priority of
> * {@code Thread.MIN_PRIORITY}. Idle threads are pruned after one
> minute.
> * @return fresh ExecutorService
> */
> public ExecutorService defaultExecutorService() {
> threadPoolExecutor = new ThreadPoolExecutor(0, Integer.MAX_VALUE,
> 60L, TimeUnit.SECONDS,
> new SynchronousQueue<Runnable>(), new ThreadFactory() {
> private final AtomicInteger counter = new AtomicInteger();
>
> @Override
> public Thread newThread(@Nonnull Runnable r) {
> Thread thread = new Thread(r, createName());
> thread.setDaemon(true);
> thread.setPriority(Thread.MIN_PRIORITY);
> return thread;
> }
>
> private String createName() {
> return "oak-executor-" + counter.getAndIncrement();
> }
> });
> threadPoolExecutor.setKeepAliveTime(1, TimeUnit.MINUTES);
> threadPoolExecutor.allowCoreThreadTimeOut(true);
> return threadPoolExecutor;
> }
> public void shutdown()
> {
> logger.debug("Initializing shutdown of Oak worker Threads");
> try {
> threadPoolExecutor.shutdown();
> scheduledThreadPoolExecutor.shutdown();
> threadPoolExecutor.awaitTermination(30, TimeUnit.SECONDS);
> scheduledThreadPoolExecutor.awaitTermination(30,
> TimeUnit.SECONDS);
> } catch (InterruptedException ex) {
> logger.error("Executor shutdown error");
> throw new RuntimeException(ex);
> }
> logger.debug("Finished shutdown of Oak worker Threads");
>
> }
>
> }
> -----Ursprüngliche Nachricht-----
> Von: Clay Ferguson [mailto:wcl...@gmail.com]
> Gesendet: Mittwoch, 5. August 2015 16:37
> An: users@jackrabbit.apache.org
> Betreff: Re: Solr Configuration
>
> Leonhard,
> Check out the example FullTextSolrSearchTest.java, and in general search
> the entire codebase for "SolrIndex". I found it impossible to learn
> Jackrabbit by reading docs. The only good way is just download the entire
> source (1.2.3 being latest), and then search for test case examples to see
> how it's done. There are so many different versions of stuff documented
> online you can really only get "hints" until you go directly into the
> source.
> Best regards,
> Clay Ferguson
> meta64.com
>
>
> On Wed, Aug 5, 2015 at 7:55 AM, lw <leonh...@wachutka.eu> wrote:
>
> > Hi all,
> >
> >
> >
> > I am trying to get a remote solr instance working with oak. Because of
> > lack of good documentation, I tried find out the proper configuration
> > myself, however, if I commit new nodes they are not send to the Solr
> > server for indexing. Also no queries seem to be forwarded to the Solar
> > instance. I use the attached code to initialize the repository and use
> > jackrabbit ocm to persist nodes.
> >
> > Debugging shows that SolrIndexEditorProvider gets invoced after a
> > session.save(), but it seems to look for
> > "SolrQueryIndex.TYPE.equals(type)". I don't get what this type is used
> > for and how set up it in the right way.
> >
> > For the first try I just want all properties of my node indexed by Solr.
> > Has someone additional resources, code or hints what I have to do?
> >
> > Thanks in advance for your help
> >
> >
> > Best Leonhard Wachutka
> >
> >
> >
> >
> >
> > protected Repository createRepository() throws Exception {
> >
> > RemoteSolrServerConfiguration remoteSolrConfig = new
> > RemoteSolrServerConfiguration(null, null, 1, 1, null, "
> > http://192.168.2.250:8983/solr/oak";);
> > SolrServerProvider solrServerProvider = new
> > RemoteSolrServerProvider(remoteSolrConfig);
> >
> > OakSolrConfiguration oakSolrconfiguration = new
> > DefaultSolrConfiguration() {
> > @Override
> > public CommitPolicy getCommitPolicy() {
> > return CommitPolicy.HARD;
> > }
> > @Override
> > public boolean useForPropertyRestrictions() {
> > return true;
> > }
> > @Override
> > public boolean useForPrimaryTypes() {
> > return true;
> > }
> > @Override
> > public boolean useForPathRestrictions() {
> > return true;
> > }
> > };
> >
> > OakSolrConfigurationProvider oakSolrConfigurationProvider =
> > new DefaultSolrConfigurationProvider(oakSolrconfiguration);
> >
> > // return JackRabbit repository.
> > log.info("Start of Apache Jackrabbit OAK Repository.\n{host:"
> > + host +"\ndbName:"+ dbName+"}");
> >
> > DB db = new MongoClient(host, 27017).getDB(dbName);
> > documentNodeStore = new DocumentMK.Builder().
> > setMongoDB(db).getNodeStore();
> >
> > oak = new Oak(documentNodeStore)
> > .with(new SolrIndexInitializer(true))
> > .with(AggregateIndexProvider.wrap(new
> > SolrQueryIndexProvider(solrServerProvider,
> oakSolrConfigurationProvider)))
> > .with(new SolrIndexEditorProvider(solrServerProvider,
> > oakSolrConfigurationProvider));
> >
> > Repository repo = new Jcr(oak).createRepository();
> > return repo;
> > }
> >
>
