Hi Mingda,

Just out of curiosity can you load somehow "large" dataset and query it on
the 16 nodes (e.g query the number of records in the dataset). I want to
make sure all NCs are participating.
If that succeed, probably you should try to load the same data using a
localfs feed and run query after the ingestion finishes.

This will help us to isolate the problem if it's a twitter feed problem or
something else.


On Fri, Oct 14, 2016 at 10:13 AM, Xikui Wang <xik...@uci.edu> wrote:

> Actually we have talked, and I don't have clear answer. The current twitter
> adaptor is getting node from (IExternalDataSourceFactory)
> AsterixAppContextInfo.INSTANCE.getMetadataProperties().
> getStores().keySet()
> and select the first partition from that node based on
> ClusterStateManager.INSTANCE.getIODevices(node).length
>
> The partition constraints of intake adaptor will only have the first node,
> 'nc1' in local case. Meaning it will run on the first node only. It's not
> obvious to me why it's working on 4 nodes but not 16 nodes...
>
> Best,
> Xikui
>
> On Thu, Oct 13, 2016 at 11:16 PM, Mike Carey <dtab...@gmail.com> wrote:
>
> > +dev
> >
> > @Xikui:  Can you help Mingda?
> >
> >
> >
> > On 10/13/16 7:45 PM, mingda li wrote:
> >
> >> Hi,
> >>
> >> Recently, I am trying to run the data feed query on a cluster of 16
> >> nodes. All the query run well without error. But each time after I
> >> disconnect the feed, there is none of tweet data stored in dataverse.
> >> However, when I try to run other AQL queries, the cluster can work well.
> >> And I have ever used 4 nodes cluster to load Tweet data successfully. I
> >> also checked the log file and find no error there. This is wired. Does
> >> anyone know why? Has anyone ever used the data feed function on a
> cluster
> >> of 16 nodes or more?
> >>
> >> Bests,
> >> Mingda
> >>
> >
> >
>



-- 

*Regards,*
Wail Alkowaileet

Reply via email to