Hi Mingda, Just out of curiosity can you load somehow "large" dataset and query it on the 16 nodes (e.g query the number of records in the dataset). I want to make sure all NCs are participating. If that succeed, probably you should try to load the same data using a localfs feed and run query after the ingestion finishes.
This will help us to isolate the problem if it's a twitter feed problem or something else. On Fri, Oct 14, 2016 at 10:13 AM, Xikui Wang <[email protected]> wrote: > Actually we have talked, and I don't have clear answer. The current twitter > adaptor is getting node from (IExternalDataSourceFactory) > AsterixAppContextInfo.INSTANCE.getMetadataProperties(). > getStores().keySet() > and select the first partition from that node based on > ClusterStateManager.INSTANCE.getIODevices(node).length > > The partition constraints of intake adaptor will only have the first node, > 'nc1' in local case. Meaning it will run on the first node only. It's not > obvious to me why it's working on 4 nodes but not 16 nodes... > > Best, > Xikui > > On Thu, Oct 13, 2016 at 11:16 PM, Mike Carey <[email protected]> wrote: > > > +dev > > > > @Xikui: Can you help Mingda? > > > > > > > > On 10/13/16 7:45 PM, mingda li wrote: > > > >> Hi, > >> > >> Recently, I am trying to run the data feed query on a cluster of 16 > >> nodes. All the query run well without error. But each time after I > >> disconnect the feed, there is none of tweet data stored in dataverse. > >> However, when I try to run other AQL queries, the cluster can work well. > >> And I have ever used 4 nodes cluster to load Tweet data successfully. I > >> also checked the log file and find no error there. This is wired. Does > >> anyone know why? Has anyone ever used the data feed function on a > cluster > >> of 16 nodes or more? > >> > >> Bests, > >> Mingda > >> > > > > > -- *Regards,* Wail Alkowaileet
