On Sat, Oct 11, 2014 at 8:52 PM, Hasitha Hiranya <[email protected]> wrote:
> > > On Sat, Oct 11, 2014 at 6:32 PM, Asitha Nanayakkara <[email protected]> > wrote: > >> Hi Hasitha, >> >> On Sat, Oct 11, 2014 at 2:50 PM, Hasitha Hiranya <[email protected]> >> wrote: >> >>> Hi Devs, >>> >>> Here are some more tasks we are planning to do along milestone releases >>> of MB. >>> >>> 1. Both come as coodinator when in localhost (should consider port as >>> well when slecting coodinator) >>> 2. Throw necessary SQL exceptions in JDBC (handled by Asitha now I >>> believe) >>> >> >> When in clustered mode if in memory mode is used an exception is thrown >> and the message stores won't get initialized. And each method in the RDBMS >> store classes throws AndesException when an SQLException occur. >> > > AFIR you were missing stack Trace from JDBC driver. Maybe this is fixed > now. > Yes. That is fixed. > > >> >> >>> 3. Need to catch exception (most generic ) and throw >>> CassandaDataAccessException in CQLDataAccessHelper >>> 4. Misleading log - in-memory message store initialized when configured >>> with Cassandra >>> >> >> This Log is due to the in memory message store that was there in >> MessagingEngine. We have removed it. With that change integrated to the >> pack this log and the initialisation of in memory message store won't >> happen. >> > > +1 > >> >> 5. Adjust packaging and put these as debug in log4j.propeties separated >>> as kernel, amqp and mqtt >>> >>> log4j.logger.org.wso2.andes.server.cassandra.QueueDeliveryWorker=DEBUG >>> log4j.logger.org.wso2.andes.server.cassandra.SlotDeliveryWorker=DEBUG >>> >>> log4j.logger.org.wso2.andes.server.cassandra.OnflightMessageTracker=DEBUG >>> >>> log4j.logger.org.wso2.andes.server.cluster.coordination.hazelcast=DEBUG >>> >>> We need to separate per message logs as trace logs? We did this in MB >>> 2.2.0. >>> >>> 6. Make Onflight Message Tracker NOT AMQP specific. >>> >>> Thanks >>> >>> On Thu, Oct 2, 2014 at 11:13 AM, Sriskandarajah Suhothayan < >>> [email protected]> wrote: >>> >>>> >>>> >>>> On Thu, Oct 2, 2014 at 11:06 AM, Asitha Nanayakkara <[email protected]> >>>> wrote: >>>> >>>>> Hi Hasitha, >>>>> >>>>> Concerns regarding *deleteMessageMetadataFromQueue(final String >>>>> queueName,List<AndesRemovableMetadata> messagesToRemove)* >>>>> >>>>> This method is used to delete meta data from a specific queue *queueName >>>>> (Note: *queueName in actual meta data would be different*). *This >>>>> methods' use case is to delete messages from a DLC. Note that there are >>>>> methods to add meta data to DLC too. (parameter to give the specific queue >>>>> to be stored to) >>>>> >>>>> *public void addMetaDataToQueue(final String queueName, >>>>> AndesMessageMetadata metadata) throws AndesException;* >>>>> *public void addMetadataToQueue(final String queueName, >>>>> List<AndesMessageMetadata> metadata) throws AndesException;* >>>>> >>>>> For the normal use case, to delete metadata there is no separate >>>>> method. We use the deleteMessages() method (meta data and content is >>>>> removed through this method). But if we are going to avoid content >>>>> duplication for topics in future we will need to add a method to >>>>> deleteMetadata as well (without specific queue name parameter) >>>>> >>>>> >>>>> +1 >>>> >>>> Suho >>>> >>>> >>>>> On Wed, Oct 1, 2014 at 8:53 PM, Pamod Sylvester <[email protected]> >>>>> wrote: >>>>> >>>>>> I've also got started on some re factoring of MQTT related logic >>>>>> we've implemented to fit in with the new changes. >>>>>> >>>>>> We also might want to further abstract out ack handling logic, since >>>>>> in MQTT the way acks are handled are quite different. So we need to >>>>>> address >>>>>> such polymorphic behaviours as well. >>>>>> >>>>>> for ex : - in AMQP when an ack arrives we purge the message, but in >>>>>> MQTT in QOS level 2 we wait for a couple of ack exchanges to consider the >>>>>> message being delivered. >>>>>> >>>>>> >>>>>> >>>>>> On Wed, Oct 1, 2014 at 7:13 PM, Hasitha Hiranya <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I am currently doing the changes suggested by Suho and the team >>>>>>> >>>>>>> 1. Introducing a Strategy Class for message store >>>>>>> (async/direct/hybrid). >>>>>>> - So far, I could make async strategy class done. >>>>>>> 2. remove business logics and threads from message store and make it >>>>>>> plain - done >>>>>>> 3. remove ackReceived from store level - done >>>>>>> 4. removed topic delivery worker - let us rewrite or merge to queue >>>>>>> delivery worker >>>>>>> 5. moved message counting from message store and moved to context >>>>>>> store >>>>>>> 6. prevented message store access from outside >>>>>>> >>>>>>> On the way I had to make modifications and improvements to the >>>>>>> existing model. >>>>>>> Also cleaned up Global queues whenever I met going thro the code >>>>>>> when doing above change. >>>>>>> >>>>>>> There are still changes needs to be done. >>>>>>> >>>>>>> >> what to do when last subscriber goes >>>>>>> >> for topic in mbeans how to get message count >>>>>>> >> for topics in mbeans for subscriptions how to get pending message >>>>>>> counts >>>>>>> >> queue browserdelivery worker, need to rewrite getSortedMessages() >>>>>>> >>>>>>> >> ack received moved to messaging engine. Logic resides at >>>>>>> messaging engine (tracking aspect) and message store manager (store >>>>>>> aspect) >>>>>>> >> reimplement topic delivery worker >>>>>>> >>>>>>> messageStore or message store manager is never given out of message >>>>>>> engine. Do not call them directly. Always use MessagingEngine for >>>>>>> message >>>>>>> operations. >>>>>>> >>>>>>> public void deleteMessages method in message store has a logic. >>>>>>> Should be removed. >>>>>>> >>>>>>> public void deleteMessageMetadataFromQueue(final String >>>>>>> queueName,List<AndesRemovableMetadata> messagesToRemove) why there is a >>>>>>> queue name? AndesRemovableMetadata has queue name inside. >>>>>>> >>>>>>> QueueSubscriptionAcknowledgementHandler need to revisit this class. >>>>>>> What does this do?? It has threads etc that never stops >>>>>>> >>>>>>> Should remove all global queue related stuff from cluster manager >>>>>>> >>>>>>> should remove all zookeeper related things from whole project up to >>>>>>> MB product from dependancy level >>>>>>> >>>>>>> moved all message count related stuff to Context store. Now, need to >>>>>>> implement thsose for jdbc. >>>>>>> >>>>>>> deleteMessageMetadataFromQueue(queueName,removableMetadata) >>>>>>> @MessageStore. Why queue name? We should get rid of it and implement. >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> On Wed, Oct 1, 2014 at 9:25 AM, Darshana Akalanka Pagoda Arachchi < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> Also I think we need to take a look at the CQL Database schema >>>>>>>> since for most of the column families including Meta Data column >>>>>>>> family and >>>>>>>> Message Content Column family we use the same schema. Because of this >>>>>>>> the >>>>>>>> message Id which is the primary key in Message Content column family >>>>>>>> has to >>>>>>>> be converted into String (which is Long) before saving. >>>>>>>> >>>>>>>> On Tue, Sep 30, 2014 at 10:56 PM, Asitha Nanayakkara < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi Shammi, >>>>>>>>> >>>>>>>>> In the existing version of MB, for topics we duplicate the content >>>>>>>>> and metadata. We thought of having reference counting to avoid >>>>>>>>> duplication >>>>>>>>> of message content. But implementing reference counting in Cassandra >>>>>>>>> is >>>>>>>>> unreliable (JDBC is ok). Also having different logic for different >>>>>>>>> database >>>>>>>>> types is inconsistent with what we wanted to achieve through the new >>>>>>>>> storage model (with abstracted out MessageStore interface and storing >>>>>>>>> strategies). >>>>>>>>> >>>>>>>>> One of the options would be to duplicate metadata with a reference >>>>>>>>> to the same content (need a new field in metadata). One of the >>>>>>>>> concerns is >>>>>>>>> how to delete content safely without deleting content prior to all the >>>>>>>>> subscribers, for the topic, successfully receive the message. >>>>>>>>> >>>>>>>>> I think we will have to discuss and relook at what is the best >>>>>>>>> approach for this. >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> >>>>>>>>> On Tue, Sep 30, 2014 at 4:21 PM, Shammi Jayasinghe < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi Hasitha, >>>>>>>>>> >>>>>>>>>> Could you please clarify what you mention my duplicating message >>>>>>>>>> metadata and content. >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> Shammi >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tue, Sep 30, 2014 at 11:21 AM, Hasitha Hiranya < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Devs, >>>>>>>>>>> >>>>>>>>>>> For MB 3.0.0 we have identified following TODOs for the moment >>>>>>>>>>> on the way to milestone 1. >>>>>>>>>>> >>>>>>>>>>> >>duplicate message metadata and content >>>>>>>>>>> >>message store manager (all methods in message store to be >>>>>>>>>>> moved to manager??) >>>>>>>>>>> >>"durable" for in-memory >>>>>>>>>>> >>depreciated methods in message store remove >>>>>>>>>>> >>Cassandra server feature remove with Cassandra profile >>>>>>>>>>> >>Ackreceived - should be removed from message store >>>>>>>>>>> >>remove all logics from message store and make it plain >>>>>>>>>>> >>why acks are sent to disruptor?? >>>>>>>>>>> >>>>>>>>>>> >>topic integration >>>>>>>>>>> >>>>>>>>>>> Thanks >>>>>>>>>>> -- >>>>>>>>>>> *Hasitha Abeykoon* >>>>>>>>>>> Senior Software Engineer; WSO2, Inc.; http://wso2.com >>>>>>>>>>> *cell:* *+94 719363063* >>>>>>>>>>> *blog: **abeykoon.blogspot.com* <http://abeykoon.blogspot.com> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Best Regards, >>>>>>>>>> >>>>>>>>>> * Shammi Jayasinghe* >>>>>>>>>> Associate Tech Lead >>>>>>>>>> WSO2, Inc.; http://wso2.com, >>>>>>>>>> mobile: +94 71 4493085 >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> *Asitha Nanayakkara* >>>>>>>>> Software Engineer >>>>>>>>> WSO2, Inc. http://wso2.com/ >>>>>>>>> Mob: + 94 77 85 30 682 >>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Dev mailing list >>>>>>>>> [email protected] >>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> *Darshana Akalanka Pagoda Arachchi,* >>>>>>>> *Software Engineer* >>>>>>>> *078-4721791 <078-4721791>* >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Dev mailing list >>>>>>>> [email protected] >>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *Hasitha Abeykoon* >>>>>>> Senior Software Engineer; WSO2, Inc.; http://wso2.com >>>>>>> *cell:* *+94 719363063* >>>>>>> *blog: **abeykoon.blogspot.com* <http://abeykoon.blogspot.com> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Dev mailing list >>>>>>> [email protected] >>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Pamod Sylvester * >>>>>> *Senior Software Engineer * >>>>>> Integration Technologies Team, WSO2 Inc.; http://wso2.com >>>>>> email: [email protected] cell: +94 77 7779495 >>>>>> >>>>>> _______________________________________________ >>>>>> Dev mailing list >>>>>> [email protected] >>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> *Asitha Nanayakkara* >>>>> Software Engineer >>>>> WSO2, Inc. http://wso2.com/ >>>>> Mob: + 94 77 85 30 682 >>>>> >>>>> >>>>> _______________________________________________ >>>>> Dev mailing list >>>>> [email protected] >>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>> >>>>> >>>> >>>> >>>> -- >>>> >>>> *S. Suhothayan* >>>> Technical Lead & Team Lead of WSO2 Complex Event Processor >>>> *WSO2 Inc. *http://wso2.com >>>> * <http://wso2.com/>* >>>> lean . enterprise . middleware >>>> >>>> >>>> *cell: (+94) 779 756 757 <%28%2B94%29%20779%20756%20757> | blog: >>>> http://suhothayan.blogspot.com/ <http://suhothayan.blogspot.com/>twitter: >>>> http://twitter.com/suhothayan <http://twitter.com/suhothayan> | linked-in: >>>> http://lk.linkedin.com/in/suhothayan >>>> <http://lk.linkedin.com/in/suhothayan>* >>>> >>> >>> >>> >>> -- >>> *Hasitha Abeykoon* >>> Senior Software Engineer; WSO2, Inc.; http://wso2.com >>> *cell:* *+94 719363063* >>> *blog: **abeykoon.blogspot.com* <http://abeykoon.blogspot.com> >>> >>> >> Thanks >> >> -- >> *Asitha Nanayakkara* >> Software Engineer >> WSO2, Inc. http://wso2.com/ >> Mob: + 94 77 85 30 682 >> >> > > > -- > *Hasitha Abeykoon* > Senior Software Engineer; WSO2, Inc.; http://wso2.com > *cell:* *+94 719363063* > *blog: **abeykoon.blogspot.com* <http://abeykoon.blogspot.com> > > -- *Asitha Nanayakkara* Software Engineer WSO2, Inc. http://wso2.com/ Mob: + 94 77 85 30 682
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
