On Sat, Oct 11, 2014 at 6:32 PM, Asitha Nanayakkara <[email protected]> wrote:
> Hi Hasitha, > > On Sat, Oct 11, 2014 at 2:50 PM, Hasitha Hiranya <[email protected]> > wrote: > >> Hi Devs, >> >> Here are some more tasks we are planning to do along milestone releases >> of MB. >> >> 1. Both come as coodinator when in localhost (should consider port as >> well when slecting coodinator) >> 2. Throw necessary SQL exceptions in JDBC (handled by Asitha now I >> believe) >> > > When in clustered mode if in memory mode is used an exception is thrown > and the message stores won't get initialized. And each method in the RDBMS > store classes throws AndesException when an SQLException occur. > AFIR you were missing stack Trace from JDBC driver. Maybe this is fixed now. > > >> 3. Need to catch exception (most generic ) and throw >> CassandaDataAccessException in CQLDataAccessHelper >> 4. Misleading log - in-memory message store initialized when configured >> with Cassandra >> > > This Log is due to the in memory message store that was there in > MessagingEngine. We have removed it. With that change integrated to the > pack this log and the initialisation of in memory message store won't > happen. > +1 > > 5. Adjust packaging and put these as debug in log4j.propeties separated as >> kernel, amqp and mqtt >> >> log4j.logger.org.wso2.andes.server.cassandra.QueueDeliveryWorker=DEBUG >> log4j.logger.org.wso2.andes.server.cassandra.SlotDeliveryWorker=DEBUG >> >> log4j.logger.org.wso2.andes.server.cassandra.OnflightMessageTracker=DEBUG >> log4j.logger.org.wso2.andes.server.cluster.coordination.hazelcast=DEBUG >> >> We need to separate per message logs as trace logs? We did this in MB >> 2.2.0. >> >> 6. Make Onflight Message Tracker NOT AMQP specific. >> >> Thanks >> >> On Thu, Oct 2, 2014 at 11:13 AM, Sriskandarajah Suhothayan <[email protected] >> > wrote: >> >>> >>> >>> On Thu, Oct 2, 2014 at 11:06 AM, Asitha Nanayakkara <[email protected]> >>> wrote: >>> >>>> Hi Hasitha, >>>> >>>> Concerns regarding *deleteMessageMetadataFromQueue(final String >>>> queueName,List<AndesRemovableMetadata> messagesToRemove)* >>>> >>>> This method is used to delete meta data from a specific queue *queueName >>>> (Note: *queueName in actual meta data would be different*). *This >>>> methods' use case is to delete messages from a DLC. Note that there are >>>> methods to add meta data to DLC too. (parameter to give the specific queue >>>> to be stored to) >>>> >>>> *public void addMetaDataToQueue(final String queueName, >>>> AndesMessageMetadata metadata) throws AndesException;* >>>> *public void addMetadataToQueue(final String queueName, >>>> List<AndesMessageMetadata> metadata) throws AndesException;* >>>> >>>> For the normal use case, to delete metadata there is no separate >>>> method. We use the deleteMessages() method (meta data and content is >>>> removed through this method). But if we are going to avoid content >>>> duplication for topics in future we will need to add a method to >>>> deleteMetadata as well (without specific queue name parameter) >>>> >>>> >>>> +1 >>> >>> Suho >>> >>> >>>> On Wed, Oct 1, 2014 at 8:53 PM, Pamod Sylvester <[email protected]> wrote: >>>> >>>>> I've also got started on some re factoring of MQTT related logic we've >>>>> implemented to fit in with the new changes. >>>>> >>>>> We also might want to further abstract out ack handling logic, since >>>>> in MQTT the way acks are handled are quite different. So we need to >>>>> address >>>>> such polymorphic behaviours as well. >>>>> >>>>> for ex : - in AMQP when an ack arrives we purge the message, but in >>>>> MQTT in QOS level 2 we wait for a couple of ack exchanges to consider the >>>>> message being delivered. >>>>> >>>>> >>>>> >>>>> On Wed, Oct 1, 2014 at 7:13 PM, Hasitha Hiranya <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I am currently doing the changes suggested by Suho and the team >>>>>> >>>>>> 1. Introducing a Strategy Class for message store >>>>>> (async/direct/hybrid). >>>>>> - So far, I could make async strategy class done. >>>>>> 2. remove business logics and threads from message store and make it >>>>>> plain - done >>>>>> 3. remove ackReceived from store level - done >>>>>> 4. removed topic delivery worker - let us rewrite or merge to queue >>>>>> delivery worker >>>>>> 5. moved message counting from message store and moved to context >>>>>> store >>>>>> 6. prevented message store access from outside >>>>>> >>>>>> On the way I had to make modifications and improvements to the >>>>>> existing model. >>>>>> Also cleaned up Global queues whenever I met going thro the code when >>>>>> doing above change. >>>>>> >>>>>> There are still changes needs to be done. >>>>>> >>>>>> >> what to do when last subscriber goes >>>>>> >> for topic in mbeans how to get message count >>>>>> >> for topics in mbeans for subscriptions how to get pending message >>>>>> counts >>>>>> >> queue browserdelivery worker, need to rewrite getSortedMessages() >>>>>> >>>>>> >> ack received moved to messaging engine. Logic resides at messaging >>>>>> engine (tracking aspect) and message store manager (store aspect) >>>>>> >> reimplement topic delivery worker >>>>>> >>>>>> messageStore or message store manager is never given out of message >>>>>> engine. Do not call them directly. Always use MessagingEngine for message >>>>>> operations. >>>>>> >>>>>> public void deleteMessages method in message store has a logic. >>>>>> Should be removed. >>>>>> >>>>>> public void deleteMessageMetadataFromQueue(final String >>>>>> queueName,List<AndesRemovableMetadata> messagesToRemove) why there is a >>>>>> queue name? AndesRemovableMetadata has queue name inside. >>>>>> >>>>>> QueueSubscriptionAcknowledgementHandler need to revisit this class. >>>>>> What does this do?? It has threads etc that never stops >>>>>> >>>>>> Should remove all global queue related stuff from cluster manager >>>>>> >>>>>> should remove all zookeeper related things from whole project up to >>>>>> MB product from dependancy level >>>>>> >>>>>> moved all message count related stuff to Context store. Now, need to >>>>>> implement thsose for jdbc. >>>>>> >>>>>> deleteMessageMetadataFromQueue(queueName,removableMetadata) >>>>>> @MessageStore. Why queue name? We should get rid of it and implement. >>>>>> >>>>>> Thanks >>>>>> >>>>>> On Wed, Oct 1, 2014 at 9:25 AM, Darshana Akalanka Pagoda Arachchi < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> Also I think we need to take a look at the CQL Database schema since >>>>>>> for most of the column families including Meta Data column family and >>>>>>> Message Content Column family we use the same schema. Because of this >>>>>>> the >>>>>>> message Id which is the primary key in Message Content column family >>>>>>> has to >>>>>>> be converted into String (which is Long) before saving. >>>>>>> >>>>>>> On Tue, Sep 30, 2014 at 10:56 PM, Asitha Nanayakkara < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi Shammi, >>>>>>>> >>>>>>>> In the existing version of MB, for topics we duplicate the content >>>>>>>> and metadata. We thought of having reference counting to avoid >>>>>>>> duplication >>>>>>>> of message content. But implementing reference counting in Cassandra is >>>>>>>> unreliable (JDBC is ok). Also having different logic for different >>>>>>>> database >>>>>>>> types is inconsistent with what we wanted to achieve through the new >>>>>>>> storage model (with abstracted out MessageStore interface and storing >>>>>>>> strategies). >>>>>>>> >>>>>>>> One of the options would be to duplicate metadata with a reference >>>>>>>> to the same content (need a new field in metadata). One of the >>>>>>>> concerns is >>>>>>>> how to delete content safely without deleting content prior to all the >>>>>>>> subscribers, for the topic, successfully receive the message. >>>>>>>> >>>>>>>> I think we will have to discuss and relook at what is the best >>>>>>>> approach for this. >>>>>>>> >>>>>>>> Thanks >>>>>>>> >>>>>>>> On Tue, Sep 30, 2014 at 4:21 PM, Shammi Jayasinghe <[email protected] >>>>>>>> > wrote: >>>>>>>> >>>>>>>>> Hi Hasitha, >>>>>>>>> >>>>>>>>> Could you please clarify what you mention my duplicating message >>>>>>>>> metadata and content. >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> Shammi >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tue, Sep 30, 2014 at 11:21 AM, Hasitha Hiranya < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi Devs, >>>>>>>>>> >>>>>>>>>> For MB 3.0.0 we have identified following TODOs for the moment on >>>>>>>>>> the way to milestone 1. >>>>>>>>>> >>>>>>>>>> >>duplicate message metadata and content >>>>>>>>>> >>message store manager (all methods in message store to be moved >>>>>>>>>> to manager??) >>>>>>>>>> >>"durable" for in-memory >>>>>>>>>> >>depreciated methods in message store remove >>>>>>>>>> >>Cassandra server feature remove with Cassandra profile >>>>>>>>>> >>Ackreceived - should be removed from message store >>>>>>>>>> >>remove all logics from message store and make it plain >>>>>>>>>> >>why acks are sent to disruptor?? >>>>>>>>>> >>>>>>>>>> >>topic integration >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> -- >>>>>>>>>> *Hasitha Abeykoon* >>>>>>>>>> Senior Software Engineer; WSO2, Inc.; http://wso2.com >>>>>>>>>> *cell:* *+94 719363063* >>>>>>>>>> *blog: **abeykoon.blogspot.com* <http://abeykoon.blogspot.com> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Best Regards, >>>>>>>>> >>>>>>>>> * Shammi Jayasinghe* >>>>>>>>> Associate Tech Lead >>>>>>>>> WSO2, Inc.; http://wso2.com, >>>>>>>>> mobile: +94 71 4493085 >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> *Asitha Nanayakkara* >>>>>>>> Software Engineer >>>>>>>> WSO2, Inc. http://wso2.com/ >>>>>>>> Mob: + 94 77 85 30 682 >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Dev mailing list >>>>>>>> [email protected] >>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *Darshana Akalanka Pagoda Arachchi,* >>>>>>> *Software Engineer* >>>>>>> *078-4721791 <078-4721791>* >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Dev mailing list >>>>>>> [email protected] >>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Hasitha Abeykoon* >>>>>> Senior Software Engineer; WSO2, Inc.; http://wso2.com >>>>>> *cell:* *+94 719363063* >>>>>> *blog: **abeykoon.blogspot.com* <http://abeykoon.blogspot.com> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Dev mailing list >>>>>> [email protected] >>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> *Pamod Sylvester * >>>>> *Senior Software Engineer * >>>>> Integration Technologies Team, WSO2 Inc.; http://wso2.com >>>>> email: [email protected] cell: +94 77 7779495 >>>>> >>>>> _______________________________________________ >>>>> Dev mailing list >>>>> [email protected] >>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>> >>>>> >>>> >>>> >>>> -- >>>> *Asitha Nanayakkara* >>>> Software Engineer >>>> WSO2, Inc. http://wso2.com/ >>>> Mob: + 94 77 85 30 682 >>>> >>>> >>>> _______________________________________________ >>>> Dev mailing list >>>> [email protected] >>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>> >>>> >>> >>> >>> -- >>> >>> *S. Suhothayan* >>> Technical Lead & Team Lead of WSO2 Complex Event Processor >>> *WSO2 Inc. *http://wso2.com >>> * <http://wso2.com/>* >>> lean . enterprise . middleware >>> >>> >>> *cell: (+94) 779 756 757 <%28%2B94%29%20779%20756%20757> | blog: >>> http://suhothayan.blogspot.com/ <http://suhothayan.blogspot.com/>twitter: >>> http://twitter.com/suhothayan <http://twitter.com/suhothayan> | linked-in: >>> http://lk.linkedin.com/in/suhothayan <http://lk.linkedin.com/in/suhothayan>* >>> >> >> >> >> -- >> *Hasitha Abeykoon* >> Senior Software Engineer; WSO2, Inc.; http://wso2.com >> *cell:* *+94 719363063* >> *blog: **abeykoon.blogspot.com* <http://abeykoon.blogspot.com> >> >> > Thanks > > -- > *Asitha Nanayakkara* > Software Engineer > WSO2, Inc. http://wso2.com/ > Mob: + 94 77 85 30 682 > > -- *Hasitha Abeykoon* Senior Software Engineer; WSO2, Inc.; http://wso2.com *cell:* *+94 719363063* *blog: **abeykoon.blogspot.com* <http://abeykoon.blogspot.com>
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
