On Fri, Apr 26, 2013 at 2:57 PM, Kasun Weranga <[email protected]> wrote:
> Hi all,
>
> Since this thread is too long with lots of suggestions. Let me summarize
> all the things that has been discussed in here.
>
> Issue:-
>
> We are trying to give better user experience by handling the port offset
> so user don't have to do anything when deploying a toolbox in the default
> BAM setup after changing the port offset in carbon.xml.
>
> Suggesions and Approaches:-
>
> 1. Current implementation
> Introduced a parameter ('carbon.port.offset') to hive script and by using
> that we can control the port offset depending on whether Cassandra is
> embedded or external.
> disadvantages :- we have to change the hive script
> ('carbon.port.offset'=false) if the deployment is using external cluster.
> change is not scriptable
>
> 2. Change the Cassandra datasource url, so that we can identify the
> embedded and external Cassandra deployment and assign the Cassandra port
> value accordingly.
>
> disadvantages :- we have to change the current Cassandra datasource url
> pattern (jdbc:cassandra://Ip:port/keyspace)and there will be two urls for
> embedded one and external one.
>
>
> 3. Hive will take the decision whether it is embedded or external by
> checking the availability of Cassandra port on the local server.
>
> disadvantage :- In the external Cassandra deployment, there can be a port
> with (<defaultCassandraPort>+<portOffset>) in the local machine. In this
> case there can be an issue.
>
>
> 4. Port offset doesn't handle by Hive. :) User will change it via
> master-datasource.xml
>
> disadvantage :- If a user change the port offset and he is using embedded
> Cassandra then every time he will have to change the Cassandra port value
> in master-datasource.xml.
>
I am +1 for this approach. Asking someone to fix the Cassandra port in
master-datasource.xml if they change the port offset is fair enough, it
falls under configuration management. If there is a cassandra config
section in master-datasource.xml, then this should be modified anyway. This
way the hive scripts stay untouched and give us the flexibility to modify
the Cassandra host/port without breaking existing scripts too.!
>
>
> Thanks,
> KasunW.
>
>
> On Sat, Apr 27, 2013 at 12:01 AM, Kasun Weranga <[email protected]> wrote:
>
>>
>>
>>
>> On Fri, Apr 26, 2013 at 9:42 PM, Sinthuja Ragendran <[email protected]>wrote:
>>
>>>
>>>
>>>
>>> On Fri, Apr 26, 2013 at 8:43 PM, Kasun Weranga <[email protected]> wrote:
>>>
>>>>
>>>>
>>>> On Friday, April 26, 2013, Sinthuja Ragendran wrote:
>>>>
>>>>>
>>>>> On 26 Apr 2013 19:10, "Kasun Weranga" <[email protected]> wrote:
>>>>> >
>>>>> > I was thinking about removing this carbon.port.offset property from
>>>>> hive script and handle this situation by hive itself. What we can do is we
>>>>> can check the availability of cassandra port (<defaultCassandraPort>
>>>>> +<port
>>>>> offset>) and if it is available we can assign that port as the cassandra
>>>>> port.
>>>>>
>>>>> What do you mean by availability of cassandra port? Are you suggesting
>>>>> to check whether the cassandra port is open?
>>>>>
>>>> Yes.
>>>>
>>>
>>> Let say we have two BAM instances in the same node. One server as
>>> analyzer instance and other one as cassandra instance. And from analyzer
>>> instance I need to connect to the cassandra server instance. Since the
>>> cassandra port will also be open in the analyzer instance AFAIU in your
>>> suggested approach, the hive scripts in analyzer node will connect to the
>>> embedded cassandra server of the analyzer instance isn't?
>>>
>>
>> I ignore this case, since if he is using external Cassandra we don't
>> recommend to start the Cassandra server in analyzer node. Anyway there is a
>> issue if this connect to some other port in the same machine there can be a
>> rare possibility for that.
>>
>> Anyway now this thread seems filled with lots of suggestions :) and it is
>> difficult for others to follow.
>>
>>
>> And how can we change it to connect to the cassandra instance from
>>> analyzer instance in your suggested approach? Can you please clear me on
>>> this on your approach.
>>>
>>> Thanks,
>>> Sinthuja.
>>>
>>>
>>>
>>> Thanks,
>>>>> Sinthuja.
>>>>>
>>>>> If it is not available then we can use the port value define in the
>>>>> Cassandra data source.
>>>>> > This will handle both embedded and external cassandra usecase
>>>>> without changing the hive script and existing Cassandra datasource url
>>>>> pattern. I think this is much cleaner way to handle this situation. Isn't
>>>>> it?
>>>>> >
>>>>> > On Friday, April 26, 2013, Supun Malinga wrote:
>>>>> >>
>>>>> >> Hi,
>>>>> >>
>>>>> >> Couldn't read all the replies thoroughly.
>>>>> >> Do you suggest that we can define the port in
>>>>> master-datasources.xml?.
>>>>> >>
>>>>> >> About user mistakes I believe proper documenting how to handle a
>>>>> portoff set scenario would be a better option.
>>>>> >>
>>>>> >> thanks,
>>>>> >>
>>>>> >> On Fri, Apr 26, 2013 at 4:06 PM, Chamara Ariyarathne <
>>>>> [email protected]> wrote:
>>>>> >>>
>>>>> >>>
>>>>> >>>
>>>>> >>> On Fri, Apr 26, 2013 at 4:04 PM, Sinthuja Ragendran <
>>>>> [email protected]> wrote:
>>>>> >>>>
>>>>> >>>>
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> On Fri, Apr 26, 2013 at 3:50 PM, Kasun Weranga <[email protected]>
>>>>> wrote:
>>>>> >>>>>
>>>>> >>>>> We introduce this carbon.port.offset property to prevent issues
>>>>> due to port offset for users who are completely new to WSO2 BAM product.
>>>>> In
>>>>> this case they don't need to change the hive scripts after they change the
>>>>> port offset because they are using embedded Cassandra and port values
>>>>> change with the port offset value.
>>>>> >>>>>
>>>>> >>>>> Currently we are facing this issue in API-M since they start BAM
>>>>> in different offsets. That is the main requirement for adding this
>>>>> functionality.
>>>>> >>>>>
>>>>> >>>>> But we need to disable port offset changes for user who are
>>>>> using external Cassandra. These users are advance users and I think it is
>>>>> ok for telling them to change the hive script to disable it.
>>>>> (carbon.port.offset='false')
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> This property is going to be in each cassandra external tables
>>>>> defined, which is same as 'cassandra.port' property. Therefore there may
>>>>> be
>>>>> many places required to change if we connecting to external cassandra.
>>>>> >>>>
>>>>> >>>> I would prefer this change to be in one single place. IMHO it's
>>>>> better to remove the port from the url or having some property along with
>>>>> the url such as
>>>>> 'jdbc:cassandra://localhost:9160/EVENT_KS;APPLY_PORT_OFFSET=TRUE' would be
>>>>> better.
>>>>> >>>>
>>>>> >>> APPLY_PORT_OFFSET=TRUE in a database connection url?
>>>>> >>>
>>>>> >>> I don't think that is good!
>>>>> >>>
>>>>> >>>>
>>>>> >>>> Thanks,
>>>>> >>>> Sinthuja.
>>>>> >>>>
>>>>> >>>>
>>>>> >>>>
>>>>> >>>>>
>>>>> >>>>> I am not sure changing the cassandra datasource url according
>>>>> our requirement is the right thing to do. If we do that we need to change
>>>>> the datasource implementation too and also there will be two way of
>>>>> defining Cassandra url.
>>>>> >>>>>
>>>>> >>>>> I feel it is ok for advance BAM users to edit this value in the
>>>>> hive script.
>>>>> >>>>>
>>>>> >>>>> Thanks,
>>>>> >>>>> KasunW.
>>>>> >>>>>
>>>>> >>>>>
>>>>> >>>>>
>>>>> >>>>> On Fri, Apr 26, 2013 at 3:30 PM, Sinthuja Ragendran <
>>>>> [email protected]> wrote:
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> >>>>>> On Fri, Apr 26, 2013 at 3:18 PM, Ayashkantha Ramasinghe <
>>>>> [email protected]> wrote:
>>>>> >>>>>>>
>>>>> >>>>>>> Hi Sinthuja,
>>>>> >>>>>>>
>>>>> >>>>>>> After Kasun apply new patch, that problem will be solved.
>>>>> Because, after reading master-datasource.xml the cassandra handler is
>>>>> adding the port offset internally.
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> >>>>>> Here this property is added in the hive script within
>>>>> SERDEPROPERTIES of cassandra external table, isn't it?
>>>>> >>>>>>
>>>>> >> --
>>>>> >> Supun Malinga,
>>>>> >>
>>>>> >> Software Engineer,
>>>>> >> WSO2 Inc.
>>>>> >> http://wso2.com
>>>>> >> <http://wso2.org>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Kasun Weranga*
>>>> **
>>>> Member, Management Committee - Data Technologies
>>>> Software Engineer
>>>> *WSO2, Inc.
>>>> *lean.enterprise.middleware.
>>>> mobile : +94 772314602
>>>> <http://sanjeewamalalgoda.blogspot.com/>blog :
>>>> http://kasunweranga.blogspot.com/
>>>>
>>>
>>>
>>>
>>> --
>>> *Sinthuja Rajendran*
>>> Software Engineer <http://wso2.com/>
>>> WSO2, Inc.:http://wso2.com
>>>
>>> Blog: http://sinthu-rajan.blogspot.com/
>>> Mobile: +94774273955
>>>
>>>
>>>
>>
>>
>> --
>> *Kasun Weranga*
>> **
>> Member, Management Committee - Data Technologies
>> Software Engineer
>> *WSO2, Inc.
>> *lean.enterprise.middleware.
>> mobile : +94 772314602
>> <http://sanjeewamalalgoda.blogspot.com/>blog :
>> http://kasunweranga.blogspot.com/
>>
>
>
>
> --
> *Kasun Weranga*
> **
> Member, Management Committee - Data Technologies
> Software Engineer
> *WSO2, Inc.
> *lean.enterprise.middleware.
> mobile : +94 772314602
> <http://sanjeewamalalgoda.blogspot.com/>blog :
> http://kasunweranga.blogspot.com/
>
> _______________________________________________
> Dev mailing list
> [email protected]
> http://wso2.org/cgi-bin/mailman/listinfo/dev
>
>
--
Thanks,
Shariq.
Phone: +94 777 202 225
_______________________________________________
Dev mailing list
[email protected]
http://wso2.org/cgi-bin/mailman/listinfo/dev
