[Project Clearwater] The new project-clearwater-issues repository

[Eleanor Merry (projectclearwater.org)]

Hi All,

As we've developed Project Clearwater, we've added a fair number of new GitHub 
repositories. Each of these repositories had its own issue tracker, with a 
small number of issues per repo.

However, we've found that this model isn't actually that helpful, as it's 
really quite hard to work out which repository an issue should be raised 
against, and there's no inbuilt support in GitHub to easily move an issue from 
one repository to another. It's also harder to keep track of all the issues 
across the different repositories.

To simplify this, we've added a new Project-Clearwater-Issues repository - 
https://github.com/Metaswitch/project-clearwater-issues and migrated community 
raised issues over to this. We're going to use this going forward to track any 
issues in Project Clearwater. If you encounter any issues with Project 
Clearwater, you can either raise them in this repository, or email the mailing 
list for help.

Ellie

___
Clearwater mailing list
Clearwater@lists.projectclearwater.org
http://lists.projectclearwater.org/mailman/listinfo/clearwater_lists.projectclearwater.org

[Project Clearwater] Release note for Umbreon

[Eleanor Merry (projectclearwater.org)]

The release for Project Clearwater sprint "Umbreon" has been cut. The code for 
this release is tagged as release-114 in GitHub.

In this release we've:

* Updated our API documentation for the new cached data API (at 
https://github.com/Metaswitch/sprout/blob/dev/docs/ManagementHttpAPI.md and 
https://github.com/Metaswitch/homestead/blob/dev/docs/ManagementHttpAPI.md)

* Improved the process for recovering the etcd cluster if it's lost 
quorum - the new, more reliable process is at 
http://clearwater.readthedocs.io/en/stable/Handling_Multiple_Failed_Nodes.html.

This release also includes the following bug fixes:


* Shutting down one Sprout VM of a duplex pair causes calls to fail for 
10 seconds (https://github.com/Metaswitch/sprout/issues/1657)

* Setting the Chronos hostname in shared config means that Sprout can't 
start (https://github.com/Metaswitch/sprout/issues/1656)

* REGISTER expires 0 gets 401 Unauthorized rather than deregistering 
(https://github.com/Metaswitch/sprout/issues/1640)

* 'IMPI store operation SET failed' SAS log is unhelpful - it says 
"Reason: Error" (https://github.com/Metaswitch/sprout/issues/1601)

* nginx fails to start on Homestead nodes 
(https://github.com/Metaswitch/homestead/issues/416)

* Homestead doesn't attempt to reestablish its diameter connection to 
the HSS (https://github.com/Metaswitch/homestead/issues/407)

* HSS handover fails 
(https://github.com/Metaswitch/homestead/issues/404)

* cw-radius-auth introduces login lag 
(https://github.com/Metaswitch/clearwater-infrastructure/issues/407)

* Config files do not exist by default 
(https://github.com/Metaswitch/clearwater-etcd/issues/361)

* Force decommissioning an etcd cluster doesn't work like the docs 
suggest (https://github.com/Metaswitch/clearwater-etcd/issues/350)

* bono can't connect agentx master agent  
(https://github.com/Metaswitch/clearwater-docker/issues/43)

* Sprout sometimes fails to establish a connection with SNMPd 
(https://github.com/Metaswitch/clearwater-net-snmp/issues/10)

To upgrade to this release, follow the instructions at 
http://docs.projectclearwater.org/en/stable/Upgrading_a_Clearwater_deployment.html.
  If you are deploying an all-in-one node, the standard image 
(http://vm-images.cw-ngv.com/cw-aio.ova) has been updated for this release.

Ellie
___
Clearwater mailing list
Clearwater@lists.projectclearwater.org
http://lists.projectclearwater.org/mailman/listinfo/clearwater_lists.projectclearwater.org

Re: [Project Clearwater] How to reconfigure the cluster members?

[Eleanor Merry (projectclearwater.org)]

Hi Jace,

We don’t support changing the IPs of VMs in Project Clearwater. If you find 
yourself needing to change the IP of a VM, we recommend just deleting and 
recreating the VM (making sure to remove the VM from the clearwater deployment 
first, following the instructions in 
http://clearwater.readthedocs.io/en/latest/Clearwater_Elastic_Scaling.html).

In your case, clearwater-etcd only looks at the value of etcd_cluster on the 
initial boot – if it restarts it uses the information it’s saved to disk 
instead.

Ellie

From: Clearwater [mailto:clearwater-boun...@lists.projectclearwater.org] On 
Behalf Of jace.li...@itri.org.tw
Sent: 25 October 2016 09:38
To: clearwater@lists.projectclearwater.org
Subject: [Project Clearwater] How to reconfigure the cluster members?

Dear All,

I recently changed the IP of VMs (use manual installation if Clearwater)
I change the etcd_cluster in /etc/clearwater/local_config and reboot the node 
so that etc_process will get the new config.
But seems like it is still gets the old config as the clearwater-etcdctl 
cluster-health and clearwater-etcdctl member list all gets the old setting.

Am I miss some step or did something wrong?

Thank you.

Jace.


--
本信件可能包含工研院機密資訊，非指定之收件者，請勿使用或揭露本信件內容，並請銷毀此信件。 This email may contain 
confidential information. Please do not use or disclose it in any way and 
delete it if you are not the intended recipient.
___
Clearwater mailing list
Clearwater@lists.projectclearwater.org
http://lists.projectclearwater.org/mailman/listinfo/clearwater_lists.projectclearwater.org

[Project Clearwater] Sprint Porygon Release Note

[Eleanor Merry (projectclearwater.org)]

The release for Project Clearwater sprint "Porygon" has been cut. The code for 
this release is tagged as release-109 in github.
In this release:

  *   We've added audit logs to track what changes have been made to shared 
configuration. Now when a change is made to /etc/clearwater/shared_config and 
uploaded to etcd, we log to /var/log/syslog details of the change.
  *   We also spent some time working out ways to improve our automated testing 
of Clearwater.
This release also includes the following bug fixes:

* Clearwater Alarm Agent stops working if you upgrade clearwater-snmpd 
but not clearwater-snmp-alarm-agent 
(https://github.com/Metaswitch/clearwater-infrastructure/issues/384)

* Not all scripts in /usr/share/clearwater/infrastructure/scripts/ 
should be there 
(https://github.com/Metaswitch/clearwater-infrastructure/issues/362)

* Several Docker build errors 
(https://github.com/Metaswitch/clearwater-docker/issues/31, 
https://github.com/Metaswitch/clearwater-docker/issues/32, 
https://github.com/Metaswitch/clearwater-docker/issues/33, 
https://github.com/Metaswitch/clearwater-docker/issues/34)

To upgrade to this release, follow the instructions at 
http://docs.projectclearwater.org/en/stable/Upgrading_a_Clearwater_deployment.html.
 If you are deploying an all-in-one node, the standard image 
(http://vm-images.cw-ngv.com/cw-aio.ova) has been updated for this release.
___
Clearwater mailing list
Clearwater@lists.projectclearwater.org
http://lists.projectclearwater.org/mailman/listinfo/clearwater_lists.projectclearwater.org

Re: [Project Clearwater] Clearwater IMS in docker

[Eleanor Merry (projectclearwater.org)]

Hi Linda,

Our current source at https://github.com/Metaswitch/clearwater-docker will 
allow you to spin up a distributed Clearwater deployment. We're making some 
enhancements to clearwater-docker over the next couple of weeks as well to 
improve our docker support.

We don't have our code integrated with the docker registry at this moment.

Ellie

From: Clearwater [mailto:clearwater-boun...@lists.projectclearwater.org] On 
Behalf Of Zhu, Liu-Juan (Linda, CMS CoE China)
Sent: 30 September 2016 06:22
To: clearwater@lists.projectclearwater.org
Subject: [Project Clearwater] Clearwater IMS in docker

Dear all,

I would like to consult 2 questions:


1)  Is it possible for us to get the docker image (all-in-one or 
distributed) officially ?

2)  Does the current source code provided from 
https://github.com/Metaswitch/clearwater-docker is good enough for installing 
the Clearwater IMS in docker successfully?

Thank you very much for your support and look forward our response at your 
earliest.


Best Regards,
Linda

___
Clearwater mailing list
Clearwater@lists.projectclearwater.org
http://lists.projectclearwater.org/mailman/listinfo/clearwater_lists.projectclearwater.org

Re: [Project Clearwater] ibcf (bono) VM using up a lot of memory

[Eleanor Merry (projectclearwater.org)]

Hi,

I don’t think the output you’ve put below shows that there’s only 6% free 
memory. You want the value in the -/+ buffers/cache (which includes memory that 
Linux is using for disk caching, but can be taken by applications if they need 
more memory).

There’s possibly a memory leak somewhere though, as you had 3820636 available 
after the restart compared to 3630472, but it’s not as severe as you might 
think. To track down the memory leak, you can run Bono under a profiling tool 
(we use massif – see 
http://clearwater.readthedocs.io/en/stable/Debugging_Bono_Sprout_and_Homestead_with_GDB_and_Valgrind.html
 for more details).

Also, it’s worth noting that we wouldn’t typically expect Bono to be used in a 
production environment. We’d normally deploy Clearwater with Perimeta 
(http://www.metaswitch.com/perimeta-session-border-controller-sbc) rather than 
Bono.

Ellie


From: Clearwater [mailto:clearwater-boun...@lists.projectclearwater.org] On 
Behalf Of Gerard Ratcliffe
Sent: 04 October 2016 03:18
To: clearwater@lists.projectclearwater.org
Cc: Rene Anino; Bill Clemens; Richard Lawrence
Subject: [Project Clearwater] ibcf (bono) VM using up a lot of memory

Hi,

This is my first post on this forum I am also new to Clearwater IMS.  In fact I 
am babysitting our Clearwater IMS while our developer is on leave.

We have a manual install of bono/sprout/homestead/ibcf/dns being used as an
interface between Voip UE’s and a PSTN, running on 5 VM’s. They are connected
via a private virtual switch, and only the bono and ibcf nodes are connected to
an external network interface for external sip signalling. This setup is
working with no problems.  At present we have four of these IMSs setup.  Two of 
them in test sites (one for us one for the customer) and two in production 
sites.  Only one production site is carrying live traffic.

Last week I got an alert to say that one of the production ibcf VMs was running 
out of free memory.  Fortunately this was on the production IMS that was not 
carrying live traffic. My colleague had warned me that the VMs were slowly 
using up memory and that I should check them periodically to make sure that the 
free memory was not getting to low.

The ibcf free memory was down to less than 6%.  I restarted the VM and this 
sorted out the problem.  I also checked the other VMs in the both the 
production IMSs and they were all above 30%.  At this point I thought that the 
problem had been resolved.

However, today we received the same alaert for the ibcf VM.  Once again, the 
VM, was down to less than 6% free memory.

[bono]nec@ibcf:~$ free
 total   used   free sharedbuffers cached
Mem:   40414963809652 231844224 1515963247032
-/+ buffers/cache: 4110243630472
Swap:  20930523202092732
[bono]nec@ibcf:~$


I restarted the VM again and this dropped the free memory back to about 90%

[bono]nec@ibcf:~$ free
 total   used   free sharedbuffers cached
Mem:   4041496 3861163655380444  14528 150728
-/+ buffers/cache: 2208603820636
Swap:  2093052  02093052
[bono]nec@ibcf:~$

I have attached a file with some of the checking that I did before and after 
the restart.

Has anyone else experienced this problem and are there any checks that can be 
done to try and isolate the leaky process.

Regards,
Gerard




Gerard Ratcliffe
Technical Specialist
NEC New Zealand Limited
NEC House, Level 6, 40 Taranaki Street, PO Box: 1936, Wellington 6011, New 
Zealand
T: 043816235, M: 0275037273, F: +644380
gerard.ratcli...@nec.co.nz
nz.nec.com

[cid:image002.jpg@01D21F24.1960BE60]

Please consider the environment before printing this email

Attention:
The information contained in this message and or attachments is intended only 
for the person or entity to which it is addressed and may contain confidential 
and/or privileged material.  Any review, retransmission, dissemination, copying 
or other use of, or taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is prohibited. If you 
received this in error, please contact the sender and delete the material from 
any system and destroy any copies. NEC has no liability for any act or omission 
in reliance on the email or any attachment.  Before opening this email or any 
attachment(s), please check them for viruses. NEC is not responsible for any 
viruses in this email or any attachment(s); any changes made to this  email or 
any attachment(s) after they are sent; or any effects this email or any 
attachment(s) have on your network or computer system.
___
Clearwater mailing list
Clearwater@lists.projectclearwater.org
http://lists.projectclearwater.org/mailman/listinfo/clearwater_lists.projectclearwater.org

Re: [Project Clearwater] Integration of Open HSS_All-in-one

[Eleanor Merry (projectclearwater.org)]

Hi Surender,

To use an external HSS you'll need to set the following settings in 
/etc/clearwater/shared_config

hss_hostname=
hss_realm=
hss_port=

Both hss_hostname and hss_realm are optional. If a realm is configured, 
homestead will try NAPTR/SRV resolution on the realm to find and connect to (2 
by default) diameter peers in the realm.
If a hostname is also configured, this will be used in the Destination-Host 
field on the diameter messages, so that the messages will be routed to that 
host.
If just a hostname is configured, homestead will just attempt to create and 
maintain a single connection to that host.

You can find more information on this at 
http://clearwater.readthedocs.io/en/latest/External_HSS_Integration.html, and 
you may also find 
http://clearwater.readthedocs.io/en/latest/OpenIMSCore_HSS_Integration.html 
useful.

For your other queries:

1. You can configure subscribers to use an application server that provides 
these functions by setting their IFCs in the HSS. You can use the MMTel 
application server built into Clearwater for this - see the details at 
http://clearwater.readthedocs.io/en/latest/OpenIMSCore_HSS_Integration.html#adding-the-mmtel-application-server
 for more information.

2. The key restrictions of the all-in-one solutions are:
* hard-coded realm - the all-in-one image uses a hard-coded realm of 
example.com so your SIP URI might be sip:6505551...@example.com - by default, 
SIP uses this realm for routing but example.com won't resolve to your host so 
you need to configure an outbound proxy on all your SIP clients
* performance - since all software runs on a single virtual machine, 
performance is significantly lower than even the smallest scale deployment
* scalability - there is no option to scale up and add more virtual machines to 
boost capacity - for this, you must create a normal deployment
* fault-tolerance - since everything runs on a single virtual machine, if that 
virtual machine fails, the service as a whole fails.

3. I didn't understand this question. What's the 'other DNS'?

Ellie

From: Clearwater [mailto:clearwater-boun...@lists.projectclearwater.org] On 
Behalf Of Surender Singh
Sent: 03 October 2016 07:11
To: clearwater@lists.projectclearwater.org
Subject: [Project Clearwater] Integration of Open HSS_All-in-one

Hello,

I deployed the All in one solution in VMware machine .My IN-OUT calls are 
working to other network using BGCF/IBCF feature .

Now i want Integrate the Open HSS with Homestead node .Can you suggest me where 
I need to make the entry in aio solution .

I having some below query..


1)  Can we provision the basic supplementary  service (CF Service, Three 
Party, Call-Transfer , Call park  etc .) in open HSS.

2)  Any other drawbacks of the all-in-one solution apart from Performance 
/Production.

3)  Can i integrate the other DNS for DNS query .

Regards
Surender Singh








::DISCLAIMER::

The contents of this e-mail and any attachment(s) are confidential and intended 
for the named recipient(s) only.
E-mail transmission is not guaranteed to be secure or error-free as information 
could be intercepted, corrupted,
lost, destroyed, arrive late or incomplete, or may contain viruses in 
transmission. The e mail and its contents
(with or without referred errors) shall therefore not attach any liability on 
the originator or HCL or its affiliates.
Views or opinions, if any, presented in this email are solely those of the 
author and may not necessarily reflect the
views or opinions of HCL or its affiliates. Any form of reproduction, 
dissemination, copying, disclosure, modification,
distribution and / or publication of this message without the prior written 
consent of authorized representative of
HCL is strictly prohibited. If you have received this email in error please 
delete it and notify the sender immediately.
Before opening any email and/or attachments, please check them for viruses and 
other defects.

___
Clearwater mailing list
Clearwater@lists.projectclearwater.org
http://lists.projectclearwater.org/mailman/listinfo/clearwater_lists.projectclearwater.org

Re: [Project Clearwater] DNS Problem while registration of sip client

[Eleanor Merry (projectclearwater.org)]

Hi Manikantha,

Can you run ‘nslookup ralf.cw.pramati.com’ on your 
nodes, and see what the ‘Server’ value is?

The clearwater processes look in /etc/resolv.conf for DNS configuration. If 
your nodes don’t use localhost (which you can tell as the Server value above 
won’t be 127.0.0.1) you’ll need to update the dnsmasq configuration to point to 
this server rather than localhost. You can find details of how to do this at: 
https://github.com/Metaswitch/clearwater-docs/wiki/Clearwater-DNS-Usage#client-configuration.

Ellie

From: Clearwater [mailto:clearwater-boun...@lists.projectclearwater.org] On 
Behalf Of Manikantha Srinivas Tadi
Sent: 28 September 2016 09:12
To: clearwater@lists.projectclearwater.org
Subject: [Project Clearwater] DNS Problem while registration of sip client

Hi,

We are trying clearwater IMS to setup a call between sip client.

How ever, while registering the sip client using CSIPSimple utility , We are 
getting the following error logs in /var/log/bono and /var/log/sprout

Bono Logs :

8-09-2016 08:04:45.765 UTC Error dnscachedresolver.cpp:607: Failed to retrieve 
record for ralf.cw.pramati.com: Could not contact 
DNS servers
28-09-2016 08:04:45.765 UTC Error httpclient.cpp:623: cURL failure with cURL 
error code 6 (see man 3 libcurl-errors) and HTTP error code 404
28-09-2016 08:04:50.652 UTC Status alarm.cpp:62: sprout issued 1005.4 alarm
28-09-2016 08:04:52.100 UTC Warning (Net-SNMP): Warning: Failed to connect to 
the agentx master agent ([NIL]):

Sprout Logs :

28-09-2016 08:08:26.816 UTC Status alarm.cpp:62: sprout issued 1001.3 alarm
28-09-2016 08:08:37.986 UTC Error dnscachedresolver.cpp:607: Failed to retrieve 
record for hs.cw.pramati.com: Could not contact DNS 
servers
28-09-2016 08:08:37.986 UTC Error httpclient.cpp:623: cURL failure with cURL 
error code 6 (see man 3 libcurl-errors) and HTTP error code 404
28-09-2016 08:08:56.816 UTC Status alarm.cpp:62: sprout issued 1001.3 alarm
28-09-2016 08:09:05.374 UTC Status load_monitor.cpp:285: Maximum incoming 
request rate/second unchanged - only handled 20 requests in last 191050ms, 
minimum threshold for a change is 23881.25


Where as i am able to manually resolve the DNS Records for 
ralf.cw.pramati.com and 
hs.cw.pramati.com on the bono and sprout nodes 
respectively.

Could someone please help if we are missing something in this regards ?

Thanks
Manikantha.

___
Clearwater mailing list
Clearwater@lists.projectclearwater.org
http://lists.projectclearwater.org/mailman/listinfo/clearwater_lists.projectclearwater.org

[Project Clearwater] Diglett release note

[Eleanor Merry (projectclearwater.org)]

The release for Project Clearwater sprint "Diglett" has been cut. The code for 
this release is tagged as release-97 in github.

This release includes the following bug fixes:

* Sprout always tries to write to the remote memcached store, even when 
there isn't one (https://github.com/Metaswitch/sprout/issues/1428)

* stack_data's name array is not necessarily big enough 
(https://github.com/Metaswitch/sprout/issues/1427)

* Sprout (and memento) crash on stop 
(https://github.com/Metaswitch/sprout/issues/1408)

* Sprout crashes under load with unresponsive TAS configured 
(https://github.com/Metaswitch/sprout/issues/1407)

* RTRs don't always result in a challenge on reregistrations 
(https://github.com/Metaswitch/sprout/issues/1406)

* calls fail with mixed versions of sprout 
(https://github.com/Metaswitch/sprout/issues/1394)

* Possible deadlock - worker thread and quiescing thread can take 
connection manager lock and transport lock in different orders 
(https://github.com/Metaswitch/sprout/issues/1351)

* Possible deadlock - worker thread and transport thread can take 
_trying_timer_lock and other locks in different orders 
(https://github.com/Metaswitch/sprout/issues/1325)

* Retransmitted SIP requests do not appear in SAS.  
(https://github.com/Metaswitch/sprout/issues/353)

* Chrome can't cope with URI-encoded usernames in TURN URIs 
(https://github.com/Metaswitch/sprout/issues/93)

* Pass server-name and sprout-http-name through from sprout 
(https://github.com/Metaswitch/homestead/issues/66)

* Chronos crashes on stop 
(https://github.com/Metaswitch/chronos/issues/254)

* Cpp processes no longer do anything on a sighup/sigusr1 
(https://github.com/Metaswitch/cpp-common/issues/504)

* Cassandra can get into an unmonitored state so clustering fails 
(https://github.com/Metaswitch/clearwater-cassandra/issues/96)

* cassandra won't cluster because etcd plugin fails trying to delete a 
non-existent file (https://github.com/Metaswitch/clearwater-cassandra/issues/87)

* Changing /etc/hosts makes Docker sad 
(https://github.com/Metaswitch/clearwater-infrastructure/issues/323)

* Broken links 
(https://github.com/Metaswitch/clearwater-readthedocs/issues/169)

* Astaire crashes on stop 
(https://github.com/Metaswitch/astaire/issues/62)

* Communications loss between geo sites causes etcd failure 
(https://github.com/Metaswitch/clearwater-etcd/issues/200)


For upgrading to this release, follow the instructions at 
http://clearwater.readthedocs.org/en/latest/Upgrading_a_Clearwater_deployment/index.html.
 If you are deploying an all-in-one node, the standard image 
(http://vm-images.cw-ngv.com/cw-aio.ova) has been updated for this release.



Thanks,

Ellie

___
Clearwater mailing list
Clearwater@lists.projectclearwater.org
http://lists.projectclearwater.org/mailman/listinfo/clearwater_lists.projectclearwater.org

Re: [Project Clearwater] SIP Event Notification - resource id

[Eleanor Merry (projectclearwater.org)]

Hi Alan,

Thanks for raising this! I agree with your reading of the RFC, and I've raised 
an issue to track fixing this in Project Clearwater at 
https://github.com/Metaswitch/sprout/issues/1434.

Ellie

From: Clearwater [mailto:clearwater-boun...@lists.projectclearwater.org] On 
Behalf Of Kwon, Alan
Sent: 04 May 2016 17:35
To: clearwater@lists.projectclearwater.org
Subject: [Project Clearwater] SIP Event Notification - resource id

Hi,

I'm sending a SUBSCRIBE to sprout for a registration event and seeing that it's 
using the To-header as the resource Id instead of the Request URI.

According to RFC 3265, Section 3.1.2, the Request URI is used to identify the 
resource for which event notification is desired.
The following is a log from Sprout node:

SUBSCRIBE sip:+15557771...@x.com SIP/2.0
Via: SIP/2.0/UDP 
10.12.92.xxx:8577;branch=z9hG4bKd71ac340-dba6-4a0e-b343-a37bf91859c8
Max-Forwards: 69
P-Asserted-Identity: 
From: ;tag=2d41994f-e15b-4c8e-885e-70da99e1cf45
To: 
Call-ID: 4f5b126c-ec89-4d2a-bfa5-7f618d5a5303
CSeq: 1 SUBSCRIBE
Event: reg
Expires: 0
Accept: application/reginfo+xml
Contact: 
Content-Length: 0
Route: 

04-05-2016 15:19:31.763 UTC Debug thread_dispatcher.cpp:236: Queuing cloned 
received message 0x7f093c007128 for worker threads
04-05-2016 15:19:31.763 UTC Debug thread_dispatcher.cpp:149: Worker thread 
dequeue message 0x7f093c007128
04-05-2016 15:19:31.763 UTC Debug pjsip: sip_endpoint.c Distributing rdata to 
modules: Request msg SUBSCRIBE/cseq=1 (rdata0x7f093c007128)
04-05-2016 15:19:31.763 UTC Debug acr.cpp:48: Created ACR (0x7f09501e79d0)
04-05-2016 15:19:31.763 UTC Debug subscription.cpp:396: Process SUBSCRIBE for 
public ID sip:sprout.x.com
04-05-2016 15:19:31.763 UTC Debug subscription.cpp:407: Report SAS start marker 
- trail (d89ab)
04-05-2016 15:19:31.763 UTC Debug hssconnection.cpp:638: Making Homestead 
request for /impu/sip%3Asprout.x.com/reg-data

Instead of using the Request URI, it's using the To-header as the public ID for 
the subscription. Seems to be a bug. Would appreciate your comment.

Thank you,





[cid:image002.png@01D1A6F5.06B04A30]
[cid:image003.png@01D1A6F5.06B04A30][cid:image004.png@01D1A6F5.06B04A30][cid:image005.png@01D1A6F5.06B04A30][cid:image006.png@01D1A6F5.06B04A30]

Alan Kwon
Senior Software Engineer



T: +1 972-753-1865 (Texas)   F: +1 239-425-6845

Confidentiality Notice: The information in this e-mail and in any attachment 
may contain information which is legally privileged. It is intended only for 
the attention and use of the named recipient. If you are not the intended 
recipient, you are not authorized to retain, disclose, copy or distribute the 
message and/or any of its attachments. If you received this e-mail in error, 
please notify me and delete this message. Thank you.





___
Clearwater mailing list
Clearwater@lists.projectclearwater.org
http://lists.projectclearwater.org/mailman/listinfo/clearwater_lists.projectclearwater.org