Josh I didn't think that making any major changes to xCAT2.pm would be a good idea. In fact the solution provided at https://cwiki.apache.org/VCL/adding-support-for-partimage-and-partimage-ng-to-xcat-2x-unofficial.html is good and tested. I assume that in newer xCAT nodestat behaves differently.
To get it working again I've created new /opt/xcat/bin/vcl_nodestat script,
which communicates with socket created by listener.py.
#!/usr/bin/perl -w
use strict;
my ($remote, $answer, $peerAddress, $peerPort, $sendCommand);
$peerAddress = $ARGV[0];
$peerPort = 3001;
$sendCommand = $ARGV[1];
use IO::Socket;
$remote=IO::Socket::INET->new(Proto=>'tcp',PeerAddr => $peerAddress,PeerPort =>
$peerPort,Type =>SOCK_STREAM) or die "$peerAddress: noping\n";
print $remote "$sendCommand\n";
while ($answer=<$remote>){
print "$peerAddress:$answer\n";
}
close $remote;
Then I made some minor changes to xCAT2.pm, replacing '$XCAT_ROOT/bin/nodestat'
with '$XCAT_ROOT/bin/vcl_nodestat'.
This seems to fixed the problem.
Thanks.
On Feb 28, 2012, at 15:42 , Josh Thompson wrote:
>
> Hash: SHA1
>
> Dmitri,
>
> At some point, this did work. I'm not sure when it broke, but I never dug
> back in to it to make it work again. We've not noticed it causing failures;
> so, I just let it go. I can have a look at it again when I get a chance.
>
> Can you send some more detailed information about the conditions when you see
> the problem?
>
> One other thing to look at - what does nodestat report after "install
> <imagename>"? If it is reporting that ssh is running, you may have a
> postscript set to configure ssh that is getting run for the stateless image.
> We have things set up so that ssh does not get set up for the stateless image.
>
> Josh
>
> On Tuesday, February 28, 2012 12:58:46 PM Dmitri Chebotarov wrote:
>> Hi
>>
>> There seems to be a disconnect between xCAT2.pm and
>> /install/postscripts/partimageng script. The partimageng script starts
>> listener.py in background which suppose to report to nodestat in xCAT2.pm
>> module. xCAT2.pm is using 'nodestat nodename stat' to check on status, but
>> nodestat command doesn't use listener.py running on the node.
>>
>> Is there a workaround? May be an updated version of xCAT2.pm which checks
>> /var/log/messages on round2? Looks like partimageng is using logger to
>> report back the status to xCat master...
>>
>> B/c of this, reservation or reload is inconsistent. nodestat would report
>> "install <imagename>" for about 7-9 sec and xCAT2.pm doesn't always get it.
>>
>> Thank you for your help.
>>
>> On Feb 22, 2012, at 14:32 , Dmitri Chebotarov wrote:
>>> John,
>>>
>>> I got the link and I think I'm making some good progress.
>>> Also found this tread on the maillist
>>> (http://markmail.org/message/lotx3l34co6yv5nf?q=partimageng%2Epm#query:
>>> partimageng.pm+page:1+mid:lthvtrde7radude3+state:results) where you
>>> helped Prem, which is very helpful.
>>>
>>> Thank you
>>>
>>> On Feb 22, 2012, at 14:12 , Josh Thompson wrote:
>>>> Hash: SHA1
>>>>
>>>> Dmitri,
>>>>
>>>> Did you follow the guide for adding partimage support to xCAT?
>>>>
>>>> https://cwiki.apache.org/confluence/display/VCL/Adding+support+for+par
>>>> timage+and+partimage- ng+to+xCAT+2.x+%28unofficial%29
>>>>
>>>> You said you installed the partimage rpm. partimage is not something
>>>> that needs to be installed on the management node. It is part of the
>>>> stateless images mentioned on the above page.
>>>>
>>>> Josh
>>>>
>>>> On Tuesday 21 February 2012 4:47:06 PM Dmitri Chebotarov wrote:
>>>>> Hi All.
>>>>>
>>>>> I'm trying to capture a base image using xCAT module - the server is
>>>>> bare-metal CentOS5.7. The process starts OK, able to communicate
>>>>> with the node and then fails when trying to create .tmpl file.
>>>>>
>>>>> xCAT is installed and configured on the management node.
>>>>> partimage rpm is installed as well.
>>>>>
>>>>> Do you know what could be the issue?
>>>>>
>>>>> Thank you for your help.
>>>>>
>>>>> ...
>>>>> 2012-02-21
>>>>> 16:22:30|12755|275:261|image|xCAT2.pm:capture(827)|nodetype
>>>>> modified, node bc1-7, image name centos5image-xcatcentos5v474-v0
>>>>> 2012-02-21
>>>>> 16:22:30|12755|275:261|image|xCAT2.pm:_create_template(1065)|attemp
>>>>> ting to create tmpl file for image: centos5image-xcatcentos5v474-v0
>>>>> 2012-02-21
>>>>> 16:22:30|12755|275:261|image|xCAT2.pm:_get_image_template_path(2084
>>>>> )|manage ment node identifier argument was not specified
>>>>> 2012-02-21 16:22:30|12755|275:261|image|
>>>>
>>>> xCAT2.pm:_get_image_template_path(2115)|attempting to determine
>>>> template path>>
>>>> for image:
>>>>> |12755|275:261|image| image name: centos5image-xcatcentos5v474-v0
>>>>> |12755|275:261|image| OS install type: partimage
>>>>> |12755|275:261|image| OS source path: image
>>>>> |12755|275:261|image| xCAT 2.x OS source path: image
>>>>>
>>>>> 2012-02-21
>>>>> 16:22:30|12755|275:261|image|xCAT2.pm:_get_image_template_path(2123)
>>>>> |return ing: /opt/xcat/share/xcat/install/image 2012-02-21
>>>>> 16:22:30|12755|275:261|image|xCAT2.pm:_get_image_template_path(2084)
>>>>> |manage ment node identifier argument was not specified
>>>>> 2012-02-21 16:22:30|12755|275:261|image|
>>>>
>>>> xCAT2.pm:_get_image_template_path(2115)|attempting to determine
>>>> template path>>
>>>> for image:
>>>>> |12755|275:261|image| image name: centos5image-xcatcentos5v474-v0
>>>>> |12755|275:261|image| OS install type: partimage
>>>>> |12755|275:261|image| OS source path: image
>>>>> |12755|275:261|image| xCAT 2.x OS source path: image
>>>>>
>>>>> 2012-02-21
>>>>> 16:22:30|12755|275:261|image|xCAT2.pm:_get_image_template_path(2123)
>>>>> |return ing: /opt/xcat/share/xcat/install/image 2012-02-21
>>>>> 16:22:30|12755|275:261|image|DataStructure.pm:get_computer_private_i
>>>>> p_addre ss(1557)|returning private IP address previously retrieved:
>>>>> 10.3.1.27 2012-02-21
>>>>> 16:22:30|12755|275:261|image|utils.pm:is_inblockrequest(6164)|zero
>>>>> rows
>>>>> were returned from database select 2012-02-21
>>>>> 16:22:30|12755|275:261|image|DataStructure.pm:get_image_affiliation_
>>>>> name(20 35)|image owner id: 1 2012-02-21
>>>>> 16:22:30|12755|275:261|image|utils.pm:getnewdbh(2710)|database
>>>>> requested (information_schema) does not match handle stored in
>>>>> $ENV{dbh} (vcl:10.3.1.214) 2012-02-21
>>>>> 16:22:30|12755|275:261|image|utils.pm:getnewdbh(2761)|database
>>>>> handle
>>>>> stored in $ENV{dbh} 2012-02-21
>>>>> 16:22:30|12755|275:261|image|DataStructure.pm:retrieve_user_data(135
>>>>> 2)|atte mpting to retrieve and store data for user: user.id = '1'
>>>>> 2012-02-21
>>>>> 16:22:30|12755|275:261|image|utils.pm:getnewdbh(2710)|database
>>>>> requested (vcl) does not match handle stored in $ENV{dbh}
>>>>> (information_schema:10.3.1.214) 2012-02-21
>>>>> 16:22:30|12755|275:261|image|utils.pm:getnewdbh(2761)|database
>>>>> handle
>>>>> stored in $ENV{dbh} 2012-02-21
>>>>> 16:22:30|12755|275:261|image|DataStructure.pm:retrieve_user_data(141
>>>>> 5)|data has been retrieved for user: admin (id: 1) 2012-02-21
>>>>> 16:22:30|12755|275:261|image|utils.pm:mail(1268)|SUCCESS -- Sending
>>>>> mail To: dcheb...@gmu.edu, PROBLEM --
>>>>> 275:261|image|xCAT2.pm|bc1-7|centos5image-xcatcentos5v474-v0|admin
>>>>>
>>>>> |12755|275:261|image| ---- CRITICAL ----
>>>>> |12755|275:261|image| 2012-02-21
>>>>> |16:22:30|12755|275:261|image|xCAT2.pm:_get_base_template_filename(
>>>>> |2170)|f ailed to find suitable base image template file in
>>>>> |/opt/xcat/share/xcat/install/image 12755|275:261|image| ( 0)
>>>>> |xCAT2.pm,
>>>>> |_get_base_template_filename (line: 2170) 12755|275:261|image| (-1)
>>>>> |xCAT2.pm, _create_template (line: 1075) 12755|275:261|image| (-2)
>>>>> |xCAT2.pm, capture (line: 836)
>>>>> |12755|275:261|image| (-3) image.pm, process (line: 162)
>>>>> |12755|275:261|image| (-4) vcld, make_new_child (line: 568)
>>>>> |12755|275:261|image| (-5) vcld, main (line: 346)
>>>>> |12755|275:261|image| ---- WARNING ----
>>>>> |12755|275:261|image| 2012-02-21
>>>>> |16:22:30|12755|275:261|image|xCAT2.pm:_create_template(1077)|base
>>>>> |template filename could not be determined 12755|275:261|image| (
>>>>> |0)
>>>>> |xCAT2.pm, _create_template (line: 1077) 12755|275:261|image| (-1)
>>>>> |xCAT2.pm, capture (line: 836)
>>>>> |12755|275:261|image| (-2) image.pm, process (line: 162)
>>>>> |12755|275:261|image| (-3) vcld, make_new_child (line: 568)
>>>>> |12755|275:261|image| (-4) vcld, main (line: 346)
>>>>> |12755|275:261|image| ---- WARNING ----
>>>>> |12755|275:261|image| 2012-02-21
>>>>> |16:22:30|12755|275:261|image|xCAT2.pm:capture(840)|failed to
>>>>> |create .tmpl file for centos5image-xcatcentos5v474-v0
>>>>> |12755|275:261|image| ( 0) xCAT2.pm, capture (line: 840)
>>>>> |12755|275:261|image| (-1) image.pm, process (line: 162)
>>>>> |12755|275:261|image| (-2) vcld, make_new_child (line: 568)
>>>>> |12755|275:261|image| (-3) vcld, main (line: 346)
>>>>> |12755|275:261|image| ---- WARNING ----
>>>>> |12755|275:261|image| 2012-02-21
>>>>> |16:22:30|12755|275:261|image|image.pm:process(166)|centos5image-xc
>>>>> |atcento s5v474-v0 image failed to be captured by provisioning
>>>>> |module
>>>>> |12755|275:261|image| ( 0) image.pm, process (line: 166)
>>>>> |12755|275:261|image| (-1) vcld, make_new_child (line: 568)
>>>>> |12755|275:261|image| (-2) vcld, main (line: 346)
>>>>>
>>>>> On Feb 10, 2012, at 11:44 , Aaron Peeler wrote:
>>>>>> Yes, we're using the latest xCAT version.
>>>>>>
>>>>>> -A
>>>>>>
>>>>>> On Fri, Feb 10, 2012 at 11:42 AM, Kumar, Amit H. <ahku...@odu.edu>
> wrote:
>>>>>>> Hi Dmitri,
>>>>>>>
>>>>>>> We use xCAT 2.5.2; I am not a pro to say if xCAT 2.6.x will
>>>>>>> work.
>>>>>>> May be Aaron and his group could say more.
>>>>>>>
>>>>>>> Thank you,
>>>>>>> Amit
>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: dcheb...@gmu.edu [mailto:dcheb...@gmu.edu]
>>>>>>>> Sent: Friday, February 10, 2012 11:37 AM
>>>>>>>> To: vcl-user@incubator.apache.org
>>>>>>>> Subject: Re: VCL and xCat
>>>>>>>>
>>>>>>>>
>>>>>>>> Thank you.
>>>>>>>>
>>>>>>>> What version of xCat do you use? Since VCL supports xCat 2.x,
>>>>>>>> is it
>>>>>>>> safe to use latest xCat 2.6.10? Thanks.
>>>>>>>>
>>>>>>>> On Feb 10, 2012, at 11:27 , Kumar, Amit H. wrote:
>>>>>>>>> Hi Dmitri,
>>>>>>>>>
>>>>>>>>> SUMAVI link that you have is a real good one, I blindly
>>>>>>>>> followed
>>>>>>>>> it
>>>>>>>>
>>>>>>>> and then slowly things started to make more sense once it
>>>>>>>> worked.
>>>>>>>>
>>>>>>>>> Also xCAT forum was very helpful.
>>>>>>>>>
>>>>>>>>> Thank you,
>>>>>>>>> Amit
>>>>>>>>>
>>>>>>>>>> -----Original Message-----
>>>>>>>>>> From: dcheb...@gmu.edu [mailto:dcheb...@gmu.edu]
>>>>>>>>>> Sent: Friday, February 10, 2012 10:15 AM
>>>>>>>>>> To: vcl-user@incubator.apache.org; aaron_pee...@ncsu.edu
>>>>>>>>>> Subject: Re: VCL and xCat
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thank you, I hope we can make it work in our environment
>>>>>>>>>> as
>>>>>>>>>> well...
>>>>>>>>>>
>>>>>>>>>> I keep searching and reading xCat docs - here is good one
>>>>>>>>>> ->
>>>>>>>>>> http://sumavi.com/books/xcat-administrators-guide.
>>>>>>>>>> BTW, if you have any docs outside of xcat.org site, please
>>>>>>>>>> share...
>>>>>>>>>> At this point I'm looking for xCat overview docs, i.e. to
>>>>>>>>>> understand
>>>>>>>>>> the process.
>>>>>>>>>>
>>>>>>>>>> Looks like xCat makes extensive use of DHCP server to
>>>>>>>>>> manage
>>>>>>>>>> nodes.
>>>>>>>>>> Are you using different DHCPd servers for VCL VMs and xCat
>>>>>>>>>> nodes, or
>>>>>>>>>> just keep everything on one DHCP server and being careful
>>>>>>>>>> with
>>>>>>>>>> which
>>>>>>>>>> MAC gets what?
>>>>>>>>>>
>>>>>>>>>> Thanks again for your help...
>>>>>>>>>>
>>>>>>>>>> On Feb 10, 2012, at 9:30 , Aaron Peeler wrote:
>>>>>>>>>>> Dmitri,
>>>>>>>>>>>
>>>>>>>>>>> Just to add to Amit, comments.
>>>>>>>>>>>
>>>>>>>>>>> We use xCAT extensively here at ncsu. From provisioning
>>>>>>>>>>> end-user
>>>>>>>>>>> desktops and our hypervisors. xCAT very helpful when
>>>>>>>>>>> needed to
>>>>>>>>>>
>>>>>>>>>> install
>>>>>>>>>>
>>>>>>>>>>> a chassis full of blades with esxi.
>>>>>>>>>>>
>>>>>>>>>>> As your installing xCAT You'll want to review
>>>>>>>>>>> https://cwiki.apache.org/VCL/adding-support-for-partimag
>>>>>>>>>>> e-and-
>>>>>>>>>>
>>>>>>>>>> partimag
>>>>>>>>>>
>>>>>>>>>>> e-ng-to-xcat-2x-unofficial.html We added in VCL support
>>>>>>>>>>> for
>>>>>>>>>>> partimage
>>>>>>>>>>> - which is used to capture the disk images of end-user
>>>>>>>>>>> environments/desktops for windows and linux.
>>>>>>>>>>>
>>>>>>>>>>> When your adding nodes to vcl through the manage
>>>>>>>>>>> computers
>>>>>>>>>>> tool,
>>>>>>>>>>> make sure to choose "xCAT 2.x" . If your doing windows
>>>>>>>>>>> bare-metal,
>>>>>>>>>>> you'll also need to include sysprep and related mass
>>>>>>>>>>> storage
>>>>>>>>
>>>>>>>> drivers
>>>>>>>>
>>>>>>>>>>> so it can be reloaded successfully.
>>>>>>>>
>>>>>>>>>>> For sysprep review step 8 on this page:
>>>>>>>> https://cwiki.apache.org/confluence/display/VCL/VCL+2.2.1+Mana
>>>>>>>> gement
>>>>>>>> +
>>>>>>>>
>>>>>>>>>> N
>>>>>>>>>>
>>>>>>>>>>> ode+Installation
>>>>>>>>>>>
>>>>>>>>>>> Hope this info helps,
>>>>>>>>>>> Aaron
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Feb 9, 2012 at 4:03 PM, Dmitri Chebotarov
>>>>>>>>
>>>>>>>> <dcheb...@gmu.edu>
>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>>> Amit,
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks a lot.
>>>>>>>>>>>>
>>>>>>>>>>>> On Feb 9, 2012, at 15:57 , Kumar, Amit H. wrote:
>>>>>>>>>>>>> Hi Dmitri,
>>>>>>>>>>>>>
>>>>>>>>>>>>> You general idea is correct to my best
>>>>>>>>>>>>> understanding.
>>>>>>>>>>>>>
>>>>>>>>>>>>> "Install Path" in the Edit management Node will be
>>>>>>>>>>>>> the
>>>>>>>>>>>>> path for
>>>>>>>>>>
>>>>>>>>>> xCAT(bare metal) image repository. In addition when you
>>>>>>>>>> configure
>>>>>>>>
>>>>>>>> the
>>>>>>>>
>>>>>>>>>> "Site" table in xCAT you will set the "installdir" to
>>>>>>>>>> point to
>>>>>>>>>> the
>>>>>>>>>> location where you will store your images.
>>>>>>>>>>
>>>>>>>>>>>>> We have IBM E Blade Chassis with HS22 blade servers.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Best,
>>>>>>>>>>>>> Amit
>>>>>>>>>>>>>
>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>> From: dcheb...@gmu.edu [mailto:dcheb...@gmu.edu]
>>>>>>>>>>>>>> Sent: Thursday, February 09, 2012 3:44 PM
>>>>>>>>>>>>>> To: vcl-user@incubator.apache.org
>>>>>>>>>>>>>> Subject: Re: VCL and xCat
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Amit,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'm trying to get general idea about xCat/VCL :
>>>>>>>>>>>>>> so,
>>>>>>>>>>>>>> basically
>>>>>>>>>>>>>> xCat needs to be installed and configured on VCL
>>>>>>>>>>>>>> Mgm
>>>>>>>>>>>>>> node(s)
>>>>>>>>>>>>>> before it can be used with VCL, and I should be
>>>>>>>>>>>>>> able to
>>>>>>>>>>>>>> successfully provision xCat nodes/clients with
>>>>>>>>>>>>>> different
>>>>>>>>>>>>>> images.
>>>>>>>>>>>>>> Once xCat is working then I can add bare-metal
>>>>>>>>>>>>>> computers
>>>>>>>>>>>>>> - blade
>>>>>>>>>>>>>> with xCat 2.x engine. Does it sound right?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> How do you tell VCL where to get image files for
>>>>>>>>>>>>>> xCat
>>>>>>>>>>>>>> nodes - is
>>>>>>>>>>
>>>>>>>>>> it
>>>>>>>>>>
>>>>>>>>>>>>>> 'Install Path' setting under Management Nodes ->
>>>>>>>>>>>>>> Edit
>>>>>>>>>>>>>> management
>>>>>>>>>>>>>> node information?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> What kind of hardware do you have for xCat nodes?
>>>>>>>>>>>>>> Are
>>>>>>>>>>>>>> those IBM
>>>>>>>>>>>>>> blade servers or/and PCs?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Feb 9, 2012, at 15:16 , Kumar, Amit H. wrote:
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> We do not need an additional/dedicated server to
>>>>>>>>>>>>>>> do
>>>>>>>>>>>>>>> both xCAT
>>>>>>>>>>>>>>> and VCL
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> together, your management node can serve as your
>>>>>>>>>>>>>> xCAT
>>>>>>>>>>>>>> server.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I would recommend getting xCAT to work by itself
>>>>>>>>>>>>>>> before using
>>>>>>>>>>>>>>> VCL to
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> manage your images and guest OS.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> This has worked well for us so far.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> :Amit
>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>>> From: dcheb...@gmu.edu
>>>>>>>>>>>>>>>> [mailto:dcheb...@gmu.edu]
>>>>>>>>>>>>>>>> Sent: Thursday, February 09, 2012 3:03 PM
>>>>>>>>>>>>>>>> To: vcl-user@incubator.apache.org
>>>>>>>>>>>>>>>> Subject: VCL and xCat
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi All
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Anyone is using xCat with VCL?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I have generic question about xCat and VCL -
>>>>>>>>>>>>>>>> where
>>>>>>>>>>>>>>>> do I start?
>>>>>>>>>>>>>>>> Do I need working xCat environment before I
>>>>>>>>>>>>>>>> can add
>>>>>>>>>>>>>>>> it to VCL?
>>>>>>>>>>>>>>>> Does xCat need dedicated server? Or does it
>>>>>>>>>>>>>>>> run on
>>>>>>>>>>>>>>>> the VCL
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> management
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> node?
>>>>>>>>>>>>>>>> Are there any special hardware requirements
>>>>>>>>>>>>>>>> for xCat
>>>>>>>>
>>>>>>>> "clients"?
>>>>>>>>
>>>>>>>>>>>>>>>> Thank you.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thank you.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -----------------
>>>>>>>>>>>>>> Dmitri Chebotarov
>>>>>>>>>>>>>> dcheb...@gmu.edu
>>>>>>>>>>>>>> 703-993-6175
>>>>>>>>>>>>>> -----------------
>>>>>>>>>>>>
>>>>>>>>>>>> Thank you.
>>>>>>>>>>>>
>>>>>>>>>>>> -----------------
>>>>>>>>>>>> Dmitri Chebotarov
>>>>>>>>>>>> dcheb...@gmu.edu
>>>>>>>>>>>> 703-993-6175
>>>>>>>>>>>> -----------------
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> -----BEGIN PGP SIGNATURE-----
>>>>>>>>>>>>
>>>>>>>>>>>> iQEcBAEBAgAGBQJPNDStAAoJEMRKpymufl8aAxkH/iCGHjBRUMR85l
>>>>>>>>>>>> 4+D3cl
>>>>>>>>>>>> zzem
>>>>>>>>>>>> eb4+RtsjvqfnJO7IjjF3Ajs73labFWvoEgpJrbFous2v6ou/gNsUgB
>>>>>>>>>>>> 82hWic
>>>>>>>>>>>> qNTm
>>>>>>>>>>>> 5+ZKkdncM2qWl2QI1ED6kvZt0OHmG/cUbRk0XRlPSjf99dIYgjqPZB
>>>>>>>>>>>> L4xFNj
>>>>>>>>>>>> e62X
>>>>>>>>>>>> xzNNPCEC7S5lixouyUcFPh7KhVR+11zvZsliV19CYH6HqXLpSphRt/
>>>>>>>>>>>> gN5HIK
>>>>>>>>>>>> 77gD
>>>>>>>>>>>> xNpfymdAPf2iJ1Dur5MNngnJIYTgdNgYq8pTvmrxPNEixumYsTU1UQ
>>>>>>>>>>>> zYFbij
>>>>>>>>>>>> pctI
>>>>>>>>>>>> yL8gUhP2kX2Yd45mVGzZxmJXDrOkg0rLxmIxVrrmGH2/iLSHpSFE+8
>>>>>>>>>>>> mY0kqb
>>>>>>>>>>>> TmA=
>>>>>>>>>>>> =86fD
>>>>>>>>>>>> -----END PGP SIGNATURE-----
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Aaron Peeler
>>>>>>>>>>> Program Manager
>>>>>>>>>>> Virtual Computing Lab
>>>>>>>>>>> NC State University
>>>>>>>>>>>
>>>>>>>>>>> All electronic mail messages in connection with State
>>>>>>>>>>> business
>>>>>>>>
>>>>>>>> which
>>>>>>>>
>>>>>>>>>>> are sent to or received by this account are subject to
>>>>>>>>>>> the NC
>>>>>>>>
>>>>>>>> Public
>>>>>>>>
>>>>>>>>>>> Records Law and may be disclosed to third parties.
>>>>>>>>>>
>>>>>>>>>> Thank you.
>>>>>>>>>>
>>>>>>>>>> -----------------
>>>>>>>>>> Dmitri Chebotarov
>>>>>>>>>> dcheb...@gmu.edu
>>>>>>>>>> 703-993-6175
>>>>>>>>>> -----------------
>>>>>>>>
>>>>>>>> Thank you.
>>>>>>>>
>>>>>>>> -----------------
>>>>>>>> Dmitri Chebotarov
>>>>>>>> dcheb...@gmu.edu
>>>>>>>> 703-993-6175
>>>>>>>> -----------------
>>>>>
>>>>> Thank you.
>>>>>
>>>>> -----------------
>>>>> Dmitri Chebotarov
>>>>> dcheb...@gmu.edu
>>>>> 703-993-6175
>>>>> -----------------
>>>>
>>>> - --
>>>> - -------------------------------
>>>> Josh Thompson
>>>> VCL Developer
>>>> North Carolina State University
>>>>
>>>> my GPG/PGP key can be found at pgp.mit.edu
>>>>
>>>> All electronic mail messages in connection with State business which
>>>> are sent to or received by this account are subject to the NC Public
>>>> Records Law and may be disclosed to third parties.
>>>
>>> Thank you.
>>>
>>> -----------------
>>> Dmitri Chebotarov
>>> dcheb...@gmu.edu
>>> 703-993-6175
>>> -----------------
>>
>> Thank you.
>>
>> -----------------
>> Dmitri Chebotarov
>> dcheb...@gmu.edu
>> 703-993-6175
>> -----------------
> - --
> - -------------------------------
> Josh Thompson
> VCL Developer
> North Carolina State University
>
> my GPG/PGP key can be found at pgp.mit.edu
>
> All electronic mail messages in connection with State business which
> are sent to or received by this account are subject to the NC Public
> Records Law and may be disclosed to third parties.
>
>
Thank you.
-----------------
Dmitri Chebotarov
dcheb...@gmu.edu
703-993-6175
-----------------
signature.asc
Description: Message signed with OpenPGP using GPGMail
