Diego, Good spot! Yes, that's almost certainly the issue. I've raised https://github.com/Metaswitch/crest/issues/95 and we hope to fix in the next day or two.
In the meantime, running the following command over the *.casscli script should fix the problem. sed -e "s/version='1.0' encoding='UTF-8'/version=\\\\'1.0\\\\' encoding=\\\\'UTF-8\\\\'/g" Thanks for raising this! Matt From: [email protected] [mailto:[email protected]] On Behalf Of Diego Lugones Sent: 30 October 2013 20:06 To: [email protected] Subject: Re: [Clearwater] Clearwater Digest, Vol 6, Issue 15 Hi Matt In addition to last Davide's email, let me add that after checking impu and impi tables we found that the entries corresponding to the automatically generated numbers are incomplete. So, we open the 'users.auto.create_homestead_cache.casscli' file and found some content like: SET impu['sip:[email protected]<mailto:sip%[email protected]>']['ims_subscription_xml'] = '<?xml version='1.0' encoding='UTF-8'?> <IMSSubscription xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:noNamespaceSchemaLocation="CxDataType.xsd"><PrivateID>[email protected]<mailto:[email protected]></PrivateID><ServiceProfile><InitialFilterCriteria><TriggerPoint><ConditionTypeCNF>0</ConditionTypeCNF><SPT><ConditionNegated>0</ConditionNegated><Group>0</Group><Method>INVITE</Method><Extension /></SPT></TriggerPoint><ApplicationServer><ServerName>sip:mmtel.clearwater.aws.info<http://mmtel.clearwater.aws.info></ServerName><DefaultHandling>0</DefaultHandling></ApplicationServer></InitialFilterCriteria><PublicIdentity><Identity>sip:[email protected]<mailto:sip%[email protected]></Identity></PublicIdentity></ServiceProfile></IMSSubscription>'; But this content is not actually in the table. We inserted this command manually in the cassandra-cli bash, and we got a format error in the line: '<?xml version='1.0' encoding='UTF-8'?> So, we replaced it by: '<?xml version=\'1.0\' encoding=\'UTF-8\'?> After this change we could register! some bulk-generated numbers Bottom line: I think the problem can be in the python script generating the 'users.auto.create_homestead_cache.casscli' file, particularly the line mentioned above. We are writing a parser to fix the syntax error in users.auto.create_homestead_cache.casscli but probably you can help us to confirm the error and change the .py accordingly (if this makes sense to you) Thanks a lot Diego El miércoles, 30 de octubre de 2013, <[email protected]<mailto:[email protected]>> escribió: > Send Clearwater mailing list submissions to > > [email protected]<mailto:[email protected]> > > To subscribe or unsubscribe via the World Wide Web, visit > http://lists.projectclearwater.org/listinfo/clearwater > or, via email, send a message with subject or body 'help' to > > [email protected]<mailto:[email protected]> > > You can reach the person managing the list at > > [email protected]<mailto:[email protected]> > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Clearwater digest..." > > > Today's Topics: > > 1. Re: bulk provisioning (Davide Cherubini) > 2. Re: bulk provisioning (Matt Williams) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 30 Oct 2013 17:00:31 +0000 > From: Davide Cherubini <[email protected]<mailto:[email protected]>> > Cc: > "[email protected]<mailto:[email protected]>" > > <[email protected]<mailto:[email protected]>> > Subject: Re: [Clearwater] bulk provisioning > Message-ID: > > <caf88vyxjpugpoab0stwqswvif-t4azp4lqbcpucsadqbopd...@mail.gmail.com<mailto:caf88vyxjpugpoab0stwqswvif-t4azp4lqbcpucsadqbopd...@mail.gmail.com>> > Content-Type: text/plain; charset="iso-8859-1" > > Hi Matt, > > thanks for your quick reply. > > Yes, I could create and register a user through Ellis. > Following your instructions, this is what I get: > >> list impi; > > RowKey: [email protected]<mailto:[email protected]> > > => (column=_exists, value=, timestamp=1383142617627000) > > => (column=digest_ha1, value=71b115c7ecad54fc81aeb371de63de6c, > timestamp=1383142617635000) > > => > (column=public_id_sip:[email protected]<mailto:public_id_sip%[email protected]>, > value=sip:[email protected]<mailto:sip%[email protected]>, > timestamp=1383142617637000) > > ------------------- > > RowKey: [email protected]<mailto:[email protected]> > > => (column=_exists, value=, timestamp=1382809021476256) > > => (column=digest_ha1, value=5e45502272d8f0910a0740b2dcc52892, > timestamp=1382809021476256) > > => > (column=public_id_sip:[email protected]<mailto:public_id_sip%[email protected]>, > value=, > timestamp=1382809021476256) > > > The first user (7775550000 )is provisioned via the bulk script, the second > one (6505550817) with Ellis. The only difference I can see is that the > second one doesn't have a value for the public_id. Is that a problem? > >> list impu; (more interesting IMHO) > > RowKey: sip:[email protected]<mailto:sip%[email protected]> > > => (column=_exists, value=, timestamp=1383142617640000) > > ------------------- > > RowKey: sip:[email protected]<mailto:sip%[email protected]> > > => (column=_exists, value=, timestamp=1383050347241690) > > => (column=ims_subscription_xml, value=<?xml version='1.0' > encoding='UTF-8'?> > > <IMSSubscription xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; > xsi:noNamespaceSchemaLocation="CxDataType.xsd"><PrivateID>Unspecified</PrivateID><ServiceProfile><InitialFilterCriteria><TriggerPoint><ConditionTypeCNF>0</ConditionTypeCNF><SPT><ConditionNegated>0</ConditionNegated><Group>0</Group><Method>INVITE</Method><Extension > /></SPT></TriggerPoint><ApplicationServer><ServerName>sip:mmtel.aws.info<http://mmtel.aws.info> > </ServerName><DefaultHandling>0</DefaultHandling></ApplicationServer></InitialFilterCriteria><PublicIdentity><Identity> > sip:[email protected]<mailto:sip%[email protected]></Identity></PublicIdentity></ServiceProfile></IMSSubscription>, > timestamp=1383050347241690) > > > As you can see, here the user provisioned with Ellis has all the columns > while the one created with the script is missing the ims_subscription_xml > one. > > > Thanks again for your help! > > Davide > > > > > > 2013/10/30 Matt Williams > <[email protected]<mailto:[email protected]>> > >> Davide,**** >> >> ** ** >> >> Yes, you can query the data in the Cassandra databases on the homestead >> and homer nodes using the "cassandra-cli" tool. If the problem is with the >> database contents, it will almost certainly be with the homestead database >> - missing data from the homer database doesn't matter for registration.*** >> * >> >> ** ** >> >> On homestead, once you're in cassandra-cli, type "use homestead_cache ;" >> and then "list impi;" and "list impu;".**** >> >> **? **For "list impi;", you should see a list of rows named by >> the private user ID (the SIP URI with the "sip:" prefix stripped) and each >> should have an _exists column, a digest_ha1 column and a public_id_<SIP >> URI> column.**** >> >> **? **For "list impu;", you should see a list of rows named by >> the public user ID (the full SIP URI) and each should have an _exists >> column and an ims_subscription_xml column.**** >> >> ** ** >> >> Please check that these are present. If not, please retry provisioning >> and let me know any errors you see.**** >> >> ** ** >> >> If the rows and columns are all present, I think you'd be best to check >> the bono, sprout and homestead logs, as described in >> https://github.com/Metaswitch/clearwater-docs/wiki/Troubleshooting-and-Recovery. >> If you need help with this, please let me know.**** >> >> ** ** >> >> Incidentally, have you successfully created and registered a subscriber >> through ellis? If not, I'd suggest doing this first - this is simpler so >> we can check that's working before moving to bulk-provisioning.**** >> >> ** ** >> >> BTW, I notice you're not subscribed to the mailing list ( >> http://lists.projectclearwater.org/listinfo/clearwater) - I'd suggest >> subscribing so that you can see other **** >> >> ** ** >> >> Thanks,**** >> >> ** ** >> >> Matt**** >> >> ** ** >> >> *From:* >> [email protected]<mailto:[email protected]> >> [mailto: >> [email protected]<mailto:[email protected]>] >> *On Behalf Of *Davide >> Cherubini >> *Sent:* 30 October 2013 13:53 >> *To:* >> [email protected]<mailto:[email protected]> >> *Subject:* [Clearwater] bulk provisioning**** >> >> ** ** >> >> Hi, **** >> >> I am trying to use the bulk provisioning 'recipe' in **** >> >> https://github.com/Metaswitch/crest/blob/dev/docs/Bulk-Provisioning%20Numbers.md**** >> >> ** ** >> >> but my clients don't register. >> >> Is there any way to check that users have been correctly provisioned to >> homer and homestead?**** >> >> ** ** >> >> Thanks,**** >> >> Davide**** >> >> > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > <http://lists.projectclearwater.org/pipermail/clearwater/attachments/20131030/2fd73e30/attachment-0001.html> > > ------------------------------ > > Message: 2 > Date: Wed, 30 Oct 2013 19:37:32 +0000 > From: Matt Williams > <[email protected]<mailto:[email protected]>> > To: Davide Cherubini <[email protected]<mailto:[email protected]>> > Cc: > "[email protected]<mailto:[email protected]>" > > <[email protected]<mailto:[email protected]>> > Subject: Re: [Clearwater] bulk provisioning > Message-ID: > > <[email protected]<mailto:[email protected]>> > Content-Type: text/plain; charset="us-ascii" > > Davide, > > Good investigation - thanks for that! > > The ipmi public_id_* issue is a good spot, but it's benign. I've committed a > fix for this anyway - it should match what ellis would configure. > > The impu ims_subscription_xml issue is the cause. When registering, we need > to get the list of implicitly-registered public IDs, and this is part of the > subscription XML. Please can you > > * confirm the version of the bulk_create.py script you're running (or > the homestead package it came from via "dpkg-query -s homestead") > > * share the *.create_homestead_cache.casscli file you ran to import > this > > * share the output of running *.create_homestead.sh? > > Thanks, > > Matt > > From: > [email protected]<mailto:[email protected]> > > [mailto:[email protected]<mailto:[email protected]>] > On Behalf Of Davide Cherubini > Sent: 30 October 2013 17:01 > Cc: > [email protected]<mailto:[email protected]> > Subject: Re: [Clearwater] bulk provisioning > > Hi Matt, > > thanks for your quick reply. > > Yes, I could create and register a user through Ellis. > Following your instructions, this is what I get: > >> list impi; > RowKey: > [email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>> > => (column=_exists, value=, timestamp=1383142617627000) > => (column=digest_ha1, value=71b115c7ecad54fc81aeb371de63de6c, > timestamp=1383142617635000) > => > (column=public_id_sip:[email protected]<mailto:public_id_sip%[email protected]><mailto:public_id_sip%[email protected]<mailto:public_id_sip%[email protected]>>, > > value=sip:[email protected]<mailto:sip%[email protected]><mailto:sip%[email protected]<mailto:sip%[email protected]>>, > timestamp=1383142617637000) > ------------------- > RowKey: > [email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>> > => (column=_exists, value=, timestamp=1382809021476256) > => (column=digest_ha1, value=5e45502272d8f0910a0740b2dcc52892, > timestamp=1382809021476256) > => > (column=public_id_sip:[email protected]<mailto:public_id_sip%[email protected]><mailto:public_id_sip%[email protected]<mailto:public_id_sip%[email protected]>>, > value=, timestamp=1382809021476256) > > The first user (7775550000 )is provisioned via the bulk script, the second > one (6505550817) with Ellis. The only difference I can see is that the second > one doesn't have a value for the public_id. Is that a problem? >> list impu; (more interesting IMHO) > RowKey: > sip:[email protected]<mailto:sip%[email protected]><mailto:sip%[email protected]<mailto:sip%[email protected]>> > => (column=_exists, value=, timestamp=1383142617640000) > ------------------- > RowKey: > sip:[email protected]<mailto:sip%[email protected]><mailto:sip%[email protected]<mailto:sip%[email protected]>> > => (column=_exists, value=, timestamp=1383050347241690) > => (column=ims_subscription_xml, value=<?xml version='1.0' encoding='UTF-8'?> > <IMSSubscription xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; > xsi:noNamespaceSchemaLocation="CxDataType.xsd"><PrivateID>Unspecified</PrivateID><ServiceProfile><InitialFilterCriteria><TriggerPoint><ConditionTypeCNF>0</ConditionTypeCNF><SPT><ConditionNegated>0</ConditionNegated><Group>0</Group><Method>INVITE</Method><Extension > > /></SPT></TriggerPoint><ApplicationServer><ServerName>sip:mmtel.aws.info<http://mmtel.aws.info><http://mmtel.aws.info></ServerName><DefaultHandling>0</DefaultHandling></ApplicationServer></InitialFilterCriteria><PublicIdentity><Identity>sip:[email protected]<mailto:sip%[email protected]><mailto:sip%[email protected]<mailto:sip%[email protected]>></Identity></PublicIdentity></ServiceProfile></IMSSubscription>, > timestamp=1383050347241690) > > As you can see, here the user provisioned with Ellis has all the columns > while the one created with the script is missing the ims_subscription_xml one. > > Thanks again for your help! > Davide > > > > 2013/10/30 Matt Williams > <[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>> > Davide, > > Yes, you can query the data in the Cassandra databases on the homestead and > homer nodes using the "cassandra-cli" tool. If the problem is with the > database contents, it will almost certainly be with the homestead database - > missing data from the homer database doesn't matter for registration. > > On homestead, once you're in cassandra-cli, type "use homestead_cache ;" and > then "list impi;" and "list impu;". > > * For "list impi;", you should see a list of rows named by the > private user ID (the SIP URI with the "sip:" prefix stripped) and each should > have an _exists column, a digest_ha1 column and a public_id_<SIP URI> column. > > * For "list impu;", you should see a list of rows named by the public > user ID (the full SIP URI) and each should have an _exists column and an > ims_subscription_xml column. > > Please check that these are present. If not, please retry provisioning and > let me know any errors you see. > > If the rows and columns are all present, I think you'd be best to check the > bono, sprout and homestead logs, as described in > https://github.com/Metaswitch/clearwater-docs/wiki/Troubleshooting-and-Recovery. > If you need help with this, please let me know. > > Incidentally, have you successfully created and registered a subscriber > through ellis? If not, I'd suggest doing this first - this is simpler so we > can check that's working before moving to bulk-provisioning. > > BTW, I notice you're not subscribed to the mailing list > (http://lists.projectclearwater.org/listinfo/clearwater) - I'd suggest > subscribing so that you can see other > > Thanks, > > Matt > > From: > [email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>> > [mailto: -- Diego
_______________________________________________ Clearwater mailing list [email protected] http://lists.projectclearwater.org/listinfo/clearwater
