(added dev@ back in the to: list) And not supporting a current version on a new feature is pretty lame.. Sounds like you both feel it's important enough for another RC.. Any other opinions (in agreement or not)?
On Thu, Dec 22, 2016 at 1:55 PM, David Neuman <[email protected]> wrote: > Sorry riak 2.1.3 not 1.3 > > On Thu, Dec 22, 2016 at 13:52 David Neuman <[email protected]> wrote: >> >> This was functionality introduced in 1.7. Either we say we don't support >> riak > 1.3 or we fix it. >> >> On Thu, Dec 22, 2016 at 13:43 Dan Kirkwood <[email protected]> wrote: >>> >>> just trying to follow the guidelines for the release.. Question is >>> >>> whether this is a regression from a previous release. If it is, >>> >>> then it definitely qualifies. If not, then it's up for debate. >>> >>> Not pushing one way or the other, just want to bring up that point... >>> >>> >>> >>> -dan >>> >>> >>> >>> On Thu, Dec 22, 2016 at 1:30 PM, Hank Beatty <[email protected]> wrote: >>> >>> > I'm going to say it probably did but, we are jumping 1.7. We're going >>> > from >>> >>> > 1.6 -> 1.8. >>> >>> > >>> >>> > On 12/22/16 3:19 PM, Dan Kirkwood wrote: >>> >>> >> >>> >>> >> Haven't read thru the whole thread, but did this problem exist in >>> >> 1.7? >>> >>> >> >>> >>> >> >>> >>> >> On Thu, Dec 22, 2016 at 12:48 PM, Dave Neuman <[email protected]> >>> >> wrote: >>> >>> >>> >>> >>> >>> No problem. That worries me though. This might warrant a -1 on >>> >>> 1.8.0 >>> >>> >>> RC5 >>> >>> >>> so we can cherry pick that change into 1.8. Thoughts? >>> >>> >>> >>> >>> >>> >>> >>> >>> On Thu, Dec 22, 2016 at 12:41 Hank Beatty <[email protected]> wrote: >>> >>> >>>> >>> >>> >>>> >>> >>> >>>> Changing the line in RiakAdapter.pm and re-saving the cert fixed the >>> >>> >>>> issue. >>> >>> >>>> >>> >>> >>>> >>> >>> >>>> >>> >>> >>>> Thanks for all of your help. I really appreciate it. >>> >>> >>>> >>> >>> >>>> >>> >>> >>>> >>> >>> >>>> Hank >>> >>> >>>> >>> >>> >>>> >>> >>> >>>> >>> >>> >>>> On 12/22/16 1:30 PM, David Neuman wrote: >>> >>> >>>> >>> >>> >>>>> Can you try changing line 104 >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> of traffic_ops/app/lib/Connection/RiakAdapter.pm from >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> return $ua->put( $fqdn, Content => $value ); >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> to >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> return $ua->put( $fqdn, Content => $value, 'Content-Type'=> >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> $content_type ); >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> ? >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> 1.8: >>> >>> >>>>> >>> >>> >>>>> >>> >>>>> https://github.com/apache/incubator-trafficcontrol/blob/RELEASE-1.8.0-RC5/traffic_ops/app/lib/Connection/RiakAdapter.pm#L104 >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> master: >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>>> >>> >>> >>>>> >>> >>>>> https://github.com/apache/incubator-trafficcontrol/blob/master/traffic_ops/app/lib/Connection/RiakAdapter.pm#L102 >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> This was reported as an issue that I couldn't reproduce but fixed >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> anyway, I am wondering if it is specific to the newer vesions of >>> >>>>> Riak. >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> On Thu, Dec 22, 2016 at 10:39 AM, Hank Beatty <[email protected] >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> <mailto:[email protected]>> wrote: >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> On 12/22/2016 11:32 AM, Dave Neuman wrote: >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> Are you running the 1.8 version of Traffic Ops? You need >>> >>>>> to be >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> running >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> at least 1.7 for migration and sslkeys endpoint to work. >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> I'm running the 1.8 version of TO. >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> If so, here are a couple of things to check in Riak: >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> First, run this to make sure the schema is there: curl >>> >>>>> -kvs >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> "https://admin:pass@riakserver:8088/search/schema/sslkeys" >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> It should return the XML schema. >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> This does return the schema: >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> < HTTP/1.1 200 OK >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> < Server: MochiWeb/1.1 WebMachine/1.10.8 (that head fake, tho) >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> < Date: Thu, 22 Dec 2016 17:25:51 GMT >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> < Content-Type: application/xml >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> < Content-Length: 1860 >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> < >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> <?xml version="1.0" encoding="UTF-8" ?><schema name="schedule" >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> version="1.5"> <fields> <field name="cdn" type="string" >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> indexed="true" stored="true" /> <field name="deliveryservice" >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> type="string" indexed="true" stored="true" /> <field >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> name="hostname" type="string" indexed="true" >>> >>>>> stored="true" /> >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> <field name="certificate.crt" type="string" indexed="false" >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> stored="true" /> <field name="certificate.key" type="string" >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> indexed="false" stored="true" /> <!-- All of these fields are >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> required by Riak Search --> <field name="_yz_id" >>> >>>>> type="_yz_str" >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> indexed="true" stored="true" multiValued="false" >>> >>>>> required="true"/> >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> <field name="_yz_ed" type="_yz_str" indexed="true" >>> >>>>> stored="false" >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> multiValued="false"/> <field name="_yz_pn" type="_yz_str" >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> indexed="true" stored="false" multiValued="false"/> <field >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> name="_yz_fpn" type="_yz_str" indexed="true" stored="false" >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> multiValued="false"/> <field name="_yz_vtag" type="_yz_str" >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> indexed="true" stored="false" multiValued="false"/> <field >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> name="_* Connection #0 to host riak1 left intact >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> * Closing connection #0 >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> yz_rk" type="_yz_str" indexed="true" stored="true" >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> multiValued="false"/> <field name="_yz_rt" type="_yz_str" >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> indexed="true" stored="true" multiValued="false"/> <field >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> name="_yz_rb" type="_yz_str" indexed="true" stored="true" >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> multiValued="false"/> <field name="_yz_err" type="_yz_str" >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> indexed="true" stored="false" multiValued="false"/> <!--catch >>> >>>>> all >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> field--> <dynamicField name="*" type="ignored" /> </fields> >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> <uniqueKey>_yz_id</uniqueKey> <types> <fieldType >>> >>>>> name="string" >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> class="solr.StrField" sortMissingLast="true" /> <!-- YZ >>> >>>>> String: >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> Used for non-analyzed fields --> <fieldType name="_yz_str" >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> class="solr.StrField" sortMissingLast="true" /> <!-- Used >>> >>>>> for >>> >>> >>>>> the >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> catch all field --> <fieldtype name="ignored" stored="false" >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> indexed="false" multiValued="true" class="solr.StrField" /> >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> </types></schema> >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> Next, make sure the index was created correctly by running: >>> >>> >>>>> curl >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> -kvs >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> "https://admin:pass@riakserver:8088/search/index/sslkeys" >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> It should return some json >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> like: {"name":"sslkeys","n_val":3,"schema":"sslkeys"} >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> this also appears to work: >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> < HTTP/1.1 200 OK >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> < Server: MochiWeb/1.1 WebMachine/1.10.8 (that head fake, tho) >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> < Date: Thu, 22 Dec 2016 17:28:10 GMT >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> < Content-Type: application/json >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> < Content-Length: 47 >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> < >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> * Connection #0 to host riak1 left intact >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> * Closing connection #0 >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> {"name":"sslkeys","n_val":3,"schema":"sslkeys"} >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> Finally check the bucket props to make sure the bucket has >>> >>>>> the >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> index on >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> it by running the following (you might need to run >>> >>>>> "riak-admin >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> security >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> grant riak_core.get_bucket on ssl to admin" from one of the >>> >>> >>>>> riak >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> servers >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> first to add permissions): curl -kvs >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>>> >>> >>> >>>>> >>> >>>>> "https://admin:[email protected]:8088/buckets/ssl/props >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>>> >>> >>> >>>>> >>> >>>>> <https://admin:[email protected]:8088/buckets/ssl/props>" >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> It should return json and you should see >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> "search_index":"sslkeys" in the >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> json >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> This appears to be there as well: >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> < HTTP/1.1 200 OK >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> < Vary: Accept-Encoding >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> < Server: MochiWeb/1.1 WebMachine/1.10.8 (that head fake, tho) >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> < Date: Thu, 22 Dec 2016 17:31:46 GMT >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> < Content-Type: application/json >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> < Content-Length: 485 >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> < >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> * Connection #0 to host riak1 left intact >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> * Closing connection #0 >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>>> >>> >>> >>>>> >>> >>>>> {"props":{"allow_mult":false,"basic_quorum":false,"big_vclock":50,"chash_keyfun":{"mod":"riak_core_util","fun":"chash_std_keyfun"},"dvv_enabled":false,"dw":"quorum","last_write_wins":false,"linkfun":{"mod":"riak_kv_wm_link_walker","fun":"mapreduce_linkfun"},"n_val":3,"name":"ssl","notfound_ok":true,"old_vclock":86400,"postcommit":[],"pr":0,"precommit":[],"pw":0,"r":"quorum","rw":"quorum","search_index":"sslkeys","small_vclock":50,"w":"quorum","write_once":false,"young_vclock":20}} >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> Let me know if all of those things check out. If all of >>> >>>>> those >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> things >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> are there then this should work: curl -kvs >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>>> >>> >>> >>>>> >>> >>>>> "https://admin:password@riakserver:8088/search/query/sslkeys?wt=json&q=cdn:cdn10" >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> this still says numFound: 0 >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> < HTTP/1.1 200 OK >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> < Server: MochiWeb/1.1 WebMachine/1.10.8 (that head fake, tho) >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> < Date: Thu, 22 Dec 2016 17:34:35 GMT >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> < Content-Type: application/json; charset=UTF-8 >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> < Content-Length: 734 >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> < >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>>> >>> >>> >>>>> >>> >>>>> {"responseHeader":{"status":0,"QTime":6,"params":{"riak1.atl-priv.kabletown.test:8093":"_yz_pn:64 >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> OR (_yz_pn:61 AND (_yz_fpn:61)) OR _yz_pn:60 OR _yz_pn:57 OR >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> _yz_pn:48 OR _yz_pn:45 OR _yz_pn:33 OR _yz_pn:21 OR >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>>> >>> >>> >>>>> >>> >>>>> _yz_pn:9","shards":"riak1.atl-priv.kabletown.test:8093/internal_solr/sslkeys,riak2.atl-priv.kabletown.test:8093/internal_solr/sslkeys,riak3.atl-priv.kabletown.test:8093/internal_solr/sslkeys","q":"cdn:cdn10","riak2.atl-priv.kabletown.test:8093":"_yz_pn:54 >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> OR _yz_pn:51 OR _yz_pn:42 OR _yz_pn:30 OR _yz_pn:18 OR >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> _yz_pn:6","riak3.atl-priv.kabletown.test:8093":"_yz_pn:39 OR >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> _yz_pn:36 OR _yz_pn:27 OR _yz_pn:24 OR _yz_pn:15 OR _yz_pn:12 >>> >>>>> OR >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>>> >>> >>> >>>>> >>> >>>>> _yz_pn:3","wt":"json"}},"response":{"numFound":0,"start":0,"maxScore":0.0,"docs":[]}} >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> In addition to running the migration script, you could also >>> >>> >>>>> open >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> the ssl >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> cert page for a DS, go to Paste Existing Certs and save. >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> I did this also. Checked to make sure the "version" of the cert >>> >>>>> was >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> incremented using the API. It was but, the query still returns >>> >>>>> the >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> same as above. >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> On Thu, Dec 22, 2016 at 6:57 AM, Hank Beatty >>> >>>>> <[email protected] >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> <mailto:[email protected]> >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> <mailto:[email protected] <mailto:[email protected]>>> >>> >>>>> wrote: >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> On 12/22/2016 08:30 AM, Hank Beatty wrote: >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> Also my riakuser doesn't appear to have the ability >>> >>>>> to >>> >>> >>>>> query >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> even after >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> running: >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> I figured this out. >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> I re-ran: >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> riak-admin security add-source riakuser 0.0.0.0/0 >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> <http://0.0.0.0/0> <http://0.0.0.0/0> >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> password >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> In my testing I deleted the riakuser at some point and >>> >>> >>>>> missed >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> re-running this. >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> I can query using the riakuser now but, getting the >>> >>>>> same >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> thing as >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> when I run as admin. >>> >>> >>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>>>> >>> >>> >>>> >>> >>> >>> >>> >>> > >>> >
