Re: [gentoo-user] Major site redesign, SEO, and 301 redirects

[Mick]

On Friday 02 Oct 2015 00:00:08 Walter Dnes wrote:
> On Thu, Oct 01, 2015 at 03:35:48PM +0100, Mick wrote
> 
> > PS.  I hope someone will show them the door if they suggest designing
> > a new Flash based web interface ...
> 
>   Especially true given that Ipads/Iphones do not support Flash.  Another
> major problem is websites that parse the user agent, and assume that any
> browser they don't recognize is a mobile device.  It's a standing joke
> amongst geeks... see https://xkcd.com/869/ and https://xkcd.com/1174/
> Back in the day when "smartphones" only had 320x240 pixel screens, a
> separate mobile site may have been necessary.  

I think the reason was that at the time we did not have CSS3 and responsive 
web design was not available for the majority of CMS themes.  Many web 
designers were providing a separate generic style sheet for mobile browsers.  
If for some reason the browser was not able to process the desktop style sheet 
it would drop to the mobile style sheet.  I am sure I have seen this happening 
with Opera.  With responsive design it is left to the device to resize the 
layout elements, which is a relief for the web developers - they now only have 
to sniff MSIE8 and friends and send to them a completely different (crippled) 
layout that they are able to render.  :-D

-- 
Regards,
Mick


signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] Major site redesign, SEO, and 301 redirects

[Alan McKinnon]

On 01/10/2015 13:35, Tanstaafl wrote:
> Thanks Alan (and everyone else),
> 
> One important follow-up below...
> 
> On 9/29/2015 8:28 PM, Alan McKinnon  wrote:
>> It would be wise to clarify with the devs exactly what it is they are
>> looking for.
> 
> That is the purpose of my upcoming phone call with him.
> 
>> And overall, in your shoes I would be firm, adamant and above all polite
>> and say that infrastructure changes go through you and you alone, and
>> must be vetted by you with full transparency.
> 
> That is what I've been doing so far, but I think the boss is getting
> close to just saying 'give it to them'...

Depending on how senior you are in the place, as technical guy you have
a duty to perform diligence. Persist.

> 
> But - no one has addressed my main question...
> 
> I understand that 301 redirects are performed by web servers only, you
> can't really do these in DNS. However, some Managed DNS providers -
> DNSMadeEasy included - offer this ability as a service. DNSMadeEasy
> calls  them 'http redirects', and the actual redirect is accomplished by
> one of their own web servers they have set up to handle these.

Information is still sparse, so I'm having to fill in the blanks a lot.
Here's what I imagine is probably happening:

The only useful thing you can get out of DNS for an HTTP request is an A
record for an IP address.

Say you are example.com and do your own DNS; www.example.com is 1.2.3.4.
A SaaS provider can control your DNS and they set the TTL on that A
record very low so (like DynDNS does) they can point it at their web
servers.

A request comes in for http://www.example.com/index.html, and your DNS
cache needs to query it. The provider's DNS returns 2.3.4.5 which is the
provider's front end web server. That web server figures out the address
is your's, and issues a 301 to the user, which takes them to the
production web server with the real site on it.

Providers do this a lot so they can load balance web sites, redirect
users to local nearby web servers and other optimizations. The downside
is they need to control your DNS.

Me, personally I would never allow that, not for the entire domain. I
would rather delegate the specific address they want to control
(www.example.com) and let them tweak that all day if they like.

> Is it 'normal' to do these 301 redirects at the DNS level like that? I
> would think they should be using the current web server hosting the
> current site to start doing the redirects as they get the new landing
> pages done?

Depends what their business model is. If they deliver the full service,
they'd have to do something like I described above for it to work.

This is assuming the contractor is a full SaaS provider and not only a
web-site developement company

> Apache does this using a .htaccess file (if I'm interpreting
> my googling responses correctly).

An .htaccess file is nothing special, all it is is a config file that
can contain whatever directives are allowed in httpd.conf but applies
only to the directive .htaccess is in. Everything in .htaccess is a
valid directive that can go in httpd.conf, but not necessarily the other
way round. They are especially useful for shared hosting where you want
your customers to be able to tweak specific directives for their sites
and you can't give them access to httpd.conf and really can't be
bothered doing it for them for every requested change :-)

So when google gives a result saying "do it in .htaccess", that's the
internetz being meaningless. What it really means is "configure apache
to do a redirect for URLs that look like so"


> And now that I worded it that way - how would they do that exactly?
> Would the proper method be to redirect it to a new test domain, ie:
> 
> www.example.com/page1.htm >> www.new-example.com/newpage1.htm ?
> 
> Or save the new page on the old server, then do:
> 
> www.example.com/page1.htm >> www.example.com/newpage1.htm ?
> 
> Now I'm confusing myself...


It can get confusing. Best to ask them directly what they intend to do.
We can presume all day and never figure it out.


-- 
Alan McKinnon
alan.mckin...@gmail.com

Re: [gentoo-user] Major site redesign, SEO, and 301 redirects

[Tanstaafl]

Thanks to Alan and the others for the responses...

The main problem is this project is being managed by a non-tech manager
who apparently thinks they know a lot more than they do, and the Boss is
technically challenged, so it is easy for someone to convince him of
almost anything (like, he should delegate this to a non-tech person and
not involve his one tech guy)...

One reason he sometimes doesn't involve me until things get to this
point is because I tend to be a 'wet blanket', ruining bright shiny
sales pitches with injections of reality. You'd think he'd have learned
by now. The last time, about 5 years ago, the person who managed the
project (different person) didn't get ownership of the source code in
the contract, so we didn't get all of the source files for the Flash
junk they created, then when we wanted to make some changes to the text
embedded in  the Flash, I had to ask them for the source files, and
they wanted a bunch of money. Unbelievable.

We'll see how the dev(s) respond to my questions, but I may come back
here with more info and more advice if I need it.

Thanks again to all, it has been a big help!

On 10/1/2015 7:58 AM, Alan McKinnon  wrote:
> On 01/10/2015 13:35, Tanstaafl wrote:
>> Thanks Alan (and everyone else),
>>
>> One important follow-up below...
>>
>> On 9/29/2015 8:28 PM, Alan McKinnon  wrote:
>>> It would be wise to clarify with the devs exactly what it is they are
>>> looking for.
>>
>> That is the purpose of my upcoming phone call with him.
>>
>>> And overall, in your shoes I would be firm, adamant and above all polite
>>> and say that infrastructure changes go through you and you alone, and
>>> must be vetted by you with full transparency.
>>
>> That is what I've been doing so far, but I think the boss is getting
>> close to just saying 'give it to them'...
> 
> Depending on how senior you are in the place, as technical guy you have
> a duty to perform diligence. Persist.
> 
>>
>> But - no one has addressed my main question...
>>
>> I understand that 301 redirects are performed by web servers only, you
>> can't really do these in DNS. However, some Managed DNS providers -
>> DNSMadeEasy included - offer this ability as a service. DNSMadeEasy
>> calls  them 'http redirects', and the actual redirect is accomplished by
>> one of their own web servers they have set up to handle these.
> 
> Information is still sparse, so I'm having to fill in the blanks a lot.
> Here's what I imagine is probably happening:
> 
> The only useful thing you can get out of DNS for an HTTP request is an A
> record for an IP address.
> 
> Say you are example.com and do your own DNS; www.example.com is 1.2.3.4.
> A SaaS provider can control your DNS and they set the TTL on that A
> record very low so (like DynDNS does) they can point it at their web
> servers.
> 
> A request comes in for http://www.example.com/index.html, and your DNS
> cache needs to query it. The provider's DNS returns 2.3.4.5 which is the
> provider's front end web server. That web server figures out the address
> is your's, and issues a 301 to the user, which takes them to the
> production web server with the real site on it.
> 
> Providers do this a lot so they can load balance web sites, redirect
> users to local nearby web servers and other optimizations. The downside
> is they need to control your DNS.
> 
> Me, personally I would never allow that, not for the entire domain. I
> would rather delegate the specific address they want to control
> (www.example.com) and let them tweak that all day if they like.
> 
>> Is it 'normal' to do these 301 redirects at the DNS level like that? I
>> would think they should be using the current web server hosting the
>> current site to start doing the redirects as they get the new landing
>> pages done?
> 
> Depends what their business model is. If they deliver the full service,
> they'd have to do something like I described above for it to work.
> 
> This is assuming the contractor is a full SaaS provider and not only a
> web-site developement company
> 
>> Apache does this using a .htaccess file (if I'm interpreting
>> my googling responses correctly).
> 
> An .htaccess file is nothing special, all it is is a config file that
> can contain whatever directives are allowed in httpd.conf but applies
> only to the directive .htaccess is in. Everything in .htaccess is a
> valid directive that can go in httpd.conf, but not necessarily the other
> way round. They are especially useful for shared hosting where you want
> your customers to be able to tweak specific directives for their sites
> and you can't give them access to httpd.conf and really can't be
> bothered doing it for them for every requested change :-)
> 
> So when google gives a result saying "do it in .htaccess", that's the
> internetz being meaningless. What it really means is "configure apache
> to do a redirect for URLs that look like so"
> 
> 
>> And now that I worded it that 

Re: [gentoo-user] Major site redesign, SEO, and 301 redirects

[Tanstaafl]

Thanks Alan (and everyone else),

One important follow-up below...

On 9/29/2015 8:28 PM, Alan McKinnon  wrote:
> It would be wise to clarify with the devs exactly what it is they are
> looking for.

That is the purpose of my upcoming phone call with him.

> And overall, in your shoes I would be firm, adamant and above all polite
> and say that infrastructure changes go through you and you alone, and
> must be vetted by you with full transparency.

That is what I've been doing so far, but I think the boss is getting
close to just saying 'give it to them'...

But - no one has addressed my main question...

I understand that 301 redirects are performed by web servers only, you
can't really do these in DNS. However, some Managed DNS providers -
DNSMadeEasy included - offer this ability as a service. DNSMadeEasy
calls  them 'http redirects', and the actual redirect is accomplished by
one of their own web servers they have set up to handle these.

Is it 'normal' to do these 301 redirects at the DNS level like that? I
would think they should be using the current web server hosting the
current site to start doing the redirects as they get the new landing
pages done? Apache does this using a .htaccess file (if I'm interpreting
my googling responses correctly).

And now that I worded it that way - how would they do that exactly?
Would the proper method be to redirect it to a new test domain, ie:

www.example.com/page1.htm >> www.new-example.com/newpage1.htm ?

Or save the new page on the old server, then do:

www.example.com/page1.htm >> www.example.com/newpage1.htm ?

Now I'm confusing myself...

Re: [gentoo-user] Major site redesign, SEO, and 301 redirects

[Tanstaafl]

On 9/30/2015 3:36 AM, Mick  wrote:
> I couldn't agree more with all the warnings that have been posted.  However, 
> it may simply be that they want to build a new website and they want to 
> redirect your DNS from your currently hosted server to theirs.

You mean change the DNS servers at the Domain Registrar?

That would be even worse - they would need to completely reproduce
everything that is in there prior to transferring it, and then our DNS
is no longer ours.

> Are they offering SaaS, or will you be hosting the new website on
> prem?

That is one of my questions. Currently the site is hosted at Rackspace.

> In any case, they could just ask you to do this, if you agree. Given
> that "possession is nine-tenths of the law" I would not let them
> anywhere near your DNS records - period.

Hmmm... above it sounded like  you were ok with their desire to
'redirect our DNS from our currently hosted server to theirs'. Did I
misunderstand?

> With regards to being blacklisted by Google, you have to be careful indeed.  
> Google will blacklist bad code and malicious code.

At this point I'm more worried about bad links/hundreds of thousand
pages suddenly giving 404 errors, etc...

Re: [gentoo-user] Major site redesign, SEO, and 301 redirects

[Mick]

On Thursday 01 Oct 2015 13:21:48 Tanstaafl wrote:
> Thanks to Alan and the others for the responses...
> 
> The main problem is this project is being managed by a non-tech manager
> who apparently thinks they know a lot more than they do, and the Boss is
> technically challenged, so it is easy for someone to convince him of
> almost anything (like, he should delegate this to a non-tech person and
> not involve his one tech guy)...

This is uncanny!  Do you work in the same company as I?  O_o


> One reason he sometimes doesn't involve me until things get to this
> point is because I tend to be a 'wet blanket', ruining bright shiny
> sales pitches with injections of reality. You'd think he'd have learned
> by now. The last time, about 5 years ago, the person who managed the
> project (different person) didn't get ownership of the source code in
> the contract, so we didn't get all of the source files for the Flash
> junk they created, then when we wanted to make some changes to the text
> embedded in  the Flash, I had to ask them for the source files, and
> they wanted a bunch of money. Unbelievable.
> 
> We'll see how the dev(s) respond to my questions, but I may come back
> here with more info and more advice if I need it.

I bet they will ask to point the DNS record for your domain to their own (or 
Rackspace's) nameservers instead of your current nameservers.  However, as it 
has been commented already, the sane thing to do is develop the whole new 
website on their own subdomain with an appropriate robots.txt file, to stop 
spiders indexing it at this stage.  Once it is ready for UAT and assuming the 
boss approves it, they can ask *you* to point the DNS record to their 
Rackspace nameservers.

This way *you* can also point it back to a holding page/mirror/new site, when 
they no longer serve your needs.


> Thanks again to all, it has been a big help!

PS.  I hope someone will show them the door if they suggest designing a new 
Flash based web interface ...
-- 
Regards,
Mick


signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] Major site redesign, SEO, and 301 redirects

[Walter Dnes]

On Thu, Oct 01, 2015 at 03:35:48PM +0100, Mick wrote
> 
> PS.  I hope someone will show them the door if they suggest designing
> a new Flash based web interface ...

  Especially true given that Ipads/Iphones do not support Flash.  Another
major problem is websites that parse the user agent, and assume that any
browser they don't recognize is a mobile device.  It's a standing joke
amongst geeks... see https://xkcd.com/869/ and https://xkcd.com/1174/
Back in the day when "smartphones" only had 320x240 pixel screens, a
separate mobile site may have been necessary.  But not today with
pinch+zoom and smartphones/tablets with higher pixel counts than many
notebooks.  And with the idiots at Mozilla going off the deep end, there
are multiple forks of Firefox out there (Seamonkey, Palemoon, etc), by
people who are disgusted with Firefox's insanity.  A company that kicks
those browsers off their main website to a mobile site, or demands they
download an "app", will lose customers.

-- 
Walter Dnes 
I don't run "desktop environments"; I run useful applications

Re: [gentoo-user] Major site redesign, SEO, and 301 redirects

[Mick]

On Wednesday 30 Sep 2015 01:28:51 Alan McKinnon wrote:
> On 29/09/2015 22:00, Tanstaafl wrote:
> > Hi all,
> > 
> > I am not a web (or SEO) guy, but I manage our DNS and have for a long
> > time.
> > 
> > The boss has contracted with a web development company to do a full
> > redesign of our website.
> > 
> > Our website has hundreds of thousands of pages, and years of SEO behind
> > it. The guys who was her until recently was adamant that we must be very
> > carefl with the redesign so as not to totally break SEO, and possibly
> > getting blacklisted by Google.
> > 
> > The web developers are insisting that they need full access to our DNS
> > (hosted by DNSMadeEasy), and the only reason I can think of for this is
> > they plan on setting up HTTP redirects (DNSMadeEasy equivalent of a 301
> > redirect) for these pages - but hundreds of thousands of them?
> 
> I've been thinking about this some more.
> 
> We all assumed "full access" means "so we can change stuff". Maybe it
> really means they want to see what's in "dig axfr" (a zone transfer)
> which they normally can't see. There are TXT records in DNS that they
> might be interested in.
> 
> It would be wise to clarify with the devs exactly what it is they are
> looking for.
> 
> And overall, in your shoes I would be firm, adamant and above all polite
> and say that infrastructure changes go through you and you alone, and
> must be vetted by you with full transparency.
> 
> > Wouldn't this be better done at the web server level? Or am I just
> > ignorant?
> > 
> > Would love to hear experiences (good and bad), and a recommendation for
> > what I should do.
> > 
> > thanks

I couldn't agree more with all the warnings that have been posted.  However, 
it may simply be that they want to build a new website and they want to 
redirect your DNS from your currently hosted server to theirs.  Are they 
offering SaaS, or will you be hosting the new website on prem?  In any case, 
they could just ask you to do this, if you agree.  Given that "possession is 
nine-tenths of the law" I would not let them anywhere near your DNS records - 
period.

With regards to being blacklisted by Google, you have to be careful indeed.  
Google will blacklist bad code and malicious code.  If your code is clean, you 
don't fill your metadata with repetitive cr*ap and your topic is not faced 
with a competition of millions selling exactly the same undifferentiated 
product, then you should be OK in organic listing rankings.  Having mirrored 
websites on different DNS' will also blacklist you, although DNS or http 
redirects are of course legit.

A lot of so called SEO companies are not actually streamlining the content and 
metadata, but exploiting paid-for Google Ads and in a non-transparent way to 
milk the customer, on top of the Google charges.  Most of these companies set 
up Google Ads once and rarely if ever come back to to tune it.  I couldn't 
care to list the number of websites we switched off Google Ads and saw no 
discernible different in the rankings.

BTW, although SEO is not rocket science its not something you would leave to 
your marketing people alone, or for that matter to your coding people alone.  
You need both.  
-- 
Regards,
Mick


signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] Major site redesign, SEO, and 301 redirects

[Alan McKinnon]

On 29/09/2015 22:00, Tanstaafl wrote:
> Hi all,
> 
> I am not a web (or SEO) guy, but I manage our DNS and have for a long time.
> 
> The boss has contracted with a web development company to do a full
> redesign of our website.
> 
> Our website has hundreds of thousands of pages, and years of SEO behind
> it. The guys who was her until recently was adamant that we must be very
> carefl with the redesign so as not to totally break SEO, and possibly
> getting blacklisted by Google.
> 
> The web developers are insisting that they need full access to our DNS
> (hosted by DNSMadeEasy), and the only reason I can think of for this is
> they plan on setting up HTTP redirects (DNSMadeEasy equivalent of a 301
> redirect) for these pages - but hundreds of thousands of them?

I've been thinking about this some more.

We all assumed "full access" means "so we can change stuff". Maybe it
really means they want to see what's in "dig axfr" (a zone transfer)
which they normally can't see. There are TXT records in DNS that they
might be interested in.

It would be wise to clarify with the devs exactly what it is they are
looking for.

And overall, in your shoes I would be firm, adamant and above all polite
and say that infrastructure changes go through you and you alone, and
must be vetted by you with full transparency.



> 
> Wouldn't this be better done at the web server level? Or am I just ignorant?
> 
> Would love to hear experiences (good and bad), and a recommendation for
> what I should do.
> 
> thanks
> 


-- 
Alan McKinnon
alan.mckin...@gmail.com

Re: [gentoo-user] Major site redesign, SEO, and 301 redirects

[Alan McKinnon]

On 29/09/2015 22:19, J. Roeleveld wrote:
> On 29 September 2015 22:00:58 CEST, Tanstaafl  
> wrote:
>> Hi all,
>>
>> I am not a web (or SEO) guy, but I manage our DNS and have for a long
>> time.
>>
>> The boss has contracted with a web development company to do a full
>> redesign of our website.
> 
> Good luck with that. Hope you found a good company. :)
> 
>> Our website has hundreds of thousands of pages, and years of SEO behind
>> it. The guys who was her until recently was adamant that we must be
>> very
>> carefl with the redesign so as not to totally break SEO, and possibly
>> getting blacklisted by Google.
> 
> I never did anything with SEO. Would a mistake with that really get a site 
> blacklisted?
> 
>> The web developers are insisting that they need full access to our DNS
>> (hosted by DNSMadeEasy), and the only reason I can think of for this is
>> they plan on setting up HTTP redirects (DNSMadeEasy equivalent of a 301
>> redirect) for these pages - but hundreds of thousands of them?
> 
> Redirects with DNS?
> I can only think of adding subdomains (like about.example.com or similar)
> 
>> Wouldn't this be better done at the web server level? Or am I just
>> ignorant?
> 
> Page redirects are, afaik, only possible with a webserver. They are part of 
> the HTTP protocol. 
> 
>> Would love to hear experiences (good and bad), and a recommendation for
>> what I should do.
> 
> I would ask them what they actually want to achieve. Don't forget that your 
> email and all other services are dependent of the DNS settings.
> I can't think of many companies allowing a supplier for a website full access 
> to a different part of the infrastructure.
> 
> Most companies I deal with wouldn't even let the people responsible for the 
> databases to reconfigure the storage for said database directly.

I agree with Joost, needing access to all your DNS is off-the-wall. Any
changes they need done, and they will be few, can be given to you as a
support ticket for action just like everyone else gets to do.

I would also have them specify exactly in their proposal what they
intend to do, with full engineering. Any sane service provider will do
that in their tender, and yours looks like a rather big tender.

Lastly, get a second opinion of the changes they make. SEO tweaks can
very easily get you blacklisted on search engines and a lot of methods
out there are interpreted by Google as being dodgy.


-- 
Alan McKinnon
alan.mckin...@gmail.com

[gentoo-user] Major site redesign, SEO, and 301 redirects

[Tanstaafl]

Hi all,

I am not a web (or SEO) guy, but I manage our DNS and have for a long time.

The boss has contracted with a web development company to do a full
redesign of our website.

Our website has hundreds of thousands of pages, and years of SEO behind
it. The guys who was her until recently was adamant that we must be very
carefl with the redesign so as not to totally break SEO, and possibly
getting blacklisted by Google.

The web developers are insisting that they need full access to our DNS
(hosted by DNSMadeEasy), and the only reason I can think of for this is
they plan on setting up HTTP redirects (DNSMadeEasy equivalent of a 301
redirect) for these pages - but hundreds of thousands of them?

Wouldn't this be better done at the web server level? Or am I just ignorant?

Would love to hear experiences (good and bad), and a recommendation for
what I should do.

thanks

Re: [gentoo-user] Major site redesign, SEO, and 301 redirects

[J. Roeleveld]

On 29 September 2015 22:00:58 CEST, Tanstaafl  wrote:
>Hi all,
>
>I am not a web (or SEO) guy, but I manage our DNS and have for a long
>time.
>
>The boss has contracted with a web development company to do a full
>redesign of our website.

Good luck with that. Hope you found a good company. :)

>Our website has hundreds of thousands of pages, and years of SEO behind
>it. The guys who was her until recently was adamant that we must be
>very
>carefl with the redesign so as not to totally break SEO, and possibly
>getting blacklisted by Google.

I never did anything with SEO. Would a mistake with that really get a site 
blacklisted?

>The web developers are insisting that they need full access to our DNS
>(hosted by DNSMadeEasy), and the only reason I can think of for this is
>they plan on setting up HTTP redirects (DNSMadeEasy equivalent of a 301
>redirect) for these pages - but hundreds of thousands of them?

Redirects with DNS?
I can only think of adding subdomains (like about.example.com or similar)

>Wouldn't this be better done at the web server level? Or am I just
>ignorant?

Page redirects are, afaik, only possible with a webserver. They are part of the 
HTTP protocol. 

>Would love to hear experiences (good and bad), and a recommendation for
>what I should do.

I would ask them what they actually want to achieve. Don't forget that your 
email and all other services are dependent of the DNS settings.
I can't think of many companies allowing a supplier for a website full access 
to a different part of the infrastructure.

Most companies I deal with wouldn't even let the people responsible for the 
databases to reconfigure the storage for said database directly.

--
Joost 


-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.