Thanks, That's kinda the route I've already taken. On my site, www.startfound.com, if you click on any company to see more details it goes to a bookmarkable page. Same with any tag. Maybe if I've already got that much, I shouldn't concern myself with the fact that page 2 of my list is not bookmarkable but reachable by google bot. Or maybe I should just add a noindex meta tag on every page that's not page 1.
It'd be kinda ridiculous to require login to see past page 1. That may be good for SEO but it'll drive people away. > -----Original Message----- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On > Behalf Of Jeremy Thomerson > Sent: Thursday, April 03, 2008 10:00 PM > To: users@wicket.apache.org > Subject: Re: Removing the jsessionid for SEO > > I've been building a community-driven hunting and fishing site in Texas > for > the past year and a half. Since I have converted it to Wicket from > ColdFusion, our search engine rankings have gone WAY UP. That's right, > we're on the first page for tons of searches. Search for "texas hunting" > - > we're second under only the Texas Parks and Wildlife Association. > > How? With Wicket? Yes - it requires a little more work. What I do is > that > for any link that I want Google to be able to follow, I have a subclass of > Link specific to that. For instance, ViewThreadLink, which takes the ID > for > the link and a model (detachable) of the thread. Then I mount an > IRequestTargetUrlCodingStrategy for each big category of things in my > webapp. I've made several strategies that I use over and over, just > giving > them a different mount path and a different parameter to tell it what kind > of article, etc, that it will match to. This is made easier because over > 75% of the objects in our site are all similar enough that the extend from > a > base class that provides the basic functionality for an article / thread / > etc that has a title, text, pictures, comments, the standard stuff. > > So, yes, it takes work. But that's okay - SEO always takes work. I also > have given a lot of care to use good page titles, good semantic HTML and > stuff things into the URL that don't have anything to do with locating the > resource, but give the search engines a clue as to what the content is. > > Yes, some pages end up with a jsessionid - and I don't like it (example: > http://www.google.com/search?hl=en&client=firefox-a&rls=com.ubuntu%3Aen- > US%3Aofficial&q=%22south+texas+management+buck%22&btnG=Search). > But, most don't because almost all of my links are bookmarkable. When the > user clicks something that they can only do as a signed-in user, then it > redirects them to the sign in page, they sign in, and are taken back to > the > page they were on. Then they can pick up, and I don't worry about > bookmarkable URLs for anything that requires user-authentication (wizards > to > post a new listing, story, admin links, etc). > > Jeremy Thomerson > TexasHuntFish.com > > On Thu, Apr 3, 2008 at 8:09 PM, Dan Kaplan <[EMAIL PROTECTED]> > wrote: > > > Ok, at least I'm not missing anything. I understand the benefits it's > > providing with its stateful framework. Developing a site with Wicket is > > easier than with any other framework I've used. But this statefulness, > > which makes websites so easy to develop, seems to be counter productive > to > > SEO: > > > > GoogleBot will follow and index stateful links. Worst case scenario, > > these > > actually become visible to google users and when they click the link it > > takes them to an "invalid session" page. They think, "This site is > > broken" > > and move on to the next link of their search result. > > > > Another approach to solving this is to block all the stateful pages in > my > > robots.txt file. But how can I block these links in robots.txt since > they > > change per session? Is there any way to know what the url will resolve > to > > when googlebot tries to visit my site so I can tell it to disallow: > > /?wicket:interface=:10:1::: and ?wicket:interface=:0:1::: and ...? > > > > > > > -----Original Message----- > > > From: Igor Vaynberg [mailto:[EMAIL PROTECTED] > > > Sent: Thursday, April 03, 2008 5:45 PM > > > To: users@wicket.apache.org > > > Subject: Re: Removing the jsessionid for SEO > > > > > > On Thu, Apr 3, 2008 at 5:31 PM, Dan Kaplan <[EMAIL PROTECTED]> > > > wrote: > > > > Ok I did a little preliminary research on this. Right now > > > PagingNavigator > > > > uses PagingNavigationLink's to represent its page. This extends > > Link. > > > I'm > > > > supposed to override PagingNavigator's newPagingNavigationLink() > > method > > > to > > > > accomplish this (I think) but past that, this isn't very > > > straightforward to > > > > me. > > > > > > > > Do I need to create my own BookmarkablePagingNavigationLink? When > I > > > do... > > > > what next? I really don't know enough about bookmarkablePageLinks > to > > > do > > > > this. Right now, all the magic happens inside > PagingNavigationLink. > > > Won't > > > > I have to move all that logic into the WebPage that I'm passing > into > > > > BookmarkablePagingNavigationLink? This seems like a lot of work. > Am > > I > > > > missing something critical? > > > > > > no, you are not missing anything. you see, when you go stateless, like > > > what you want, then you have to recreate all the magic stuff that > > > makes stateful links Just Work. Without state you are back to the > > > servlet/mvc programming model: you have to encode the state that you > > > want into the link, then on the trip back decode it, recreate > > > something from it, and then apply that something onto the components. > > > This is the crapwork that wicket does for you usually. > > > > > > -igor > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > From: Igor Vaynberg [mailto:[EMAIL PROTECTED] > > > > > > > > > > > > > Sent: Thursday, April 03, 2008 3:40 PM > > > > > To: users@wicket.apache.org > > > > > Subject: Re: Removing the jsessionid for SEO > > > > > > > > > > you subclass the pagenavigator and make it use bookmarkable links > > > > > also. it has factory methods for all the links it uses. > > > > > > > > > > -igor > > > > > > > > > > > > > > > On Thu, Apr 3, 2008 at 3:36 PM, Dan Kaplan > <[EMAIL PROTECTED] > > > > > > > > wrote: > > > > > > I wasn't talking about the links that are on the list (I > already > > > make > > > > > those > > > > > > bookmarkable). I'm talking about the links that the Navigator > > > > > generates. > > > > > > How do I make it so page 2 is bookmarkable? > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > From: Igor Vaynberg [mailto:[EMAIL PROTECTED] > > > > > > > > > > > > > > > > > > Sent: Thursday, April 03, 2008 3:30 PM > > > > > > To: users@wicket.apache.org > > > > > > Subject: Re: Removing the jsessionid for SEO > > > > > > > > > > > > instead of > > > > > > > > > > > > item.add(new link("foo") { onclick() }); > > > > > > > > > > > > do > > > > > > > > > > > > item.add(new bookmarkablepagelink("foo", page.class)); > > > > > > > > > > > > -igor > > > > > > > > > > > > > > > > > > On Thu, Apr 3, 2008 at 3:28 PM, Dan Kaplan > > > <[EMAIL PROTECTED]> > > > > > wrote: > > > > > > > How? I asked how to do it before and nobody suggested this > as > > a > > > > > > > possibility. > > > > > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: Igor Vaynberg [mailto:[EMAIL PROTECTED] > > > > > > > Sent: Thursday, April 03, 2008 3:26 PM > > > > > > > To: users@wicket.apache.org > > > > > > > Subject: Re: Removing the jsessionid for SEO > > > > > > > > > > > > > > dataview can work in a stateless mode, just use > bookmarkable > > > links > > > > > inside > > > > > > it > > > > > > > > > > > > > > -igor > > > > > > > > > > > > > > > > > > > > > On Thu, Apr 3, 2008 at 3:22 PM, Dan Kaplan > > > <[EMAIL PROTECTED]> > > > > > > wrote: > > > > > > > > Regardless, at the very least this makes your site look > > > "weird" > > > > > and > > > > > > > > unprofessional when google puts a jsessionid on your > url. > > > There > > > > > has > > > > > > got > > > > > > > to > > > > > > > > be some negative effect when google visits it the second > > > time and > > > > > the > > > > > > > > jsessionid has changed but it sees the same exact > content. > > > Worst > > > > > > case, > > > > > > > > it'll think you're trying to trick it. > > > > > > > > > > > > > > > > About those 404s, I'm finding that with the fix I > provided > > I > > > > > don't get > > > > > > a > > > > > > > > 404, but the links refresh the page I'm already on. IE: > > If > > > I'm > > > > > on A, > > > > > > and > > > > > > > a > > > > > > > > link to B is non-bookmarkable, clicking B refreshes A. > > > > > > > > > > > > > > > > This issue is very disconcerting to me. It's one of the > > > reasons > > > > > I > > > > > > wish > > > > > > > that > > > > > > > > DataView had an option to work in stateless mode. Cause > > if > > > I ban > > > > > > cookies > > > > > > > > and Googlebot visits my home page (with a navigator on > > it), > > > it'll > > > > > try > > > > > > to > > > > > > > > follow all these page links and from its perspective, > they > > > all > > > > > lead > > > > > > back > > > > > > > to > > > > > > > > the first page. So it's kinda a catch-22: Include the > > > jsessionid > > > > > in > > > > > > the > > > > > > > > urls and get bad SEO or remove the jsessionid and get > bad > > > SEO :( > > > > > > > > > > > > > > > > Perhaps the answer to my prayers is a combination of the > > > > > > noindex/nofollow > > > > > > > > meta tag with a sitemap.xml. I'm thinking I can put a > > > nofollow > > > > > on the > > > > > > > home > > > > > > > > page (so googlebot doesn't try to follow the navigator > > > links) and > > > > > use > > > > > > the > > > > > > > > sitemap.xml to point out the individual pages I want it > to > > > index. > > > > > > > > > > > > > > > > > > > > > > > > Matej: can you go into more detail about your hybrid URL > > > > > statement? > > > > > > > Won't > > > > > > > > google index, for example, /home and /home.1 if I use > it? > > > When > > > > > it > > > > > > > follows > > > > > > > > the next page, won't the url become /home.1.2 or > > something? > > > That > > > > > .2 > > > > > > is a > > > > > > > > page version: If google indexes that and tries to visit > it > > > again, > > > > > > won't > > > > > > > it > > > > > > > > report about an invalid session? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: Matej Knopp [mailto:[EMAIL PROTECTED] > > > > > > > > Sent: Thursday, April 03, 2008 11:10 AM > > > > > > > > To: users@wicket.apache.org > > > > > > > > Subject: Re: Removing the jsessionid for SEO > > > > > > > > > > > > > > > > On the other hand, crawling non-bookmarkable pages is > not > > > very > > > > > useful > > > > > > > > anyway, since ?wicket:interface url will always get page > > > expired > > > > > when > > > > > > > > you click on the result. > > > > > > > > > > > > > > > > However, preserving session makes lot of sense with > hybrid > > > url. > > > > > Google > > > > > > > > remembers the original url (without page instance) while > > > indexing > > > > > the > > > > > > > > real page (after redirect). > > > > > > > > > > > > > > > > I think though that the crawler is quite advanced. I'm > > would > > > > > think it > > > > > > > > supports cookies (at least JSESSIONID) as well as it > > > evaluates > > > > > some of > > > > > > > > the javascript on page. > > > > > > > > > > > > > > > > -Matej > > > > > > > > > > > > > > > > On Thu, Apr 3, 2008 at 6:56 PM, Igor Vaynberg > > > > > > <[EMAIL PROTECTED]> > > > > > > > > wrote: > > > > > > > > > right. if you strip sessionid then all your > > > nonbookmarkable > > > > > urls > > > > > > will > > > > > > > > > resolve to a 404. that will probably drop your rank a > > lot > > > > > > faster.... > > > > > > > > > > > > > > > > > > -igor > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Apr 3, 2008 at 9:16 AM, Johan Compagner > > > > > > <[EMAIL PROTECTED]> > > > > > > > > wrote: > > > > > > > > > > the problem is that then you have to have all > > stateless > > > > > pages. > > > > > > Else > > > > > > > > google > > > > > > > > > > can't crawl your website. > > > > > > > > > > And if that is the case then you could be > completely > > > > > stateless > > > > > > so > > > > > > > you > > > > > > > > dont > > > > > > > > > > have a session (id) to worry about at all. > > > > > > > > > > > > > > > > > > > > johan > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Apr 3, 2008 at 4:54 PM, Zappaterrini, > Larry > > < > > > > > > > > > > [EMAIL PROTECTED]> wrote: > > > > > > > > > > > > > > > > > > > > > When Google asks to not have special treatment > for > > > their > > > > > bot, > > > > > > > they > > > > > > > > are > > > > > > > > > > > referring to content more than anything. > Regarding > > > the > > > > > session > > > > > > id > > > > > > > > being > > > > > > > > > > > coded in the URL, see the Technical guidelines > > > section of > > > > > > > Google's > > > > > > > > > > > Webmaster Guidelines - > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://www.google.com/support/webmasters/bin/answer.py?answer=35769#desi > > > > > > > > > > > gn > > > > > > > > > > > > > > > > > > > > > > It specifically recommends "allow(ing) search > bots > > > to > > > > > crawl > > > > > > your > > > > > > > > sites > > > > > > > > > > > without session IDs or arguments that track > their > > > path > > > > > through > > > > > > > the > > > > > > > > > > > site." > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > > From: Johan Compagner > [mailto:[EMAIL PROTECTED] > > ] > > > > > > > > > > > Sent: Thursday, April 03, 2008 7:35 AM > > > > > > > > > > > To: users@wicket.apache.org > > > > > > > > > > > Subject: Re: Removing the jsessionid for SEO > > > > > > > > > > > > > > > > > > > > > > isnt google always saying that you shouldn't > alter > > > > > behavior of > > > > > > > your > > > > > > > > site > > > > > > > > > > > depending of it is there bot or not? > > > > > > > > > > > > > > > > > > > > > > On Thu, Apr 3, 2008 at 1:00 PM, Artur W. > > > > > <[EMAIL PROTECTED]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi! > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > igor.vaynberg wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > also by doing what you have done users with > > > cookies > > > > > > disabled > > > > > > > > wont be > > > > > > > > > > > > > able to use your site... > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In my opinion session id is a problem. Google > > > index the > > > > > same > > > > > > > page > > > > > > > > > > > again > > > > > > > > > > > > and > > > > > > > > > > > > again. > > > > > > > > > > > > > > > > > > > > > > > > About the users without cookies we can do like > > > this: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > static class Unbuffered extends > > WebResponse > > > { > > > > > > > > > > > > > > > > > > > > > > > > private static final String[] > > > botAgents > > > > > = { > > > > > > > > > > > "onetszukaj", > > > > > > > > > > > > "googlebot", > > > > > > > > > > > > "appie", "architext", > > > > > > > > > > > > "jeeves", "bjaaland", > > > "ferret", > > > > > > > "gulliver", > > > > > > > > > > > > "harvest", "htdig", > > > > > > > > > > > > "linkwalker", "lycos_", > > > "moget", > > > > > > > > > > > "muscatferret", > > > > > > > > > > > > "myweb", "nomad", > > > > > > > > > > > > "scooter", > > > > > > > > > > > > > "yahoo!\\sslurp\\schina", > > > > > "slurp", > > > > > > > > "weblayers", > > > > > > > > > > > > "antibot", "bruinbot", > > > > > > > > > > > > "digout4u", > > > > > > > > > > > > "echo!", "ia_archiver", > > > > > "jennybot", > > > > > > > > "mercator", > > > > > > > > > > > > "netcraft", "msnbot", > > > > > > > > > > > > "petersnews", > > > > > > > > > > > > "unlost_web_crawler", > > > "voila", > > > > > > > "webbase", > > > > > > > > > > > > "webcollage", "cfetch", > > > > > > > > > > > > "zyborg", > > > > > > > > > > > > "wisenutbot", "robot", > > > "crawl", > > > > > > "spider" > > > > > > > }; > > > > > > > > /* > > > > > > > > > > > and > > > > > > > > > > > > so on... */ > > > > > > > > > > > > > > > > > > > > > > > > public Unbuffered(final > > > > > HttpServletResponse > > > > > > res) > > > > > > > { > > > > > > > > > > > > super(res); > > > > > > > > > > > > } > > > > > > > > > > > > > > > > > > > > > > > > @Override > > > > > > > > > > > > public CharSequence encodeURL(final > > > CharSequence > > > > > url) > > > > > > { > > > > > > > > > > > > return isAgent() ? url : > > > > > super.encodeURL(url); > > > > > > > > > > > > } > > > > > > > > > > > > > > > > > > > > > > > > private static boolean > isAgent() > > { > > > > > > > > > > > > > > > > > > > > > > > > String agent = > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ((WebRequest)RequestCycle.get().getRequest()).getHttpServletRequest().ge > > > > > > > > > > > tHeader("User-Agent"); > > > > > > > > > > > > > > > > > > > > > > > > for(String bot : > > botAgents) > > > { > > > > > > > > > > > > if > > > > > > > > (agent.toLowerCase().indexOf(bot) != > > > > > > > > > > > -1) > > > > > > > > > > > > { > > > > > > > > > > > > return > > > true; > > > > > > > > > > > > } > > > > > > > > > > > > } > > > > > > > > > > > > > > > > > > > > > > > > return false; > > > > > > > > > > > > } > > > > > > > > > > > > } > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I didn't test this code but I do similar thing > > in > > > my > > > > > old > > > > > > > > application > > > > > > > > > > > in > > > > > > > > > > > > Spring and it works. > > > > > > > > > > > > > > > > > > > > > > > > Take care, > > > > > > > > > > > > Artur > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > View this message in context: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://www.nabble.com/Removing-the-jsessionid-for-SEO- > > > > > tp16464534p1646739 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 6.html<http://www.nabble.com/Removing-the-jsessionid-for-SEO- > > > > > tp16464534p1646 > > > > > > > > 7396.html> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Sent from the Wicket - User mailing list > archive > > at > > > > > > Nabble.com. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------ > > > ----- > > > > > ---- > > > > > > > > > > > > To unsubscribe, e-mail: users- > > > > > [EMAIL PROTECTED] > > > > > > > > > > > > For additional commands, e-mail: > > > > > > [EMAIL PROTECTED] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ______________ > > > > > > > > > > > > > > > > > > > > > > The information contained in this message is > > > proprietary > > > > > > and/or > > > > > > > > > > > confidential. If you are not the > > > > > > > > > > > intended recipient, please: (i) delete the > message > > > and > > > > > all > > > > > > > copies; > > > > > > > > (ii) do > > > > > > > > > > > not disclose, > > > > > > > > > > > distribute or use the message in any manner; and > > > (iii) > > > > > notify > > > > > > the > > > > > > > > sender > > > > > > > > > > > immediately. In addition, > > > > > > > > > > > please be aware that any message addressed to > our > > > domain > > > > > is > > > > > > > subject > > > > > > > > to > > > > > > > > > > > archiving and review by > > > > > > > > > > > persons other than the intended recipient. Thank > > > you. > > > > > > > > > > > _____________ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------ > > > ----- > > > > > ---- > > > > > > > > > > > To unsubscribe, e-mail: users- > > > > > [EMAIL PROTECTED] > > > > > > > > > > > For additional commands, e-mail: users- > > > > > [EMAIL PROTECTED] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------ > > > --- > > > > > > > > > To unsubscribe, e-mail: users- > > > [EMAIL PROTECTED] > > > > > > > > > For additional commands, e-mail: users- > > > [EMAIL PROTECTED] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Resizable and reorderable grid components. > > > > > > > > http://www.inmethod.com > > > > > > > > > > > > > > > > > > ------------------------------------------------------------ > > > ----- > > > > > ---- > > > > > > > > To unsubscribe, e-mail: > > [EMAIL PROTECTED] > > > > > > > > For additional commands, e-mail: users- > > > [EMAIL PROTECTED] > > > > > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------ > > > ----- > > > > > ---- > > > > > > > > To unsubscribe, e-mail: > > [EMAIL PROTECTED] > > > > > > > > For additional commands, e-mail: users- > > > [EMAIL PROTECTED] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------- > > > ----- > > > > > - > > > > > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > > > > > For additional commands, e-mail: users- > [EMAIL PROTECTED] > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------- > > > ----- > > > > > - > > > > > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > > > > > For additional commands, e-mail: users- > [EMAIL PROTECTED] > > > > > > > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------ > > > --- > > > > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------ > > > --- > > > > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > > > > ------------------------------------------------------------------- > -- > > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
