New SEO Tools: Robots.txt Builder, Term Targeting & Crawl Test

Three new SEO tools came out this week that are worth a look. Dave Naylor released a robots.txt builder tool, while SEOmoz released term targeting and crawl test tools. Some more about the tools, below: The Robots.txt Builder Tool David Naylor, the creator of the tool, explains that one of his clients ...
Related Searches: Robots.txt | Seo Tools | Tools

Apr. 20, 2007 at 12:30pm Eastern by Barry Schwartz

Up Close & Personal With Robots.txt

The Robots.txt Summit at Search Engine Strategies New York 2007 was the latest in a series of special sessions with the intent to open a dialog between search engines representatives and web site publishers. Past summits featured discussion on comment spam on blogs, indexing issues and redirects. The ...
Related Searches: Robots.txt | Sitemap | Xml Sitemap

Apr. 16, 2007 at 1:15pm Eastern by Christine Churchill

Search Illustrated: Blocking Search Engines With Robots.txt

... only gets one copy (the "canonical" version, in search engine parlance). Today's Search Illustrated illustrates how you can use the "robots.txt" file as a "keep out" notice for search engine cawlers: Graphic by Elliance, an eMarketing firm specializing in results-driven ...
Related Searches: Robots.txt File | Robots.txt | Canonical

May. 23, 2007 at 3:25am Eastern by Elliance

Meta Keywords Tag 101: How To "Legally" Hide Words On Your Pages For Search Engines

... on topic when writing for the page. And yes, we've been arguing about commas versus no commas for a decade. I am in the no comma camp because if the robots read and/or ever use the tag we believe it allows them to form whatever "terms" they want to out of the word jumble - over, under, sideways ...
Related Searches: Meta Keywords | Metatag Keywords | Meta Tags | Meta

Sep. 5, 2007 at 7:42pm Eastern by Danny Sullivan

ACAP Launches, Robots.txt 2.0 For Blocking Search Engines?

After a year of discussions, ACAP -- Automated Content Access Protocol -- was released today as a sort of robots.txt 2.0 system for telling search engines what they can or can't include in their listings. However, none of the major search engines support ACAP, and its future remains firmly one of ...
Related Searches: Acap

Nov. 29, 2007 at 12:02pm Eastern by Danny Sullivan

SEOs Want The NOINDEX Tag To Not Show A Page In The Index

... Explains The NOINDEX, NOFOLLOW, NOARCHIVE & NOSNIPPET META Tags from last year has more about the various meta commands Google allows, and Meta Robots Tag 101: Blocking Spiders, Cached Pages & More goes into depth about how each feature is used for particular search engines.
Related Searches: Noindex

Feb. 25, 2008 at 8:53am Eastern by Barry Schwartz

Robots.txt Study Shows Webmasters Favor Google; BotSeer Robots.txt Search Engine Released

... search engines in terms of allowing access to their web sites. An associated BotSeer search engine that allows searching across a collection of robots.txt files was also released. The study looked at which robots or crawlers were listed in a web site's robots.txt file, and Google was listed more often ...
Related Searches: Robots.txt

Nov. 15, 2007 at 1:19pm Eastern by Barry Schwartz

Irony: If Google Can't Reach Your Robots.txt File, It Might Not List Your Site

I reported at the Search Engine Roundtable this morning that Google said if your robots.txt is unreachable, your site might not make it into the Google index. By unreachable, Google means that if your server simply times out and does not return any server response when Googlebot attempts to access ...
Related Searches: Robots.txt

Jun. 18, 2008 at 8:34am Eastern by Barry Schwartz

Yahoo Search Weather Update & Support For X-Robots Tag

The Yahoo Blog issued a weather report for changes to rankings in Yahoo Search, along with news that they are now supporting the X-Robots-Tag directive -- a way to control indexing of content that cannot accept meta robots tags. Google also supports X-Robots, which gives webmasters the ability to define ...

Dec. 5, 2007 at 2:08pm Eastern by Barry Schwartz

Yahoo!, Google, Microsoft Clarify Robots.txt Support

Today, Google, Yahoo!, and Microsoft have come together to post details of how each of them support robots.txt and the robots meta tag. While their posts use terms like "collaboration" and "working together," they haven't joined together to implement a new standard (as they did ...
Related Searches: Acap

Jun. 3, 2008 at 12:11pm Eastern by Vanessa Fox