feed hat blog

I know you’re used to getting your recap on Fridays, but tomorrow’s our Halloween party! That translates to the entire BCI crew in spirited costumes having wicked good fun. What can I say? We love to dress up!

So there’s a good chance tomorrow will be a pretty hectic day, and since I plan to be among the undead, I can’t really count on much brain capacity. I figured why not serve up some recap while I’ve got the necessary presence of mind?

The Internet commemorates its 40th birthday today. Kind of. There are lots of milestones that could be counted as the “birth” of the Internet but the one that happened forty years ago today is the connection of to nodes of ARPANET. It’s like a big deal or something. Yay, Internet! [Officially Over the Hill. --Susan]

Turns out spending a couple hours a day online may actually be good for your mental health. A study suggests that the onset of dementia may be prevented or slowed by the brain stimulation caused by surfing the Web. If that’s the case, in sixty years there will be a whole bunch of sharp, old SEOs out there still doing their thing!

Or course, every tool that can be used for good can be just as easily used for trickery. This week a slew of celebrity robberies were blamed in part on the Internet.

A group of teens (who went to school in my home town, actually — go Chargers!) are suspected of burglarizing a slew of homes belonging to young Hollywood stars. Police say the thieves used celebrity Web sites to track the celebrities’ whereabouts, providing the burglars easy access to empty homes and all the cool stuff therein. Spooooky! [Dear horrible people on the Internet. I never leave home and I have a shotgun. Love and kisses, Susan]

One of the suspects blamed her criminal activity on bad friends. Now that Google’s testing out the social space, it may be a little easier to stay connected with friends — hopefully just the good ones. Social Search launched in Google Labs this week and a bald-headed Matt Cutts has the honor of the intros.

News around Yahoo! was slightly less celebratory as Geocities said its goodbyes, taking a slew of personal, free Web sites with it. Bing, meanwhile, got into the holiday spirit with Halloween-themed ads. Together, the pair postponed their self-imposed search deal deadline citing the “complex nature of the transaction”.

Members of the search industry obviously keep up with the big three engines, but have you ever considered your alternatives? Over at Search Engine Watch Carrie Hill rounds up and categorizes AdWords alternatives, including PPC, cost-per-impression and cost-per-lead ad programs.

Directories have been touted as a potential source for inbound links. But one of the oldest and most authoritative directories, DMOZ, is notoriously difficult to get into. Seeking to uncover this thorn in the SEO industry’s side, Debra Mastaler talked to the Open Directory Project’s editor in chief in a revealing interview posted on Search Engine Land.

Finally, as October rolls to a close, so does National Breast Cancer Awareness Month. But the need for care and giving carries on. The search and social media marketers are supporting the cause through TwakeUp, a social media campaign in which all money raised will be given to the National Breast Cancer Foundation. You can make donations to this campaign through November.

Things I learned from Boing Boing this week:

• One pro photographer has shared his tips for taking incredible pics with nothing more than a li’l ol’ iPhone 3GS.
• The White House went open source this week.
• Bees may have developed a taste for human tears. [AUGH! --Susan]
• Disney’s Haunted Mansion has real-world roots!

Original post:
Friday Recap - Thursday Edition

I wrote a post yesterday about the dangers of allowing (some) custom shortened URLs. Really though, I only talked about one particular name on one particular shortener, so I’ll try to make this a little more general. Incidentally, bit.ly appear to have fixed their robots.txt as of a couple of hours ago – they obviously thought it was a problem!

There were a couple of reasons I was pretty sure about Google following the redirect on robots.txt – the main one being Google’s own Webmaster Tools! If you believe their own tool, Google were quite happily following the redirect and trying to parse a blog as a robots.txt. Depending on the random chance I mentioned yesterday, this sometimes came up with the “other” robots.txt.

This brings me neatly onto the next potential problem – verifying your URL shortener in Google Webmaster Tools or Yahoo Site Explorer.

Webmaster Tools

One of the metrics Google Webmaster Tools uses to decide if you have control over a particular site is for you to place an empty file with a specified name on your site. What happens if I try to add bit.ly to my Webmaster Tools account?

It asks me to create http://bit.ly/google4516834ef16f6fa21.html (or thereabouts)

Now, bit.ly won’t let me create a custom name with a dot in, but until this morning they would silently strip them out.

So, one could just make http://bit.ly/google4516834ef16f6fa21html instead (note the lack of a dot). When Google come along and try to verify the existence of the file, they will do so with the dot in there; bit.ly silently dropped it, redirected to somewhere – anywhere, it doesn’t matter – and the account is marked as verified. Considering there are some fairly odd looking removal requests in there, I don’t think I’m the first person to think of it.

When checking of the specially named file, Google also look for a file they are not expecting to find. If it doesn’t return an error code, they reject the verification. This is due to many misconfigured sites (including URL shorteners!) that fail to return a 404 error code for paths that do not exist. This happens to stop the procedure here from working on several other shorteners that I tried, but luckily for the purposes of this article, Yahoo Site Explorer isn’t so picky.

Site Explorer

Considering bit.ly seem to have fixed their problems, let’s pick on a different target. How about tr.im? They’re another big player for several reasons, the biggest of which is that now they’re open source there are dozens of clones out there. This works on all of them that I tried.

To verify a URL for Site Explorer, you can create a file in a similar manner to the Google one above.

For tr.im, Yahoo asked me to create: http://tr.im/y_key_a41e58c3dc1e462a.html (or again, close enough!)

They don’t allow dots in their short names either, nor is the box big enough to enter the whole string. Unfortunately, they will ignore any extension added to the URL, so creating a short name of y_key_a41e58c3dc1e462a is all that is required for the requested link to work.

Rather than relying on your 404 reporting to be working, Yahoo require that a particular string appears in the content of the file they request; that way they can be sure it’s not an error page and you have control over the content. In my case this looked like fc140e122451a103. As you will probably have guessed by now, they will follow a redirect before looking for that string. Make the destination URL for your short link point to a page containing that value (like, say, this one!) and Yahoo will verify the site as your own.

Summary

No, your URL shortener is not going to get hacked, the world is not going to end, and these particular issues are easy to avoid. Having said that, please think carefully about how your own URLs can be used against you.

Is there any good reason for Google and Yahoo to follow redirects when verifying ownership of a site? Removing this would seem to be a good idea to me. I suspect following redirects on robots.txt is a more complicated issue but perhaps some restrictions could be considered there. Search engines currently have an uneasy relationship with URL shorteners and I think that will only continue.

Thanks for all the comments.

James

Read the original here:
Don’t Make The Same Mistakes As Bit.ly and Tr.im

I wrote a post yesterday about the dangers of allowing (some) custom shortened URLs. Really though, I only talked about one particular name on one particular shortener, so I’ll try to make this a little more general. Incidentally, bit.ly appear to have fixed their robots.txt as of a couple of hours ago – they obviously thought it was a problem!

There were a couple of reasons I was pretty sure about Google following the redirect on robots.txt – the main one being Google’s own Webmaster Tools! If you believe their own tool, Google were quite happily following the redirect and trying to parse a blog as a robots.txt. Depending on the random chance I mentioned yesterday, this sometimes came up with the “other” robots.txt.

This brings me neatly onto the next potential problem – verifying your URL shortener in Google Webmaster Tools or Yahoo Site Explorer.

Webmaster Tools

One of the metrics Google Webmaster Tools uses to decide if you have control over a particular site is for you to place an empty file with a specified name on your site. What happens if I try to add bit.ly to my Webmaster Tools account?

It asks me to create http://bit.ly/google4516834ef16f6fa21.html (or thereabouts)

Now, bit.ly won’t let me create a custom name with a dot in, but until this morning they would silently strip them out.

So, one could just make http://bit.ly/google4516834ef16f6fa21html instead (note the lack of a dot). When Google come along and try to verify the existence of the file, they will do so with the dot in there; bit.ly silently dropped it, redirected to somewhere – anywhere, it doesn’t matter – and the account is marked as verified. Considering there are some fairly odd looking removal requests in there, I don’t think I’m the first person to think of it.

When checking of the specially named file, Google also look for a file they are not expecting to find. If it doesn’t return an error code, they reject the verification. This is due to many misconfigured sites (including URL shorteners!) that fail to return a 404 error code for paths that do not exist. This happens to stop the procedure here from working on several other shorteners that I tried, but luckily for the purposes of this article, Yahoo Site Explorer isn’t so picky.

Site Explorer

Considering bit.ly seem to have fixed their problems, let’s pick on a different target. How about tr.im? They’re another big player for several reasons, the biggest of which is that now they’re open source there are dozens of clones out there. This works on all of them that I tried.

To verify a URL for Site Explorer, you can create a file in a similar manner to the Google one above.

For tr.im, Yahoo asked me to create: http://tr.im/y_key_a41e58c3dc1e462a.html (or again, close enough!)

They don’t allow dots in their short names either, nor is the box big enough to enter the whole string. Unfortunately, they will ignore any extension added to the URL, so creating a short name of y_key_a41e58c3dc1e462a is all that is required for the requested link to work.

Rather than relying on your 404 reporting to be working, Yahoo require that a particular string appears in the content of the file they request; that way they can be sure it’s not an error page and you have control over the content. In my case this looked like fc140e122451a103. As you will probably have guessed by now, they will follow a redirect before looking for that string. Make the destination URL for your short link point to a page containing that value (like, say, this one!) and Yahoo will verify the site as your own.

Summary

No, your URL shortener is not going to get hacked, the world is not going to end, and these particular issues are easy to avoid. Having said that, please think carefully about how your own URLs can be used against you.

Is there any good reason for Google and Yahoo to follow redirects when verifying ownership of a site? Removing this would seem to be a good idea to me. I suspect following redirects on robots.txt is a more complicated issue but perhaps some restrictions could be considered there. Search engines currently have an uneasy relationship with URL shorteners and I think that will only continue.

Thanks for all the comments.

James

Original post:
Don’t Make The Same Mistakes As Bit.ly and Tr.im

In the last week I’ve had 3 emails from makers of Automated Blogging tools, scripts and plugins (or RSS to Blog tools) asking me to promote their plugins and systems.

These tools all claim to be able to help you create content for your blog without you having to do anything except set it up, choose a keyword/s for your blog to be about.

The tools sales pages usually make claims like:

• “create targeted blog posts on any topic without writing anything!”
• “start hundreds of blogs on any topic and never have to lift a finger to keep them pumping out as much content as you like!”
• “generate traffic, money and blog posts while you sleep!”
• “Achieve Higher Search Engine Rankings And Massive Affiliate Revenue With Self Updating Blogs”

You get the picture - the list of the hyped up claims that the developers of automated blogging tools make goes on and on!

The fact that these people are asking me to promote these kinds of tools scares me a little as I’ve been pretty anti them in the past and don’t want to be associated with the in any way.

However it also makes me wonder how many bloggers are innocently signing up for them without knowing the dangers of doing so. After-all the sales copy on many of these tools sounds too good to be true - blogging made easy, lots of money, no work….

As a result I thought I’d put together a list of reasons why I would avoid ‘auto blogging’ tools at all costs.

Reasons to avoid Automated blogging Tools and Services:

1. Non Unique Content - at the heart of every successful and profitable blog that I’ve come across is unique content. Auto blogging tools all take content from other places on the web and automatically pull them together on your blog. They replicate what others are doing. They create duplicates of other people’s work. It’s not unique, it’s not original and it creates clutter.

Many of the automated blog tools sales pages say you can add to the content that these auto blogging tools use to add uniqueness to your blog but I’d argue that if you’re creating hundreds of blogs it’s pretty unlikely that you’ll be adding unique posts to many of them.

Blogs that are not unique, that don’t have a personal voice, that contain no original thought don’t tend to get links from other blogs, don’t tend to attract subscribers, readers or comments and don’t generally rank well in Google or other search engines.

2. Useless Content - the other main factor in successful blogs is that they create ‘useful’ content - the type of content that solves people’s problems, helps them solve a problem and makes their lives better in some way.

While some might argue that automated blogging tools can help people by finding this type of information my observation of most of them in action is that they are very hit and miss. Most rely upon you identifying keywords that you want your blog to be about and they then go searching for all kinds of content on those keywords.

As a result you can be publishing who knows what on your blog. Some of it may be useful but some of it might be completely irrelevant and even potentially harmful to readers. Many automated blogs that I come across are a step up from being ‘gibberish’.

3. Personal Satisfaction - early in my own blogging I created a number of blogs that I called ‘link blogs’. They looked at what others were writing online and manually (no tools) collated some of it onto one site. I added some of my own thoughts and it did provide usefulness to readers because it was high quality and all in the one place for readers - but the process almost killed my passion for blogging. It was an empty process for me with no real sense of satisfaction. I stopped doing these kinds of blogs (even though they did make me money and readers complained that it was useful to them).

At it’s best - blogging is an exciting, interactive and fun experience that can give you inspiration, ideas and energy. This kind of blogging (ie using these automated tools) is about none of that.

4. Risk - all of the sales pages on these tools talk about how you can use these tools with all kinds of content legally by using content from sites with APIs, open source content or creative commons content. However almost every time I’ve come across a blog using automated blogging tools they have been scraping content from other blogs without permission from their RSS feeds.

Some blogs allow you to use their content but most do not. There’s real risk in using content from other sites in this way on a number of levels:

• Breaking Copyright - use the wrong persons content without your permission and you could end up on the end of legal proceedings.
• DMCAs and Risk to Your Hosting and Ad Partner Relationships - when I catch someone scraping my content I generally give them a warning but follow that up by issuing DMCAs to them, their site’s host and sometimes their advertisers (like AdSense). This can lead to you losing your hosting and being banned from ad networks (for example AdSense don’t allow you to put your ads on pages where you don’t own the copyright of the content). I know a lot of bloggers who issue DMCAs without warning and push a lot harder on these issues than I do - it can be a nightmare to have to work through these kinds of things.
• Damage to Your Brand - many bloggers skip the DMCA process and go with a ‘name and shame’ approach and publically call out those who steal their content. This can have a lasting impact upon your brand and personal name. There’s nothing worse than doing a Google search for your name and seeing the #1 result being a post an angry blogger wrote about you stealing their content.
• Google Penalties - ever heard of ‘duplicate content’? It’s what Google calls content that appears in more than one place on the web. I don’t know exactly how they treat this content but do know that they try to weed it out of their search results. They don’t get it all but they do get a lot of it and I suspect that a site that is largely classified as ‘duplicate’ will never be seen as an authoritative site on Google.

5. Create Something Worthwhile - my take home advice for bloggers is to create something online that is worthwhile, something that matters, something that inspires, informs and educates. Do this over the long haul and you’ll create something that not only means something but that has every chance of having lasting success.

I’ve heard from a few bloggers that they’ve had some success with automated blogging tools (although most of these were a couple of years back) but in every case they tell me that it’s usually temporary. They start blogs, see a bit of Google traffic before being banned from Google.

Their blogs never really amount to anything, they never build their own profile or become known as authorities in their niches, they never create useful sites that become niche leaders and to make money they have to keep starting new blogs over and over again.

To me this seems like an empty existence.

Me - I’d rather create something worthwhile that will not only survive but that will grow in momentum, build my brand and mean something to people.

What are you experiences of automated blogging tools?

Post from: Blog Tips at ProBlogger.

Why Automated Blogging Tools Should Be Avoided

Share This

Read the original post:
Why Automated Blogging Tools Should Be Avoided

After a great day at Microsoft Search Summit yesterday, I ended up having dinner with Will Carling ( @willcarling )who could have been quite a good Scottish rugby player apparently and who really just talks bollocks most of the time ;), but he is spot on with his views on social media. Twitter has allowed fans to hear Will’s real voice, unedited, not taken out of context. He admits that he doesn’t know anything technical about websites, so step up to the plate Dave Tosh (@davetosh ) of Elgg : a powerful, professional open source social engine.

BTW elgg.com are looking for developers, so hit Dave up on twitter. Between them they have created Rucku.com branded as a virtual club. Will has done a great job at building a community, crap job on Seo but I’m going to send DaveT some pointers, so that the ever professional Mr. Carling  seen here making me laugh and spoiling the first Photo can get some awesome traffic.

at least he didn’t mess the second one up … dick!

I would also like to thank Mel Carson for a great event and awesome dinner.

DaveN

Excerpt from:
Rucku Will Carling