Menu

Craigslist feeds blocked from Yahoo Pipes via 403 error

It looks like Yahoo Pipes, the incredibly amazing feed/data aggregator/sorter, is being blocked by Craigslist via a 403  Forbidden error. In response to my 2 month old to-do item, I went into Pipes to update my job feeds and notcied several errors.

Yahoo Pipes being blocked from accessing Craiglist feeds
Yahoo Pipes being blocked from accessing Craiglist feeds

After thinking for a minute, I realized I wasn’t getting my daily emails either. A quick search confirmed what I thought originally: Pipes is being blocked from accessing RSS feeds from Craiglist. Strangely, though, the info I found was 6+ months old. I’m not sure why this would have been working for me for many months and now just tank.

I used to feed job information from San Diego, Los Angeles, and Seattle into a Yahoo Pipe sort that discarded all the ads that didn’t fit what I do. I understand about 10% of what Pipes is capable but this 10% worked very well for me. Every morning, I had a list of available web development gigs in the nearby area that I could apply to. Not that anyone gets much of a response from Craiglist but, still, it was good to be getting myself out there. This little hiccup is not the end of the world but I’m confused. Why would Craigslist make it so much less convenient to use them? Why lose the community goodwill? It’s doubful that many people were using the Pipes/RSS combination so why not just let us do it?

I’m assuming that they’re trying to reduce data scraping (where sites/robots/people mine data on one site and use it as their own)  but why is the RSS option still there?

Update 2/16/2009: The block might be temporary… who’s ruining it for the rest of us??

craig_tweet

Update 2/16/2009: emailed Craig, as directed, received a response:

craig@craigslist.org to me

asking tech team now regarding problem, thanks!

Replies

Total: 4

Your email address will not be published. Required fields are marked *


  1. Josh

    February 17, 2010 at 8:02 am  •  Reply

    Josh says:

    Ugh, you’ve got to be kidding… so short-sighted. Is it really sapping that much bandwidth?


  2. Monty H

    February 17, 2010 at 5:49 am  •  Reply

    Monty H says:

    Looks as though Twitter atom search has blocked Pipes too as of today!

    Getting 403 for this:
    http://search.twitter.com/search.atom?q=search

    Shame!


  3. Josh

    January 8, 2010 at 8:51 am  •  Reply

    Josh says:

    Seemed like it was working for a while there… too bad. I guess I’ll have to gi with Google Alerts…


  4. Leo

    January 8, 2010 at 8:47 am  •  Reply

    Leo says:

    Both Craigslist and Zeke’s Express sites have been hit with the 403 and 404 forbidden msgs. This happened suddenly on 01/07/10 at @11:15. it appears that someone has decided that their will shall be imposed on the rest…

"Policy"

If you've got something to say about the above, now is your time. I moderate for spam, relevance, and abuse but, aside from that, this is an open forum. I will not publish your email address but feel free to be anonymous. If you just have a general question or want to get in touch, my contact form is the best place for that. Thanks in advance!