Craigslist feeds blocked from Yahoo Pipes via 403 error
It looks like Yahoo Pipes, the incredibly amazing feed/data aggregator/sorter, is being blocked by Craigslist via a 403 Forbidden error. In response to my 2 month old to-do item, I went into Pipes to update my job feeds and notcied several errors.
After thinking for a minute, I realized I wasn’t getting my daily emails either. A quick search confirmed what I thought originally: Pipes is being blocked from accessing RSS feeds from Craiglist. Strangely, though, the info I found was 6+ months old. I’m not sure why this would have been working for me for many months and now just tank.
I used to feed job information from San Diego, Los Angeles, and Seattle into a Yahoo Pipe sort that discarded all the ads that didn’t fit what I do. I understand about 10% of what Pipes is capable but this 10% worked very well for me. Every morning, I had a list of available web development gigs in the nearby area that I could apply to. Not that anyone gets much of a response from Craiglist but, still, it was good to be getting myself out there. This little hiccup is not the end of the world but I’m confused. Why would Craigslist make it so much less convenient to use them? Why lose the community goodwill? It’s doubful that many people were using the Pipes/RSS combination so why not just let us do it?
I’m assuming that they’re trying to reduce data scraping (where sites/robots/people mine data on one site and use it as their own) but why is the RSS option still there?
Update 2/16/2009: The block might be temporary… who’s ruining it for the rest of us??
Update 2/16/2009: emailed Craig, as directed, received a response:
firstname.lastname@example.org to me
asking tech team now regarding problem, thanks!