It’s just data

HTTP 301

Phillip Pearson: There's one thing I'd like to mention: the importance that the method used doesn't require separate RSS feeds to be generated for old and new locations.

To cope with existing aggregators that understand HTTP redirects but not the new redirect semantics, we want to be able to use an HTTP 301 redirect to point from the old to the new location. However, this means that RSS fetched from both URLs will be *identical*.

This makes perfect sense to me.  HTTP 301 redirect is the RESTful solution.  Blog hosting services should offer it.  RSS aggregators should not only follow it, but follow the instructions specified in RFC1945:

The requested resource has been assigned a new permanent URL and any future references to this resource should be done using that URL.

The problem?  Some aggregators are not currently layered properly to handle this.  Dave Winer indicates that Radio is one such aggregator at the moment. I suspect that this problem is widespread.  Update: Dave Winer indicates that he is testing a fix!  Bravo!

The solution?  The solution is not to do something instead of an HTTP redirect.  If there is a desire to do something in addition to an HTTP redirect, then any solution must deal with constraint that a redirect happened "under the covers" and that the feed the aggregator is actually reading is, in fact, the desired target feed.

Jon Udell seems to prefer one such solution.  I describe how it would work here.  Other solutions are possible.

My preference, however, would be to have HTTP 301 supported correctly on both the clients and servers.


Sam, you're reading it wrong. Radio does understand the HTTP redirect, it just doesn't record the redirect, yet. We can't rely on HTTP-level redirects in all cases. Been over this before, many times.

Posted by Dave Winer at

Dave, I did not say that Radio does not understand redirects. I said that while Radio understands the redirect at one level, it does not fully implement the instructions of the RFC, namely to ensure that future references should be done using that URL. I also seriously doubt that Radio is unique in this regard.

It is precisely because of this behavior that solutions which rely on the "old" RSS feed looking different than the "new" RSS feed will run into troubles when faced with HTTP 301 redirects.

Posted by Sam Ruby at

I've worked within the Radio code and tcp.client <b>does not</b> have a way at this point in time to emit something back up to the caller that would accomodate redirection. It has a ctFollowRedirects integer but it does not have a callback that would allow the caller to 'be informed' of what final destination was found. So when xml.rss.readservice asks for the data it's only getting back string data. This isn't a flaw, it's just the way it's constructed.

So Dave, Sam IS correct in his statement. It will, of course, be good to see <i>all</i> tools move forward in handling redirects better.

Posted by Bill Kearney at

Correction, the function is tcp.httpClient but the point remains.

Sorry for the HTML codes.

Posted by Bill Kearney at

Bill, last night I added a table parameter to tcp.httpClient that accumulates redirection info, and the aggregator watches that, and makes the change in the database (Murphy willing, it's new code of course).

Posted by Dave Winer at

FWIW, Aggie properly handles 301's

Posted by Simon Fell at

Thus further validating Sam's original comment. It's always good to be working on evolving things.

As it stood until 'last night', Radio did not pay any mind to redirects with regard to using as a means to have the aggregator learn about using a new URL. The verb within tcp.httpClient could only be told how many layers of redirect to follow, not to filter that back up. This, in and of itself is perfectly fine. The issue is more that xml.readService simply called tcp.httpClient with a redirect count of 5 and didn't do a thing to learn from the process. Many aggregators make this mistake and will hopefully make improvements.

As early as a year ago I suggested it would be useful to factor redirection into feed handling. [1]

Morbus, another person you delight in demonizing, touched on it even before that. [2]

In Radio's own discussion group a user complained about redirect handling. [3]

But issues about redirection in feeds is well older than that even. [4]

It's nice to see the aggregator programs finally playing catch up.

[1] http://groups.yahoo.com/group/syndic8/message/100
[2] http://groups.yahoo.com/group/syndication/message/1120
[3] http://radio.userland.com/discuss/msgReader$10168?mode=topic
[4] http://groups.yahoo.com/group/rss-dev/message/264


Posted by Bill Kearney

at

That must be Aggie-from-CVS that handles 301s correctly? I just grabbed a fresh copy of RC4 to be sure, and it's not changing the URL in myChannels.opml.

Posted by Phil Ringnalda at

its in RC4, but it doesn't change the myChannels.opml, it tracks it in the http index file that has the etag and last-modified data in as well [look in the ./http directory]

Posted by Simon Fell at

Let them eat cake

Dave Winerintroduces a new XML format, guaranteed to make existing aggregators go Perhaps I may use it when all other approaches have failed. Meanwhile I have too much respect for my non-Radio readers to simply cut them off like that.Meanwhile, I stil... [more]

Trackback from Sam Ruby

at

In brief: insomniac edition

A little of everything.  I mean, why not?  I'm up anyway.... [more]

Trackback from dive into mark

at

exploding archives

Astute readers of the actual HTML representation of this site will notice that the archives section just exploded. In a desire to aggregate (literally) nearly everything I've published online, I've replicated each Stating the Obvious piece I...

Excerpt from this is sippey.com at

Phillip Pearson: This makes perfect sense to me. HTTP 301 redirect is the RESTful solution. Blog hosting services should offer it. RSS aggregators should not only follow it, but follow the instructions specified in RFC1945:The requested resource...

Excerpt from phil ringnalda dot com: Redirecting RSS redirection: Comments at

Sam Ruby

Phillip Pearson: There's one thing I'd like to mention: the importance that the method used doesn't require separate RSS feeds to be generated for old and new locations. To cope with existing aggregators that understand HTTP redirects but not the...

Excerpt from phil ringnalda dot com: Redirecting RSS redirection: Comments at

Sam Ruby

Food for thought. Reportedly Aggie properly supports HTTP 301's. I haven't heard back yet from the author of Amphetadesk, but clearly he is aware of the issue.. Now it looks like Radio is well onto its way to supporting 301 completely. Perhaps The...

Excerpt from phil ringnalda dot com: Redirecting RSS redirection: Comments at

rss 301 redirects

I'm researching the right way to redirect pages, and I want to do it as smoothly as possible for feed...... [more]

Trackback from lovelinks

at

And then there was one (RSS feed)

As promised, I've done away with all but one of my RSS feeds for the main weblog. The RSS feed for kottke.org is located at [link] (RSS 2.0 format, if that matters), contains all posts except the remaindered links (rl feed here), includes only a short excerpt of each post, and contains no HTML, not even links (except for the book posts). You shouldn't need to worry about this, but...... [more]

Trackback from kottke.org

at

Sam Ruby: HTTP 301

Sam Ruby: HTTP 301 permanent redirects should work even for feed URLs...

Excerpt from AndrewSW: Blog at

英语 英语培训 英语考试

Posted by 英语 at

HubMed.org

HubMed.org is now the main address for HubMed. Some bug fixes and updates have already been done, and there are a few more to come. RSS/Atom feeds should update automatically (if your newsreader supports HTTP 301 redirects), and old addresses should...

Excerpt from HubLog at

Redirecting comment feeds

This has lead to my RSS comment feeds producing warnings, and I can’t have that.  So effective immediately, all my per entry and overall RSS comment feeds permanently redirect to equivalent Atom 1.0 feeds. If your aggregator has problem with ti... [more]

Trackback from Sam Ruby

at

Feed Redirects Implemented: Full-Text RSS 2.0 uber alles!

I’ve recently decided to consolidate the various syndication feeds for my blog. Historically I’ve offered the following feeds: rss2.xml: a full-text RSS 2.0 feed. This is by far the most used. It’s what my RSS auto-discovery link advertises....

Excerpt from Jeremy Zawodny's blog at

PHP 301 Redirect Function with Headers Check and Javascript Fallback

If you are changing your URL structure, for example when moving to a search engine friendly URLs system, you need to be able to let Google and all the other search engines that the page has moved to the new URL. You really don’t want to display the...

Excerpt from Edmonds Commerce Blog at

Feed readers and 301 redirects

I am moving my news feed off our our terribly horrible 6 year old custom solution onto wordpress. Everything is working fine all I need to do is update the RSS feeds. I did a bit of research and tested my 301 redirect on google reader, and it seems...

Excerpt from A Day in Paradise at

Add your comment