Dave Winer: I only want changes.xml to have weblogs in it. I want to set up some kind of web app that makes sure that happens. Any thoughts on how to do that?
Those that read my weblog may have detected a passion I have for increasing the connections between things. Today, we have an another opportunity to decrease entropy.
In the comments section, there are a number of good suggestions. Including one for a "smackup". Whereby people are expected to vouch for other people. A system where there is no foes, only friends.
There is prior art for this. It is called Friend Of A Friend, or FOAF for short. It can do much more than what I am about to describe, but lets keep things simple and focus on the problem at hand.
weblogs.com keeps track of existing weblogs for the purposes of detecting if a ping truly represents a change. So lets treat the existing weblogs as known good. Now lets look at what happens when we get a new ping.
If the site identified in the ping is already on the list, then all proceeds normally.
If that site is not on the list, then we need to check further. Even though he is an established blogger, let's take James Snell for example. Fetch his page, and look for <link> tags. When you find one with type="meta" and type="application/rdf+xml", use the href.
Go there and you will find a file. Don't freak, it has some other things that you don't care about at this time. In this case, all we are looking for is rdfs:seeAlso tags. These identify people that James says are his friends. But we can't just take his word for it. We can go there.
The first friend listed happens to be me. Let's go there. This time, we are looking for two things. First we look for a rdfs:seeAlso link back to James Snell to verify the link. Seeing that it is there, we have a level of validation. Now we have to decide whether that is just a shill. So we do one additional validation. We look for a foaf:homepage and see if it is in the list of existing weblog.com websites. If so, we are done.
Simple. Effective. Works with and builds on prior art.
Building a FOAF file is easy. There even is a FOAF-a-matic to help you.
Actually, FOAF was meant originally as a means of discovery, not a means of restriction.
How to say this without seeming to be beating on you and Dave...well, not matter what I do it will seem like I'm beating on you and Dave, so...
Both you and Dave are looking at putting technical restrictions up that are going to filter out those people who most need to connect -- those people who don't know about FOAF, or who don't have a site whereby they can arbitrarily created files (such as Blogger). They don't know how to do these things, they even know where to start.
These technical solutions to social problems only serve to perpetuate a technical elitism over a social openness.
I don't have to ask Dan, or Leigh, or Edd to see what they're reaction would be to your suggested use of FOAF as a gatekeeper.
This is nothing more than an electronic bouncer at an insider's nightclub.
Shelley, I understand your perspective. You have been consistently against an insiders nightclubs.
Weblogs.com has been the polar opposite to date. it has been open to everybody. At some point, this will be an open invite to spammers. We can wait and react, or we can plan ahead.
I thought what I was proposing was fairly minimal. Being vouched for by anybody who has a weblog is a pretty low bar.
Newcomers often don't have someone vouching for them. You propose putting up a limit for people that might actually hinder some from entering. What about the casual blogger in Africa? Nobody knows her and since she can't ping, nobody will discover her. Not good.
People who are long enough member of some community often forget those people that are not members. Those people might be put off by some bully at the front door, even if it is just the electronical version of a bully ;-)
I like the idea, but:
The barrier to entry it too high. Requires the blogger to know what a foaf file is to create one. I'd like both of these, but it isn't going to happen soon enough. It also requires the blogger to know another blogger that has a foaf file. Very difficult, especially for new bloggers.
And this doesn't actually solve the problem (I think) of trying to restrict pages to weblogs. Can't rdfs:seeAlso refer to any page, not just weblogs?
Or consider that foaf:Group allows a site to offer up information about it's membership, if so desired. From this perspective you could, conceivably, have one service 'trust' another. That is, Syndic8's foaf service could be inquired of to see if the site is a known one. Likewise being able to state whether it knows the contact for that site.
The idea is that foaf isn't just for people alone, groups can use it as well.
And, as I'm sure everyone would likely inquire, anything offering up a foaf info would certainly do well to let it's users know what's going on.
For the solution to weblogs.com not doing it's own dirty work, it could always use the Syndic8 xml-rpc calls [1] to see if there's a known RSS feed for that site's URL. Were other sites that possessed catalogs or directories of links to likewise start sharing them, either via web services or in a RESTful fashion, it might be a good start.
Does a remote site's statement count as vouching for you? That's up to the site asking the questions. If it finds the remote site doesn't give it reliable references then it's certainly welcome to cease using it. Couple enough of these references together, maybe not on every request, and you start building a weight. Enough 'points' and the URL's good enough to continue.
More of that 'semantic web' stuff some of us keep going on about...
How do any of these solutions stop non-weblogs from pinging?
The syndic8 solution of seeing if the site offers an RSS feed and then somehow using that offers nothing: many sites on syndic8 offer RSS feeds and aren't blogs.
As for Sams idea, I agree that the barrier is too high. How many bloggers know what FOAF is? How many of those have bothered to implement it? I don't think the numbers would be as high as it is in the technical circles that visit this site. We'd end up excluding the newcomers who don't understand FOAF, RDF or anything similar. If they can't ping in the first place to get traffic, how are they supposed to get an existing member of the "allowed" blogs to verify and vouch for them?
It's like a bouncer saying: regulars and their friends only. If you're not a regular, or don't know one, you're left out in the cold feeling like a second-class citizen.
Also, how many weblogs know they're weblogs, but ping anyway? How do we say what is a blog and what isn't? Many have tried, all have failed.
Stuart: good point. The thing to do is to go to intertwingly.net and look for a link tag vouching for that foaf file.
Gary: I now know about your weblog. I might even be inclined to vouch for it. And I didn't even have to go to weblogs.com to find out about it.
Sam, as Greg iterated, the cool thing is that anyone can have a blog and ping weblogs.com to direct some traffic their way (personally, I doubled my hits on day one of pinging). You found my weblog because I had something to say about your ideas, but it's not too difficult to imagine a new blogger who is only interested in a self-contained journal-style blog (see 99% of LiveJournal) without being drawn into other peoples blogs and still wants to let the world know they exist. For now, pinging weblogs.com allows this. Would it be possible in the future?
A weaker point, but one worth considering, how often do you update your FOAF file? Personally, my profile has had 1 minor change since I created. I'm too busy (read: have no inclination) to modify and upload it every time I find a new blog I like in an effort to get it verified. It's too much hassle. I wonder how many people feel the same...
I agree that weblogs.com should be protected against the potential spammers, but can't see any particularly good way of doing it.
Sorry for the late reply, only just found this discussion (www.feedster.com++).
Re "I don't have to ask Dan, or Leigh, or Edd to see what they're reaction would be to your suggested use of FOAF as a gatekeeper."...
...maybe Edd and Leigh are more easily mind-read, but I've got a tinfoil hat so my thoughts can't be remotely accessed.
I have several views on this.
Firstly, I absolutely agree with Shelley that it would be awful to encourage the use of FOAF, XML, RDF etc technology to make weblogging (and web-page writing) even more exclusionary. I didn't get the impression that was Sam's intention.
I do believe, however, that there may be a role for a very broad notion of endorsement of (otherwise unknown) parties, both in a Web writing and in email contexts. My experiments with FOAF for so-called whitelist spam-filtering were in a similar vein (and are working quite nicely on my inbox, coupled with spamassassin to filter unknown senders from obviously-spammer unknown senders).
http://www.w3.org/2001/12/rubyrdf/util/foafwhite/intro.html
The basic idea is that of holding hands in the spamstorm... there is so much crap swirling around the Internet that its sensible human users need to hold on to known connections or be swept away... Well OK maybe that metaphor doesn't quite work, but the idea is intended to be inclusive.
It isn't that one would need 'approval' before being able to do anything online, more that additional facilities would open up to parties on receiving end of even a single welcoming link from someone else already acknowledged (as a non-spamming human being, weblog writer, whatever) in the online community.
Until I started using a list of known mailboxes plus spamassassin, I was missing mail due to the levels of spam. Now, everything in my inbox is either legitimate mail, or on (thankfully rare) occasion, a forged message whose From: field matches my list of legit mailboxes. This does, as Shelley points out, increase the risk that mail from people new to the online community could be ignored. I do check my 'unknown senders' folder, but it is 50% spam (since spamassassin isn't perfect and I set the threshold cautiously, and v rarely check my probably-spam folder). But this does guarantee that no mail from anyone on my 25,000 long list of addresses will be accidentally dismissed as spam.
So there are no easy answers here. Without filtering we're overwhelmed by spam or irrelevance; with filtering there's a risk of excluding people/content. I don't think the "no filtering ever" argument is sustainable... attention will have to shift to making sure filtering is done in socially responsible, inclusive and transparent ways.