User Participation Drives Retooled Wikia Search Engine – Yahoo! News
One concern with a community-based system is that users will try to “game the system,” deleting references to competitors or engaging in trash talk in the comments. “We should see some fierce edit wars on this one,” noted Erick Schonfeld on the TechCrunch blog.

As if no one ever games Google or Yahoo!…  At least with this approach, one can see the results and then decide, whereas with the other players, you never know who is gaming, nor do you know if you can actually trust the engine itself.  With Wikia, you can get the code…  It’s called Lucene.

In order for a project to be OPEN, it must put out releases.  In the early stages, these releases are very limited in functionality.  They may only do one or two things well and a whole lot of things average and a fair number of things poorly. How many of you remember Google’s first releases?  How does it compare with today?

“But one analyst said he isn’t so sure Wales’ efforts do much to improve the current model.”

My god, give me a break.  It’s an ALPHA release.  This is what people don’t get about open source.  In open source, you put everything out there almost all the time.  People can see and criticize it and improve it all the time.  You don’t keep secrets in open source.  Seriously, you never even get to see Google’s alpha releases, the best you get is beta and even that is more like a “public beta” as opposed to an internal beta.  Lucene’s early releases sucked, by today’s standards, but I don’t think you would say it hasn’t radically changed the search landscape.  So, if an ALPHA release is, by this analyst’s account, at the same level as the current model, then Wikia probably is on to something.  The fact is, they are putting the alpha version out there to attract people to try it out and get feedback as well as attract people interested in making it better.

Really, how can knowing why your results rank where they do be a bad thing?  Even if Google or Y! is pure in their results, one can never know because they don’t tell you.   This was the vision of creating Nutch by Doug Cutting in the first place and now Wikia has finally stepped up and put the computing resources into making this happen.

Bottom line, cut Wikia some slack.  If in a year it is gone or still the same, then this analyst is right, but to say an ALPHA release does little to change things is premature.   If anything, maybe it will pressure the big boys into opening up some (even Matt Cutt’s admits competition is a good thing in his comments at http://www.mattcutts.com/blog/wikia-launches/), as Yahoo! has recently done with Hadoop (which, by the way, is also going to put pressure on Google in the long term as well.)

“I think he doesn’t really understand the scale of what Google has to handle in terms of the queries from around the world and the amount of traffic that flows to it and the attempts that are made to try to manipulate it,” said Danny Sullivan, editor of the Web site Search Engine Land.

With all due respect to Mr. Sullivan, isn’t this the point of what Mr. Wales is trying to do?  That is, isn’t this an attempt to make search more discernible and meaningful to more people?  To bring it out of Google’s ultra-secret, do no evil, hands and into the public.

Large scale search has been tied up in the hands of very few people for a good long time.  Projects like Nutch and Lucene can provide the means to do large scale search, but have a hard time demonstrating it in public because it requires a lot of computing horsepower (unless, of course, you take the time to figure out what sites are using them, at which point you realize how big it can go).  Perhaps Mr. Wales’ attempt will resolve that, maybe it won’t.  However, until there is a viable, open search engine, we will all be left wondering who paid for which position in the search results.