Tuesday, February 12, 2008

Open Source and Startups: the next step forward

It is clear that the open source platform LAMP was a revolution for startups. Open source projects like MySql, Apache or PHP brought “vitamine” to many startups to reach their objectives. Some of them could have not reached success if they had had to buy this kind of technology.

LAMP is currently the standard to meet with when starting new Internet business. Lamp (or some of its variations) is used in a high percentage of new websites. Nevertheless, as projects become bigger, the limits of the LAMP architecture appear. There are scalability issues. The biggest one is usually the impossibility to scale the database (usually MySql).

But, from my point of view, a new revolution is here. Two projects,
Lucene and Hadoop, have come to take the next step forward. They do not come to replace LAMP software, they come to complete it.
Lucene is a search engine library. It can index a big amount of text data into an index file. Such index can answer keyword queries. The library is fast, and can index more data and perform queries faster than database-like search systems.
Hadoop is an open source implementation of the Google MapReduce distributed system. It allows the processing of big amounts of data over a cluster of commodity computers. I do think that this is one of the most important open source projects nowadays. And I think new startups will take profit of these projects in order to develop innovative services. As an example of the size of that these projects can have, Powerset, a new startup, is trying to build a natural language search engine using Hadoop.

I will write some more about this projects and some subprojects in future posts.

No comments: