Monday, March 31, 2008

Simple DB

Simple DB is a new Amazon database service. It comes to complete the other two Amazon "elastic demand" services (the cloud computing EC2 service and the storage S3 service). As you can see in these slides, Simple DB is a database with relaxed constraints: no schema is needed, no explicit relationship and referential integrity constraints, not too many data types. On the other hand, it provides a simple API, ensured speed, easy search (all fields are indexed), high availability and on demand scalability.

Some months after the release of the beta version of Simple DB, Microsoft has launched SQL Server Data Services (SSDS). As you can see in this document, SSDS is very similar to Simple DB. It seems that Microsoft has been inspired by Simple DB to create SSDS.

From my point of view, this kind of databases are going to be very useful for developers. They provide some advantages comparing to typical relational databases. Among such advantages, easy usage and high scalability can be mentioned. And I believe Amazon is doing very well with its strategy of developing this kind of scalable services.

Zookeper Video and Slides

Zookeper is a coordination service that can provide a lot of help when developing distributed systems (See my previous post about it). An introduction to Zookeper has been recently published. You can see the video and some slides in PDF.

Zookeper uses the file system paradigm but adding some enhancements in synchronization and providing notification. It results really elegant to manage the coordination of a distributed system with Zookeper. Distributed procedures like leader election and distributed locks are simple with Zookeper.

Tuesday, March 11, 2008

Engineering Philosophy

I have found these commandments for Google engineers:

1. All developers work out of a ~single source depot; shared infrastructure!
2. A developer can fix bugs anywhere in the source tree.
3. Building a product takes 3 commands ("get, config, make")
4. Uniform coding style guidelines across company
5. Code reviews mandatory for all checkins
6. Pervasive unit testing, written by developers
7. Unit tests run continuously, email sent on failure
8. Powerful tools, shared company-wide
9. Rapid project cycles; developers change projects often; 20% time
10. Peer-driven review process; flat management structure
11. Transparency into projects, code, process, ideas, etc.
12. Dozens of offices around world => hire best people regardless of location

Nice philosophy. I would like to highlight some of them from the point of view of startups.

A uniform coding style is needed if you want your code to grow fast but without losing evolvability and understandability. Setting some standards at the start of the project will improve things in the long term.

Startups need to grow fast, very fast. Pervasive unit testing is the unique way to grow fast without making mistakes. If you want to avoid the risk of moving backwards with new developments, you need obsessive testing. People usually do not understand it, but unit testing improves productivity, too. With unit testing you will detect if some new development destroys or introduces bugs in the software. That improves the confidence in the developed software. In the long term, you will have more time to develop new software as you do not have to waste it solving bugs. The unit testing increases productivity. Use it if you do not want your software to be a card castle.

You have to convince developers of the ideas of developing for sharing. It is usual to find developers creating some code only for their personal use in a particular case. That is wrong. If the developer has found something that he requires, it is obvious that another one may need it in the future too. So instead of developing this piece of code thinking that you are going to be the only one to use, you have to develop it thinking that this code can be useful for other developers in the future. That means that you have to take care of the API, generalize, make documentation, mantain and share your work. That will improve the reusability of the software.

Rapid project cycles are good. Long time project cycles usually increase the complexity and time of the development.

Transparency is another attribute needed in startups. You need all your team to know everything, always to have the targets clear in their minds. That makes easy to each member of the team to know how he can contribute to the company.

Saturday, March 8, 2008

BCN Meeting

I have received this announcement from my friend Tomy Pelluz:
hackers, developers, designers, entrepreneur, thinkers, web-lovers, code-poets...

Something is happening in Barcelona Web n.0 scenario and you should pay attention...

Do not miss it ;-)

We will met here >
Temporal coordinates > Tuesday 11 - 20:35
And then we will decide what we want to do... beers, tapas, running... everything is possible

Bring your friends, bring your brain.

Nice people like that will be there. I am planning to attend. I’ll see you there ;-)

Sunday, March 2, 2008

Pig Talk

Christopher Olston gave a talk about Pig. You can see the video of the talk here and the slides here.

Pig is an Open Source tool for the processsing an analysis of big datasets developed mainly by Yahoo! Research.