Brief about internet scale....
1. Single User application - We developed applications, say MS Office, which caters to single person. We won't bother multiple items here like how much memory it eats up, how much processor it takes, how much disk space it takes etc. Why because it runs on either desktop or laptop; in fact netbooks or palm top (windows mobile based phones, iPhone, blackberry etc) has concern on processor memory and space. Program should be responsive enough for the single user. This is one class application.
2. Minimal User Application - Next, small scale application. where few people access the system and they may not be bothering about whether it is up or restarting to make little bit faster (OS like windows XP needs restart to clean up memory leaks) so here people using are highly tolerant and not the applications as such. So it can run on any normal to medium profile hardware based servers.
3. Enterprise Scale Or Limited User Application: Normally enterprise application like enterprise email, Document Management Systems or leave systems etc will be use by limited set of folks and it is mostly intranet application accessed from local network. Normally it is developed in Java EE, .Net, RoR or Python or combinations of them. Normally Webservice or some other complex distributed programming paradigm will be followed.
4. Super scalable Applications: It is current class application pioneered by Google (Consumer email, youtube, maps), Amazon, Facebook (Chat), current Twitter kind of applications. It was unnoticed and tormented in Telecom environment. All the companies did a really hardwork to come up with their platform which is kind of unique and innovative. Infrastructure is built such a way it scales infinitesimal, internet scaling. It is tough to hit, important events make people hit the website. Recent fiasco Nokia's Ovi store broke down, Tweeter's frequent outage due to spikes. Why the concurrency is not free or not easy. It is like gymnastic or performing martial arts.... needs lots of coordination and control. It is tough and needs practice. Some of the paradigm we used to make it grow like imperative programing (Object oriented programing/ procedural programing) suffered a lot of maintaining state. It is like obese people suffering from excessive fat. It distract many thing including co-ordination. So difficult to parallelize. Ok.. Is there a simple way to over come this. Yes and no. I researched a lot. Closely following Twitter and Facebook gives many idea. First of all needs good hardware and servers. Next needs infrastructure which effectively utilizes the hardware. My desktop and Laptop has dual core (I am 2 years behind extreme computing with Quadcores) and multicore GPU. But how my programming platform effectively utilizes the cores is important point to scale. It is kind of parallel to Car which goes 0-120 miles in 6 secs. What is the use if my speed limit is 30 miles? Or I can say riding a bicycle in a express high way where I can go upto 85 miles per hr. This is exactly it is happening. My hardware is so powerful but my software is so dumb and it is not multi tasking. Or even single tasking with all the cores. It is like 4 people needs to do a job but they don't know how to split it and do the work. So one person starts and others are waiting until the guy finishes. So what is the problem when i have a computer with ample amount of power and I can't normally bother between 0.7 sec or 1.4 secs of completion. Still it is fast.... But what if millions of people access it. It will keel into halt but it suppose not to. So counting every penny is important as it amounts to lot of other things. Energy wastage, environmental issues etc.
So how Twitter or Facebook addressed this issue. They moved some of the heart of the code into Functional Programming Code. For Twitter, it is Scala and for Facebook, it is Erlang. Both are actor based shared nothing function language evolved from Alanso Chruch Lambda Calculus. Academic loves these languages.
So I am in the process of learning Erlang. I found great book Joe Amrstrong's Concurrent Programing with Erlang second edition. So far it is quite interesting. It is lean and mean from the start. It is damed concurrency and fault tolerance and high scalability in its DNA.
There is no loop. What a hell are you talking about? All i need to do is unlearn my OOPS oops!!!.
I create a looping control if i need. I don't need design pattern. I am not telling how it needs to do something. As my friend Santhosh GR said, I just needs to tell what I need. No state to maintain. Oh... how can I program with the one which doesn't have or doesn't recommend one which I used for developing even adding two nos.
Having set the context, I will continue on Erlang later.... It also brings me how Java going to survive? It looks like its current state it is not fit into "Internet Scale". This is same for C# and other imperative languages. Their core is thread. Is thread is bad? It looks, sounds and feels bad. Why because thread are, at least in current state, not able to scale across multiple cores. Java is faster on single core processor with high clock speed compare to multi core highly effective modern processor. It degrades shabbily beyond quad cores and Intel has 6 core already and going to launch 8 cores and AMD story also the same. But Java, Java 6 came 2 years back and 7 still in pre beta. So time and growth of hardware and software are looking north and south.
