Internet Scale Data Storage

Last week I read a blog from BackBlaze about their custom made peta byte storage system. It drew lot of attentions from the geek community. It is really interesting to see how they made component choices and design decisions for their storage requirement. Here is the link to the blog posthttp://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage

Cost comparison from various vendors is

Case design is quite interesting....

My Thoughts:

Google popularized commodity hardware and Amazon popularized low cost elastic storage and computing infrastructure. They brought in lot of real world confidence that "this kind of bold move will definitely work", Having boosted confidence by looking out those companies lot of new player going to come up with new refined low cost ideas and which will create enough trouble for big storage and other server vendors.

Internet scale Operating System (OS)

Os should evolve from running on single system to fabric of multi system nodes. It should marry network give a seamless scalability. Virtualization is just opposite of what I am mentioning here. But OS should do virtualization too natively.

I am just thinking of intel's turbo boost (http://www.intel.com/technology/turboboost/) which scales based on single or multi thread capabilities and needs.

Basic building block for network for fabric nodes + network management built in will help scale commodity hardware well.

My usecase is "buy N no of system and tell the OS that these are your siblings, children and clone", it will integrate natively with them. It might be useful to have patterns or strategy for specific use cases -- may be i term it as siblings/children/clone -- will share the load evenly/take over on non-availability/bring into work on demand etc and few other things like monitoring health of individual components should be there. With these things, OS really will ready for Internet scale. I don't know if Linux is going to go in this direction. I also read about how Google scales its infrastructure. But if Google open sources its customized Linux server used in its data center or contribute it back to linux it can happen over night. I expect this from Google and also revet its position as don't do evil. I still think if whole world can do better by exposing some of the innovative idea, it should be done immediately.

It also sounds like peer-peer built into OS -- p2p OS.

OpenId and oAuth

I am planning to implement the openid and oAuth in Erlang. There are couple of modules already there which I am going to look into however I am interested in Attribute exchange and policy extension for multi factor authentication. Current incarnation of openid has UX (User experience) issue due to URL redirections. It is addressed in Openid UI extension, So possibly the popup based solution would come into practice.

I will update this post when I figure out the issues and possible solutions.

I am interested in oAuth enforcing standard mechanism to share different resources (http).

Opera and Html5

Opera is really great browser but it lacks great brand value. May be because it is started its life as paid browser. I am trying opera 10, it has most of the html5 thingy like all the "input" types including "date" nice date widget is incorporated. I am not able to find "datagrid" which is discarded from html5 but I do find datalist tag is implemented in opera10. So for html5 lovers opera 10 would be the best home ground to play with new tags. Thank you opera for being industry leader to complaint with new standards. This is great effort because google chrome or apple safari -- 2 big guys behind html5 is still lacking with respect to opera. hm... most of the revenue for opera coming from Google. Did I say Google?

Is Java/C# dead?

Is Java Dead?

Almost a/an year or more back I read about "java is dead, long live java" I felt angry. I felt the author is really lunatic. But ... now I have a reason to say it should die. If you work with Java and struggled to scale for few 100 users... not even for Internet scale you will wish java should die. It looks like "Yes" for web application development. But I differ from original author. Java should die not because its bloat or not SOA ready (SOA is also declared dead by Anne Thomas Manes) but because it inherent issue with Thread and shared model. It can't even be resurrected by models like Ejb to help scale application. It is like mercury and it is hazardous to environment as you need more servers to do the same scale. May be thats the reason IBM left buying SUN.

Who is playing the Game?

Java in late 90s and till 2003 is really good as hardware vendors like Intel focused more on clock speed and tailing directly with moors law. But suddenly they realized that that route won't work and changed the rule. So rule of the game now shifted to multi core. This is the reason software developed for single core with high clock is struggling to cope up with low clock speed with multi cores.

What others Think about Java?

I also found some where in the Internet, I guess it is Steve Yegge, quoted "Java proved even pig can fly". Yes I still felt IBM and other anti MS guys made and give java a Star status. It grown to 800 pound gorilla and become useless in modern Internet scale application. Earlier I was looking to see any of the internet scale companies use Java as their underlying platform. hm... they know what they want.... No where I found a Java in use. I know how complex an Ejb is. But how much it can scale? It is worthless to provide hardware, though it is cheap, to produce sub optimal results -- even cheap will become very costly in the long run. But it is good at certain aspects. I pity on guys like Gavin King who contributed Webbeans and Seam framework which will kneel down and burry itself when 100 concurrent user access it. JCP is useless and they come up with standards when even after somsthing is obsolete, for ex, JSF came and AJAX destroyed it, JSF 2.0 finalized and Websocket and comet or more specific JQuery, Dojo, YUI/Ext Js already doing heavy lifting in the client side. So why we need to do page construction in the server side?

So what is the alternative? Yes there is emphatic answer. It is Erlang. We are in the modern world. We know how Json and REST is useFull and Soap is there to cleanse us from multi language hell. So it Amazon, Facebook, meebo and others enjoyed the power of Erlang. It is like nuclear bomb and it is so powerful concurrency and message passing actor model will help scale.

HTML5 user defined tag

One of the item lacking from html5 is defining custom tag. It should have provision to make custom tag. Tags are bound by certain parameters. It needs style to render. It needs events to react. So it is possible to provide option to create custom tag and those info related to style and events can be handled (cache etc) by browser.

Secondly,

Html5 should define browser contract. It already does. It have sections for UA aka user agent. So why can;t we define background process for browser? it is possible similar to iphone push notification but here each website can optionally provide API for dealing with "Push" call. Once this is ready, email app like Gmail expose API for new message. So once thats enlisted in browser -- may be by creating html application, new message will be shown similar to how out look shows in task bar but now with application short cut. Google chrome went in this direction but not with what I expected. This will make web application feature complete and compete with desktop application. This is possible as we are again into Browser war. So how is going to come up or going to complain with new standards going to win.