Apache's [HTML5 Websocket] Conundrum

Setting the Context:

I think, few days back I created a enhancement request (bug) in apache bugzilla to add new Websocket feature. I got a couple of boisterous comment and I wanted to share that to the other folks. Here is more on the issue https://issues.apache.org/bugzilla/show_bug.cgi?id=47485.

Here is what I felt.

Death of the product is not because of the product itself but because of the people behind them, it is because of lacking foresight and shortsighted decision making. I saw death of Netscape and Fall of IE and much more. It is not because of IE is bad but because other alternatives which are too good to omit. If anything fails to catch up with competitor quickly, it means it is manhandled by those aforementioned folks. I am able to smell that in Apache. HTML5 is huge force, it can’t be omitted. I am able to play video in iPhone Safari (without Flash), Firefox 3.5, Opera (Version 10 has more than 95% HTML5 feature implemented), Webkit is in full throttle and it carries easily Safari and Google Chrome browser future forward. HTML5 also comes up with enhancement for Server. Now ball is on Server provides’/Apache’s court. So, Apache should wake up –Why? Websocket is the next driving force for the future Web platform. I don’t want to see Apache’s omission only because of Websocket support. I am able to see quite a lot of web servers are popping up. They are expecting break through – late to market is again going to be big break through for new players (server providers) who has product with those features.

I am also able to see few websocket servers. Googling will atleast give basic minimal knowledge or Answers for the questions like “why?”. I never anticipate all folks who stalk apache are Einstein but my only concern is they should not cause trouble to Apache.

I also found another comment, quite interesting. Websocket is proposed by single guy. I am not able to understand why this is matter. If million folks come up with crappy thingy doesn’t mean that – it is good. But websocket is backed by All the heavy weights – (Google, Apple, Mozilla, Opera and Microsoft (not yet sure)) So Apache’s challenge is to track these naysayers out and shift their focus on impending standards which makes them or keeps them as leader.

Adhoc Browser Performance Testing (IE8, Google Chrome2, Firefox3.5) - Round 2 (with SSD)

This time I tried with IE8 and entirely different system again I used the same link to see the performance http://ideamill.synaptrixgroup.com/jquery/tablefilter/largetabletest.htm

(I also open/close the borwser few times to bring all the moving parts close to the heart of OS -:) )

	Chrome2	IE8	Firefox3.5
Loading the above link button – iteration 1	0.5 sec	1.6 Sec	1.2 Sec
Loading the above link– iteration 2 (Close the browser, reload it again)	0.5 sec	1.7 Sec	1.1 Sec
Clicking GO button – iteration 1	1.9 Sec	Broken and it took 5.5 Sec to hide the table and stopped rendering the table below “Go” button. I tried compatible mode but it also didn't help.	4.3 Sec

For the kick, I checked with iPhone OS3.0 safari, It took 0.5 sec to show few rows and tool almost 3.5 minutes to render the entire page. I also checked it render all the rows. I tried to click the "Go" button, It did not work. Though it is not bad as it is not meant for this kind of load.

System used:

Core2Duo 3 GB DDRII RAM

Gskill Falcon 128GB SSD

All the test ran on the same system.

Disclaimer:

I considered Network latencies, Network congestion etc would be same for all the browser

as it is tested on the same system. You can still use the given Link to try it yourself.

Adhoc Browser Performance Testing (IE, Google Chrome, Firefox)

I came across complex requirement. Here is that,

It is huge grid table with context menu, filter, sorting etc, no of rows goes beyond 1000 and column is more than 16. I found Ext JS’s Grid is used. It is taking almost a while to load completely. It is really slow and it feels as if CPU 100% utilized and no memory is available. I wanted to picture the problem; here is the test something similar to my requirement http://ideamill.synaptrixgroup.com/jquery/tablefilter/largetabletest.htm

	Chrome2	IE6	Firefox3.5
Loading the above link button – iteration 1	1.6 sec	2.3 Sec	1.8 Sec
Loading the above link– iteration 2 (Close the browser, reload it again)	1.5 sec	1.8 Sec	1.9 Sec
Clicking GO button – iteration 1	3.1 Sec	25.5 Sec	6 Sec
Clicking GO button – iteration 2	3 sec	25.2 Sec	6.1 Sec

Note: I used iPhone stop watch to test it. Though it is not fair to compare when IE 8 is out, but I have to work with IE6. I am interested in last 2 rows. Even with the margin of 0.5 sec error, Google chrome and to some extend the firefox3.0 gouged the IE.

System used:

Pentium D 3.0 GHz, 2 GB of RAM

All the test ran on the same system.

There are 2 things should happen,

IE should die including IE8 or it should reinvent itself for betterment of community. Microsoft should not punish the community and enterprise. Please do Microsoft....

Erlang and SOAP Webservice

I just completed invoking SOAP Webservice from Erlang.

Here is what I did,

1. inets:start(), This is very important without which http:request will not be succeeded.
2. Took a shortcut: (hm.. not actually) I treated SOAP Call as Http POST call.
3. I used Erlang http module
1. http:request( post, URL, [{"Host" , Host}, {"SOAPAction", ""}],

"text/xml", getData ()},

[ {timeout, 5000} ], [{body_format, binary}])

2. Used case expression to handle different condition
3. getData() returns binary form of soap message. Here, correct payload of SOAP Request will be needed to make this request template. It returns binary form of the SOAP payload (String form can also be used)
4. http:request would return either
1. {ok, Result} -> Handle successful http request
2. {error, Reason} -> Handle Errors
3. {ok, saved_to_file}-> if you use “stream” option, this pattern will be invoked and it is optional.
5. Result is response payload (http payload) and can be processed

Note:

Erlang has good support for Proxy and Proxy authentication unlike python. Python is mess in this area.

Is Apple reinventing the wheel?

I went through Apple's proposal http://tools.ietf.org/html/draft-pantos-http-live-streaming-01. It is great to see Apple's push for HTTP Streaming and I appreciate the effort. But devils in details. I have different thoughts here. It looks complex as it needs, server side Segmenter. As per HTTP 1.1, server can handle Chunked transfer encoding (http://tools.ietf.org/html/rfc2616 section 3.6.1 Chunked Transfer Coding). When server/client comes to an agreement about size/blocks of chunks, it can stream the video without even the requirement of Playlist (m3p8). It simplifies another call to obtain playlist.

Apple devised this approach for iPhone. And its safari is able to handle video playback natively. When streaming natively handled in HTTP and Html5 video tag and video tag playback execution knows to utilize http transfer encoding, it can do a streaming, we don't even need complex segmenter or intermediary. It is kind of pushing the problem from one place to another.

Version Control - how to save my work?

I am a heavy user of IBM products, I heavily used ClearCase, it is good we can create config spec and do lot of bulk operation. But it needs separate Venture Capitalist to sponsor the software, it needs admin and huge hardware. Oops..... not good for me.

Software world come long way CVS is quite popular not only among open source geeks but many corporates uses it. Later new kid SVN came and addressed some of the shortcoming of CVS and its quickly become popular.

DVCS or Distributed version control system which born out of necessity. It addresses inherent issue with centrally managed version control system. User can locally create a copy of repository and as and when send a change to server. So latency, delay associated with centralized management and network activities for every action on files got alleviated. It follows certain lifecycle, (Mercurial)

1. create a local repository - Clone/Init
2. Modify the code
3. Commit -locally
4. repeat the step 2 and 3 as long as you wish
5. Push the changes to the Server - If conflict due to another users modified code, you can resolve
6. Pull from the server
7. merge

Steps 5 and 6 involves the communication to the server. Others are done locally.

There few options in choosing DVCS,

1. Mercurial

2. Git

3. Bazaar

I am using Mercurial (Bitbucket's service) almost from 2007. It has good tortoiseHg GUI which integrates natively with Windows Explorer (Microsoft).

Git and Bazaar also suppose to be good and it is based on personal taste and difference over what it offers is minor but may be matter to choose one from another.

Google uses Mercurial for its Google Code. But it uses Git for Android. Sun Microsystem uses Mercurial extensively for Open Solaris, Netbeans etc. Mozilla also uses Mercurial.

Git is used for Linux kernal and lot of others.

Github is most popular hosting service which offers Git repository. Bitbucket is offering mercurial. There are quite few offers dvcs repositories for few bucks. It is really good for startups to use these (Cloud Service) service.

Python 3.0 AmazonS3 with Proxy Authentication

I just choose Python to do prototyping for Amazon S3. I just used 3 lines of http client module and viola, done with uploading. This is the same for URL Query String authentication. Just needs to know how Amazon wants their signature in the URL. Python almost built everything in.

I hit a wall, my POC whizz-banged when I need to fiercely encounter proxy. Devil is details. Yes, proxy needs to be authenticated. Since I used Python 3.0 I never able to find a suitable open source module to elegantly handle Proxy Authentication. I tried with socket and used CONNECT method pass through proxy but some how I was not able to send a proper HTTP bits past proxy. It looked like somewhere malformed.

I again went back to Url-lib with user id and password passed as part of the url itself (http://USER_ID:PASSWORD@http_proxy_url:PORT).

Below method prepares the proxy,

self.proxyURL = http://USER_ID:PASSWORD@http_proxy_url:PORT

def prepareProxy (self) :

proxy_handler = urllib.request.ProxyHandler({'http': self.proxyURL})

proxy_auth_handler = urllib.request.HTTPBasicAuthHandler()

opener = urllib.request.build_opener(proxy_handler, proxy_auth_handler)

urllib.request.install_opener(opener)

return opener

Then, opener can be used to fire a http request.

Another issue is, url opener is not http1.1 complaint. It didn't allow me to use HTTP PUT request. After a long goolging I found, it is possible by extending Request object,

I used something similar as below,

class PutRequest(urllib.request.Request):

def __init__(self, url, data=None, headers={}, origin_req_host=None, unverifiable=False):

self._method = 'PUT'

super().__init__(url, data, headers,

origin_req_host, unverifiable)

def get_method(self):

return self._method

All the 2 parts baked together, I am able to use python3.0.

Note: Amazon has its own python API but it didn't allow me to upload in blocks. It is current implementation needs to read the entire file and send it as binary data which is very in-efficient and not possible for bulk data.

iPhone HTTP Streaming

I saw an article about how iPhone OS 3.0 Safar handles Video. It is cool, it is now supporting HTML5 video Tag. Safari understands the <video /> or <object /> tag and launches the Quick Time player. I am able to move the scrubber bar. It is kind of cool. The codec supported is not Ogg Theora, it is H.264, please visit Apple site for more.

This will unleash lots of possibilities. Custom streaming or TV Streaming or whatnot - quite possible just with HTML5 and Http Server, may be Erlang will be the language to support needed scalability.

I checked the Video capabilities by converting the video into ".mov" format and created small html page with video tag. Safari understood the video tag and displayed the play button. Once I touch (click) the play button, it launched Quick time iPhone player and it played the Video. I am also able to move scrubber, it felt like a full streaming.

Technology brings Eventual Transparency

Earlier one of my blog post (Item 9), I said technology will bring Transparency- Eventual Transparency. Thats true. We never know who or which eye (Camera) is watching. Web has the power to remember. It never forgets. It replicates like Virus or Cancer and linked as never before - But it is good for people/community/country as it brings Transparency. The reason for this post is Google's CEO Eric Schmidt relflected My Thoughts in one of his Interview before ( I am not able to locate the Video, may be if i found I will update it later).

It is him, Mr.Eric Schmidt, is victim of the SOT (Same old technology). Yes, he is caught using Blackberry as his mobile device not Android (Nothing wrong but credibility will go down, credibility is directly related to accuracy of the data (it may be in any form).

Google Chrome OS

My 2 Cents on Google OS (Chrome OS)

It definetly going to game changer and how? Google started experimenting offline web through Google Gears. It then become standard as part of the HTML5. Standardization of Offline capabilities tells that it is very important feature everybody needs. MySpace utilized Google Gears and it is able to offload 20-30% of the load to the browser (read Offline sync).

So what is my Windows usage normally,

1. I check news - especially Tech News, politics and world news
2. I check emails
3. I blog
4. I document (I mean I create documents for official or personal)
5. I watch video and listen to music
6. Connect to TV or secondary monitor
7. Instant messenger
8. Video/Photo editing

Items 1 - 4, can be easily done, most of the time, in web. Without web all the above doesn't make sense. And 99.9% of the time people buy computer to connect to web. As Sun optly put "Network is computer".

Offline document editing, needs little bit of OS support. Office documnets should be opened through web browser and it has muscle to understand it (Java Script code). So if google is able to come up with simple idea of opening documents stored in the local file system, they can easily replace native word document editor. Or, in the open source world, open office already very capable, If google is able to bring more efficient version of open office, thats it. 90% of the problem solved. It also stongly ring the death knell to Microsoft's cash cow, office suite.

Google indeed has good photo editing (glorified picasa) and drawing tool (Sketchpad). So they just needs to spin the threads to make all this available.

Why Google needs OS?

Yes Google needs it. Because, Microsoft instigated it. It is trying hard past couple of years to invade Google's Search business. Its Bing and its forefathers taught Google that, if Microsoft is able to capture 20-30% seach engine market, what would be left for Google? It is obvious for Google that it would go into OS business. They got a great momentum from Android. Yea, smart phone is kind of fledgling market but Netbook is also kind of very much in the early stage. So targetting Netbooks and Smarbooks (Microsoft doesn't have product for ARM) , it will make a good enough dent in Microsoft OS market. So, I bet even 1% market share is 1% loss to Microsoft. Google knows OS, they very much successful in server class Linux machines. So, they will bake Linux into very much palatable format. I am eagerly awainting to see its coming.

I also able to see fumes and smokes all over the net.

1. Eric Schmidt should step down from Apple Board or
2. Microsoft will not allow Google to make Chrome OS Survive blah blah.

I am thinking half glass full in Google's case. Here is why,

For Eric Schmidt Step down fumes, here is my thoughts...

1. It is that Google released Android which competes against Apple
2. Google is going to release Chrome OS to compete against Mac OS

I am not thinking it is not really a good reason, because, It is same as Google has gmail and Apple has MobileMe; Google has Chrome and Apple has Safari etc etc. So, in theory, if you work on computers, you will find multiple applications/Components which is manufactored by varous companies. So whats wrong with it? It is good for Consumer and competition is good for ecosystem. More over, the space/people with which Apple address is ultimately different. But if they fail to innovate, they will perish. I guess, Android, in few years, going to make iPhone what Microsoft's Windows made Max OS in 90s (It is not because of best OS but because of strategy- seperating harware and OS apart and making harware partner to grow). But I don't think Apple will let this happening this time. Current market trend is Mac is gaining on Windows due to failure of Vista. So if you look at the space which Google Chrome OS addresses is Netbook and above (may be Notebook and Desktop) is different and Apple is not into that. But I want Google to go against Apple territory. It is good because, Apple will open up its iPhone OS because of Android or innovate more which will make on par or more with Andriod( I am thinking few years down the line) I still believe, they are holding the Google to stop releasing muti-touch feature in Android. I like Eric should step down from Apple and up Google's throttle to innovate more.

Microsoft should be having big troubles. It can't be give it Windows 7 free after all it is one of the cash cow for Microsoft. So whatever way it goes, it affects Microsoft as Google has gained lots of loyal user and customer and they ready to jumb in if Google release Chrome OS.

If Google executes well like how Firefox did with Browser, Microsoft surely will be in big trouble.

But it is very well possible with Google. The big advantage already with Google is 1) ARM - Chrome OS will run on ARM 2) Partners - Since it is always best to use free and best OS to make Netbook/Notebook so they will gain % benefit. Android also did the same. Who will hesitate to buy a computer which is $$ less and which has all the functionality they want?.

Web application one more capability

When I started heavily using Google documents probably from early 2006 or early (I am not remembering the exact date) it is kind of awesome. It is because, it enables power of collaboration, ie true collaboration. Most of the time, in my company, I am not alone preparing documents. Mostly multiple people will be working together, we normally use microsoft office and we will pass among ourselves and in the end we could have had more than 10 copies that sits and occupies dangerouly our measely corporate mailbox. So it address many problems,

True collaboration. We can always, instantaniously see the updates - so we modify and do the changes in no time and updated version sits in the same place with all the revision history intact. So beautiful.

Other tangible and intangible benefits There is no mailbox (inbox) cluttering, awesome no need to clean up mailbox. So all the documents need not be culled out and moved to network drive or backup or Document management system. So, it eliminates,

1. Backup media
2. conserve the mailbox
3. saves the history
4. And accessing the documents is now straight forward as its location is standard
5. Location transparency or ubiquitous access

There is the tradeoff as we need to deal with sterile HTML and its associates.

No need to install Office desktop application - This is the best thing

But I have been having a problem. If all the documents stored where it (Google docs or Zoho Docs) actually want to. But if I have a local copy of Microsoft Office document, how can I seamlessly integrate? I can do it by uploading them into Zoho or Google docs, but it is again not seamless.... So what is my expectation?

I need something like file association to the web application. I mean, how in OS image or text or word documents are associated with specific and special purpose application the same way I should be able to associate my office doc to google or zoho or any other future service provider.

How is can be accomplished?

I am just drawing the lines between already available technologies and technique.....

1. Firefox we can associate Email to Gmail or Yahoo or anybody else - but it is something different as the email address is redered in the html page but our docs is in local file system. Or I can think of browser will not only associated with HTML but also docs (already it can do image, probably video and audio after HTML5) and more in future. So we need to some how associate it to browser to specific web application which is capable of handling the tasks. So there should be browser handshake and web application handshake. It is easy and do-able.
2. How can I save the document back to where I read from? There are multiple approach. We can leave it to browser as it knows where it read from so it can save it back. It is kind of complex as it needs to know multiple things like other extra elements which "web application" added to render the document and other things which I am not able to think now. Second option is just thinking in line with Opera - It recently released a feature which makes browser as local http server. Ok good. And if we combine "Web application" and browser's http cababilites and thinking RESTful way, we can save the documents back. So it is kind of mash up in highlevel but it is ultimately do-able. Probably when we think more there may be better way to do that.