SaaSGrid: Decoupled, shared storage in grids

When you're working in a grid environment like Apprenda's SaaSGrid, many people are uncertain how to deal with the need for shared state and shared binary content.

This is not unlike Amazon's EC2, though it should be noted that EC2 and SaaSGrid are not in the same category of offering -- you can run a SaaSGrid environment on top of Amazon's IaaS offering: EC2 VM images. SaaSGrid itself is a distributed application server and runtime for SaaS. EC2 is a cloud of virtual machines, part of Amazon's larger AWS suite.

With EC2, you do not have persistent storage locally on your virtualized AMIs -- when your AMI reboots or powers down, all local state is lost. Instead, Amazon offers Elastic Block Storage (EBS) for non-volatile storage of data and content used when bringing your AMIs online, database repos, etc.

Likewise, you cannot assume persistent local storage on any given SaaSGrid node -- any given request to SaaSGrid may take multiple paths to reach its destination, and any given node may leave or join the grid environment at any time. This is why SaaSGrid application development, and most other SOA architecture, best current practices dictate stateless design and implementation as much as possible. Some amount of state can be persisted in an out-of-process cache, such as memcached, but there is still often a need for a binary content repository accessible to all nodes.

I diagrammed this concept over on the SaaSGrid Developer Blog yesterday.

Philly Code Camp - Gold Sponsor!

I'm happy to announce that not only will I be attending Philly Code Camp 2010.1, but my new employer, Apprenda, will be there as Gold Sponsor, specifically of the Architecture track.

We may or may not be doing a presentation on SaaSGrid. Unfortunately we got our ducks in a row at a bit of the last minute on this, though we are eagerly awaiting a last-minute presentation slot to open up, whether that be due to a presenter's sickness, travel difficulties or sudden enrichment of their bank account. *cough* ;) As it is we missed NYC Code Camp 4 and New England Code Camp 13 :( We are also looking for future Code Camp sponsorship and presentation opportunities!

I believe we are going down to Philly the evening of Friday the 9th, and possibly considering staying as long as overnight Saturday. I don't believe I've ever been to/near Philly before ... perhaps driven on a nearby bypass?

One more note, SaaSGrid may make an appearance at CloudCampNYC on April 20th as well. TBD.

All coming together ...

As I approach the start date for a new fulltime role and work to wrap up outstanding consulting obligations, it's been a crazy busy few weeks with not enough sleep, food or fresh air! However, it all seems to be coming together.

A quick snapshot of my morning, the culmination of days of grinding it out:

• We got StorageByMail's revamped, Rackspace Cloud-converted site launched.


• We've been prototyping a creative multi-product stealth-mode SaaS offering for a NYC-area startup. I've been acting in both consulting CTO and lead dev roles, and have had the first chance in a while to work with Java and Tomcat applications commercially.


• My work continues on a near-beta Twitter business services app; that app is now growing to include LinkedIn and Facebook as well.


• I continue work on a still-stealth multi-social platform biz app I'm partnered with a great team out of NYC on. (Public beta by mid-March!)

As part of these efforts, in the last three weeks I have:

• Started using ADO.NET Entities and LINQ to Entities instead of LINQ-to-SQL. Entities seem gosh darn piggish when it comes to RAM consumption -- even when managing repository lifetime in what I understand to be best-practice fashion. I don't recall that issue with LINQ-to-SQL. Generating the model/updating the model from the database is painful when it comes to some relationships that seem to require manual removal every regeneration. I hope Microsoft gets it right in 4.0.


• Got over my fear of lambda expressions, at least fairly simple ones. I still need to fully grok compiled functions. Baby steps.


• Realized yet again how much MS ASP.NET AJAX stinks. Obscure and painful to work with.


• Written a .NET consumer for the Twitter Streaming API. I haven't come across any other full implementations in .NET, though my basic incremental HTTP consumer is modeled after others' examples. I will probably publish this to Google Code soon.


• Started working with Google Buzz.


• Written a .NET PubSubHubbub subscriber client and callback handler. This has been published to Google Code.


• Worked with the new Buzzzy API on top of Google Buzz. Useful API for my needs, but the 250 requests/hour limit is kneecapping.


• Created my own Google Buzz firehose by crawling 4.2M Google Profile IDs (crawl in-progress now, finally got a nice multi-threaded crawler purring) and subscribing to push notification for all of their Buzz feeds through pubsubhubbub.appspot.com. Unfortunately 4.2M is still not the complete set of profiles, but it's a great start.

Haven't yet had time to catch my breath or catch up on sleep ... but there's light at the end of the tunnel! More launches/betas to come! And more aggravation, learning and ... "opportunities" along the way I'm sure :)

Very Interesting SaaS Consulting CTO work

For about a month or so now I've been serving as consulting CTO for a stealth mode NYC startup in the SaaS/SCM/BI space. We've been researching and vetting a suite of products for integration as a SaaS offering. Lots of vendor phone calls, trial installs, conversations with industry leaders who really know their stuff.

My client has now inked some licensing paperwork. With the clock ticking on some expensive software licenses, it's time to move on to the rapid prototyping stage. We're looking at utilizing VPS (virtual private server) images -- perhaps "in the cloud" on EC2, perhaps at a more conventional datacenter -- to allow quick and flexible server provisioning as we explore the caveats of integrating these disparate architectures in varying configurations.

This prototype will serve multiple purposes: as stated above, we need to get to know the gotchas of operating and offering these products as a suite, as well as gain domain- and product-specific knowledge. We need to understand the strengths and weaknesses of various arrangements of the products and the supporting/surrounding infrastructure. The CEO of course needs something to put in front of potential investors and early clients.

I am really looking forward to this -- it's going to be a lot of fun. And, as always, a great learning experience.