I’ve been talking about load-testing ASP.NET applications, and what it’s like when you fail. Well, now I can finally talk about why I’ve been thinking about all this stuff. I just spent the last two weeks talking to people about our launch and getting feedback from analysts about the Strangeloop AppScaler, and now I can finally talk about it in public!
Here are the basics: You already know how application tuning impairs the development process. Not only does it take a long time for pretty limited returns, but it takes you from this lightweight, fast ASP.NET development process—the whole reason you started using ASP.NET in the first place—to this much more ponderous endeavor, where every piece of performance tuning you do places new requirements on everything else you’re coding moving forward. Well, the Strangeloop AppScaler basically takes that entire application tuning process and puts it in a box. It’s a very, very cool thing. But now that we're out in the open, what I really want to talk about is how we got here.
It all started with looking for a better way to do session. Everybody’s talked about session, and everybody knows that it could be handled better, but nobody had actually done it. The default, of course, is in-process session. Since we all start development on a single web server, in-process makes sense. But as the application becomes more important, more web servers are needed, and the idea of going out-of-process comes up.
Microsoft provides two out-of-process approaches. One is using SQL Server, which you likely already have, since it’s where you store your data. But SQL Server is kind of overkill for session, since you're just storing a blob of session data there: you don't really need the power of SQL Server for that. SQL Server is reliable, but slow. The alternative is State Server, which is substantially faster, but isn't reliable and generally isn't a great bit of software.
And switching session methods is pretty trivial, since all you have to change is the web.config file. Although one issue people occasionally run into is that they haven't marked their objects for serialization. In very rare cases, they can't serialize their objects, but for the most part, it’s just about setting properties correctly.
Typically, most people deal with the issue by leaving session in-process and using a load balancer that supports sticky session, where the load balancer uses the ASP.NET cookie (or IP address) to *stick* a given user to the same web server each time they visit the site. While it certainly solves the problem of session, it undermines the value of a web farm. If the server goes down, the session’s lost. Some servers can end up much busier than others, so you aren't really balancing your load. And server updates tend to be a major pain.
To really load balance, to get the full advantage of your web farm in terms of performance and reliability, you need to get the session data out of the web server and go out-of-process. When you do that, you can load balance properly and go to any web server you want, but it means that session processing takes longer. So originally, our mission was to really look at session and figure out a way to get in-process performance but with out-of-process flexibility.
When we did all the math to figure out exactly why doing session out-of-process was so much slower, we found it was network trips were a major part of the processing time. Every request/response pair with out-of-process session means two additional network trip pairs: to fetch the session data at the beginning of the response computation, and to write the modified session data out at the end of the response computation. But the only reason all these network trips happen is that the request travels all the way to the web server before the server realizes it needs session data. So we thought, “What if we put the session data in front of the web server, so by the time the request gets to the web server, it already has the data?”
That’s what AppScaler does (well, one of the things it does). As a request comes in, it passes through AppScaler, and AppScaler says, “I don’t care what server you’re going to, here’s the session data you need.” Then it attaches the session data onto the request. When the request arrives at the web server, the session provider strips the session data out of the request and the page processes normally. When it finishes computing the response it attaches the session data to the response and sends it back to the browser. On the way out the response passes through AppScaler, and AppScaler removes the session data and stores it away in its network cache, and everything proceeds normally from there.
So suddenly, we’d eliminated all these extra network trips, but we were still out of process, so you still have all that flexibility. Pretty cool, right? Then we took it a step further and said, “Gee whiz, since we’re already here doing this, why don’t we just do viewstate too?” As you know, viewstate can get totally out of hand, typically due to the use of third-party controls, which is why the really performance-conscious sites don’t use third-party controls at all. And giving up third-party controls means either slowing down your development process to create controls yourself, or just not using all the controls that you otherwise might. With AppScaler, you can use all the controls you want (within reason). It takes that viewstate out of the page before it goes to the browser, so you don’t pay the performance penalty.
So fixing session and viewstate were the first features of AppScaler, and the results were pretty impressive—we were really cutting down page sizes and seeing substantial performance gains. And that’s when we had the big realization: Now that we’re sitting here in front of the web server farm where we can see all this traffic, there are all kinds of smart things we can do to optimize the performance of ASP.NET applications!
Fixing browser caching was low-hanging fruit for us. With browser caching, you mark various resource files (images, js and cs files, for example) as cacheable at the browser, normally with some sort of time limit (a day, a week, etc). Once the browser caches those items, it won’t request them again for as long as the cache is valid. That gives substantial performance gains since you cut down a lot of the resource requests that make a web page work.
The downside to browser caching is when you go to update the website. Unless you’re extremely careful, you can end up with browsers messing up your new pages because they use cached items instead of new items. And of course the pages that get messed up are the ones the CEO is looking at, because he hangs out on the site all the time and has everything under the sun in the browser cache. In my experience, people abandon browser caching after an event like that, and never use it again.
AppScaler fixes browser caching by dealing with expiration properly. First off, you specify what to cache in AppScaler, so that you don’t have to fiddle with settings on your web servers. AppScaler just automatically marks those resource files for caching as they pass through on the way to the browser. But then the really clever bit comes into play: AppScaler watches the resource files on the web server so that when there is an update, it sees it and knows the underlying files have changed.
Once AppScaler knows a resource file has changed, it dynamically renames it in the request/response pairs so that the browser doesn’t have it cached. It keeps up the renaming until the cache expires. So suddenly browser caching doesn’t cause problems with website updates.
Our experience with ASP.NET has demonstrated again and again that caching is king. And when we studied the potential of caching with AppScaler, we realized that self-learning caching was the number one performance return we could offer with this idea. Being between the browser and the web farm is the perfect place to cache and to coordinate cache expiries. As a developer, you know you have to cache, and you can write code to do it, but it’s a lot of programming, and it changes the way you have to code going forward. More than that, you have to figure out what to cache. You might guess wrong. Or more likely, because of the time and effort involved, you’re probably only going to cache a few things that are obvious.
AppScaler Response Cache evolved from that experience. It started out as a system for monitoring traffic, looking for where the request/response pairs match, and how frequently a response is different for a given request. It looks at parameters, such as querystring and POST elements to identify different requests. So by watching all traffic going to and from the application, AppScaler learns what to cache, and when to expire it.
Based on those recommendations, you can tell AppScaler to actually cache the items, or you can put it into an automatic mode, where AppScaler will cache what it thinks it should. This automated caching feature is incredibly useful for dealing with Slashdot or Digg events, where suddenly traffic is up 10 or 100 times.
But ultimately, the real advantage is the lack of coding – writing caching code in ASP.NET works, but it slows down the development cycle going forward. AppScaler gives you the same benefits, but without the impact on your development.
Now for the record, if all of this sounds very straightforward, it’s because I’m just giving the highlights here. Making all of this work together has been an extremely complex, time-consuming project. Also, while I’m really excited about it, I want to be clear that this is not going to fix every problem. If your pages are a megabyte apiece and half of that is viewstate, for example, we’re going to have a tough time helping you at any significant level of scale. You’re still going to have to do some basic tuning. But it’s when you get into the really exotic tuning, when you’re doing these miniscule kinds of tweaks and breaking pages down fraction by fraction to find out where you can squeeze a little more performance out of it—the stuff that really impairs your coding more than anything else—that’s when AppScaler can really help you out. And this is just a subset of the things it can do. I listed four features here. There are more than twenty others on the books today, and the list keeps growing.