Since releasing my first app to the Mac App Store I’ve been thinking about how I wish I had more options available to me as both a developer and consumer. The following is my wishlist for the perfect desktop app store, basically it takes the best parts of Steam and the best parts of the Mac App Store and combines them.

OS/Vendor Agnostic – The perfect app store needs to work consistently across Windows, Mac, and Linux, and the only way to establish impartiality is to make sure none of the big vendors *own* the store. That said, in order to really take off it would probably need support from some of the major cross platform vendors like Adobe.

Platform Integration – The app store cannot be web only, it has to come with an installable store that manages your applications and lets you browse for new apps to purchase. On the flip side, web purchasing should also be enabled and publishers should be able to embed a framed version of the store in their website to make transactions more fluid.

Buy once, run anywhere – Publishers should be able to set options on whether applications are purchased once for all platforms or require small payments for additional platforms beyond the first one. Ideally you wouldn’t have to pay more for a different platform, but enforcing that rule may discourage publishers from putting in the time to port code.

Multi system – A single login should be usable on up to 5 systems.

Multi user accounts – Families should be able to easily add kids to their account and purchase software for them. Companies should be able to add and remove employees from a company account and assign software to different users. No need for separate company logins, everyone can just use their personal accounts.

Simple receipt system – Apple Mac Store pretty much nailed it on this one. Users don’t have to be logged in like Steam in order to run an app. The way it works is every application is responsible for finding and validating a digitally signed receipt when their app starts. Receipts are generated from app name, app version, and system id, and digital signed. Receipts can easily be downloaded when logging in to a new computer and easily voided when no longer valid.

Simple update system – Apple nailed this one as well, provide simple notifications that updates are available for certain apps, and then handle the process of installing that update.

Application upgrades – Publishers should be able to set options for users to get discounts on major upgrades if they own a previous version. Users should be able to enter serial numbers for existing applications to prove ownership of previous copies.

Cloud Storage – Apps can use up to 50mb of cloud storage per user, paid apps can choose to use a percentage of proceeds to pay for unlimited space for their users.

Developer Tools – Developers need strong guidance on how to package their apps and test them before going live. They should be able to download sample receipts for their app and test the installation process through the store itself before going live.

Freedom of Language – There should be no restriction on languages or technologies (think java) that are allowable in the store. Dependencies lists can be built over time into the store to help facilitate getting users the correct packages for certain apps to run.

I’m sure there’s a few other things I haven’t thought of yet, what would you add?

Home | Detailed Analysis | Benchmark and Rendering Engine theory

Breaking down the test

Before creating the reference design I came up with the following rules to influence the overall benchmark. I wanted to make sure that whatever I created, the results would create meaningful data for both end designers and developers as well as platform engineers at Adobe or Sun or Mozilla to be able to use.

1. The max draw rate for an application is fixed at 60 frames per second. Since mainstream LCDs are fixed at 60 hertz, most applications have no need to render above 60 frames per second and in general, the OS will throw away any draw requests it receives above that rate. While there are a select number of people still using CRTs and TV manufacturers are pushing 120hz technology, it’s a waste of cpu to generally create animations above 50 to 60 fps. With this in mind its important that no benchmark comes close to this cap. Its very difficult to visually distinguish between an animation running at 50 fps and one running at 60 fps. So the closer I can force the framerate down to 10 fps, the more visual difference there will be between the technologies. Unfortunately I underestimated the optimization capabilities of the Flash engineers.
2. Animations should run on fixed time as opposed to relative time. One of the important distinctions in the GUIMark test case is that at high framerates, the test runs smoother, not faster. The heartbeat-style animation of the test will always take 2 seconds to perform, whether running at 5 fps or at 50 fps. This gives users a good idea of how smooth they can expect their own transitions to animate.
3. The test case should be hard to optimize. I took a WYSIWYG approach to the benchmark so that there are no clear optimization paths. Each technology is limited in some way by the effects that are required in the reference design.
4. Code execution should have a negligible impact. The user code in each of the tests has almost zero impact on the test case. This is very evident when observing Silverlight running on javascript vs C#. My prediction was that I’d see no difference between the 2 code paths and it turned out to be right. I wouldn’t expect to see any improvements when porting the Flex project into a pure AS3 project either. This jsut means that the rendering engine is taking up 99% of the cpu time and we get a pretty good comparison between the different environments.
5. The rendering pipeline should remain saturated. If the fps is below 60 and the CPU isn’t sitting at 100% while running the test then there’s either something wrong with the test or theres something wrong with the runtime.

Understanding 2D Rendering Subsystems

The rendering subsystem is typically very complicated and includes alot of interworking logic to interface with user code, internal display objects and OS apis to optimize the final output. You can generally expect the 3 following pieces to be present in one way or another inside each runtime.

• Layout engine: The layout engine is responsible for calculating the location for objects to be displayed. This includes performing matrix transforms like scaling, rotations, and object positioning, as well as positioning text within a flowing text block: It is also responsible for calculating the entire parent child display object chain.
• Drawing engine: The drawing engine creates internal bitmaps of each discreet display objects. It applies colors to gradients, determines antialiasing rules, and draws fonts.
• Composition engine: The compositing engine takes the results of the layout engine and drawing engine and sends a final render to the OS to draw. The compositing engine determines the z-index order of objects, calculates final pixel values for alpha overlays and applies clipping rules.

Optimizations within the render subsystem usually come from the interactions between the different engines. For instance, the drawing engine has no need to draw outside a clipped area or beneath other z-index objects, and the compositing engine doesn’t need to update areas of the display not changed by the layout engine or draw engine.

Home | Detailed Analysis | Benchmark and Rendering Engine theory

Win Browsers: Internet Explorer 7 | Win XP | Average FPS 28.11 IE 7 pulled off the fastest rendering for the HTML test among Windows based browsers. the framerate chart to teh right baffles me more then any other chart listed on this page since the highs and lows are so rapid and don’t appear to correspond to any direct actions within the test.

Win Browsers: Firefox 2 | Win XP | Average FPS 19.35 Firefox pulled off the slowest average fps for the HTML test on Windows. Maybe Mozilla could use a chunk of that money they’re supposedly raking in to hire someone to work on their rendering engine.

Win Browsers: Opera 9.27 | Win XP | Average FPS 21.34 Opera faired slightly better then Firefox on the Windows HTML test, but the graph really shows just how much different the renderers are working in the 2 different browsers.

Mac Browsers: Safari 3 | OS X 10.5 | Average FPS 18.20 On the Mac side, Safari just barely took top spot among the other browsers in the HTML test. I had really hoped to see a better showing out of the Safari team when going head to head with Internet Explorer under Windows on the same hardware. Consider yourself officially “On Notice” Safari.

Mac Browsers: Firefox 2 | OS X 10.5 | Average FPS 12.91 Just like Windows, Firefox 2 came in last in the HTML test for Mac. Not much more to add apart from whats already been said.

Mac Browsers: Opera 9.27 | OS X 10.5 | Average FPS 17.22 Not too shabby there Opera team! The framerate exibited on the HTML test under Mac Opera was so close to Safari that any number of possible test conditions could see Opera pull ahead under different configurations. Not bad considering their not as close to the OS as the Safari team.

Flex 3 | Win XP | Average FPS 46.08 Flex 3 on Windows handily beat out all other runtimes tested on both Windows and Mac. Even though Adobe pretty much rewrote the Flash Player for version 9, it looks like they incorporated the past 10 years worth of rendering research into the latest player.

Flex 3 | OS X 10.5 | Average FPS 8.01 Flex 3 showed the biggest difference in frame rate between Mac and PC. I may be mistaken but its possible that the Flash player for Mac is still optimized for the PowerPC processor and hasn’t been moved over to the Intel base. The big dips on the graph occur when the center text block is at its largest size during the test run. The layout calculations are pretty significant at that point.

Java 5 Swing | Win XP | Average FPS 19.37 I had always read that Java could spank Actionscript when it came to code execution. I’m not sure if thats still true but I do know that Swing doesn’t hold a candle to Flash’s DisplayObject architecture. These numbers had me questioning whether I had written my Swing code correctly. It might not be as optimized as it could be, but I’m sure alot ofthis is out of teh hands of user code.

Java 5 Swing| OS X 10.5 | Average FPS 7.19 Swing has some serious issues running on Mac under Java 5. For those of you able to run the benchmark under Mac, the “Pause” and “Run Test” buttons are really there, you just can’t see them until you click them, so click around the space below the fps timer and you eventually hit them.

Silverlight 1 | Win XP | Average FPS 9.12 Of all the tests built for GUIMark, I was looking forward to building the Silverlight tests the most. I haven’t had alot of reasons to play around with the SDK and this was the perfect opportunity. From a framework perspective, XAML is a worthy competitor to Flex even though the component architecture could use some work. From a rendering perspective, Silverlight has a LOT of growing up to do. In fact, the framerate listed above is a lie. In Silverlight 1 users had to rely on the Storyboard object to fire timed events, but those events would occasionally queue up and fire twice in a row before rendering the output. Thats why on the chart you see those massive spikes in framerate. The “real” framerate is more in the range of 5 to 6 fps.

Silverlight 1 | OS X 10.5 | Average FPS 5.25 One good thing you can say about Silverlight is that rendering performance is closer together on Windows and Mac then the Flash player is.

Silverlight 2 Beta | Win XP | Average FPS 7.59 I felt that Silverlight deserved a fair chance to compete with the other environments and that Silverlight 2 with compiled C# was the truer environment that Microsoft wants to push. It also gave me a chance to test the theory that the GUIMark testcase wasn’t constrained by user code, but instead the rendering engines built in to each runtime. Instead of using the Storyboard object for creating timers, i was able to switch to the TimerDispatcher class. Although you don’t see the massive spikes anymore, Silverlight 2 is still prone to queuing up timer events and dispatching 2 at a time and you’ll notice in the chart where the framerate goes above 60 because of this problem.

Silverlight 2 Beta | OS X 10.5 | Average FPS 5.38 The Silverlight 2 framerate on Mac is practically identical to Silverlight 1, as is the chart. Again this shows just how little of an effect there is between dynamic and compiled user code when it comes to rendering performance.