This book is a work in progress, comments are welcome to: johno(at)johno(dot)se
Back to index...Goal: casual "surf-style" gameplay, completely transparent downloads.
1) Seed-idea:
-webpage links to a minimal executable (seed.exe) which is hardcoded to simply download another exe (game app)
-uses entity-tag based cache / versioning
-game app loads all assets via Core::RemoteFile? (blocking)
-optional progress callback for Core::RemoteFile::open()
2) hUb-similar idea:
-Seed (as in 1) is one time download, installer, is a browser running GML
-GML supports launching syncing/launching external exes (the games)
3) torrent-like approach:
-webpages host .seed files, which include an url to the game app
-end user installs seed.exe (analogy is a torrent client, i.e. Azureus), and associates to .seed files
-in this setup, use a normal web browser in place of hub, just host links to .seed files in order to play
-seed.exe gets the url of the .seed file, gets that file via http, which gives the url to the game exe
-since the idea is never to actually download and store .seed files, the contents of the .seed file may change (i.e. moving the hosting of the game)
080523
I have implemented the entity-tag based cache. It appears that the web server updates the cache each time a file is changed/overwritten, which was the expectation and the basis of the implementation. Yay!
The idea worked out basically as planned, i.e. file loading routines in the app need to be changed to support the cache, but this is minimal. File loading is still completely blocking as before, and in principal you still get a FILE* back from the routine that you call instead of fopen().
In my case there were other compelling reasons to have a file abstraction (core::File), but again, all that is really needed is a global method that returns a FILE* pointer (or whatever method of loading files you prefer). I think that the switching on and off of the cache should be centralised, and clients of the code can't see a difference.
Of course this method is slower than loading straight from a local HD, even when the cache is up to date, as the implementation needs to do a HEAD for each file to get the current entity tag and compare it to the local cache. There is also no cache purging at all, so this is up to the user and/or developer.
I support a callback interface for the messages, errors, and download progress, which in UfoPilot2 (the test project) shows some text and draws an animated ship. The idea is to integrate it into the visual style of your game, since it is probably a good idea to show that the game is running, especially for the initial run, which is basically an install where lots of stuff gets downloaded.
The initial run of UfoPilot2 (just into the menu where the game is responsive) is about 16 MB, compared to about 90 MB when the full game is downloaded. I think this is a big win, as you can make a really nice and compelling renderer for asset validation (core::HttpRepository::IMessenger implementations) that fits well with your game style.
Some thought will have to go into where and in what order you do your asset loading, but I think this is a good idea. There are several places in UfoPilot2 where I am loading redundant data that is already cached elsewhere, and using this thing really get's me motivated to clean that stuff up. I suspect that it's kind of like running off a slow console DVD or similar. I ended up changing the order of asset loading in order to get the fonts faster (so that the progress text would show up). I don't really see this as a problem, as the previous order of things was purely arbitrary anyway, and again, lots of cool ideas for compelling and stylistically coherent "loading/installing/validating" callback implementations come to mind (including streaming music, yes, the callbacks occur frequently enough as to allow you to update your stream without having to resort to multithreading).
Indeed, a main point of this is that it should NOT require multithreading or any kind of pre-cache/pre-download of content, which of course is VERY tempting, but contrary to the whole idea of load on demand. Thinking along those lines I guess that all access to assets should maybe therefore be abstracted, i.e. you always request an asset by name of the file or similar via some factory method, and you get for example a ddvideo::Bitmap back (you do this every single frame). This in turn leads to either explicit or implicit cache purging / garbage collection, but it is an interesting thought.
Right now there are issues with timers (Arcade mode tries to start right away, and get's big elapsed time deltas which sometimes crash, the ship outside the map etc...), so a purely browser-style approach is probably difficult for most games.
Again, the main point is keeping the download as small as possible, which in turn lets the player get into the game quicker. The browser analogy again; a user doesn't have to cache the internet to be able to surf the web a little. Load on demand etc.
I should try smaller games, like AstroCreeps or Z-Fighter, which load and cache all assets at the beginning of the program.
No work on launchers as of yet, still evaluating the above noted alternatives.
080619
Yesterday I did a test of a launcher. This was a small command line program that was hardcoded to GET a so called "app info" file, which is a simple tag delimited text file that includes the base url (directory) of a game, the name of the executable, and an explicit reference (relative to the base url) to each asset required by the game.
The "app info" file was implemented in a few lines of php, explicitly listing the exe and the base url, and then using php filesystem methods in order to list all the files in the base url directory (no current support for recursion of subdirectories as some games may require).
After parsing the "app info", the launcher checks the local cache for all of these assets, and finally launches the named exe.
The same entity-tag based scheme is used for the cache in this case, but since the game is not here exposed to the fact that it is downloaded or anything like that, we need to store the assets as their correct filenames on disk (the game is hardcoded to reference them as such). For this reason, an in-memory mapping from asset filename to entity tag is kept, and persisted using core::SingleFileContext to disk. The idea is that this asset cache is global for the whole system, and lives across several launcher executions.
I have tested with AstroCreeps, a game that doesn't have very many or very large assets. A better test would be to try UfoPilot2, and see if the user experience is better than before. I would expect it to be, since even though the launcher must do a HEAD to check the tag of each asset prior to launching the game, once this is done the game doesn't do all the (potentially) redundant HEADs at runtime. Of course, in the case of UfoPilot2, many of the file loading operations could be cleaned up, especially for front end graphics.
In the end, much of this is a two edged sword. Much is dependent on legacy expectations of games. That the remote enabled version of UfoPilot2 doesn't assume anything is the same ever might be a bit overkill, one could for example assume that each file only need be checked once per execution, keeping an in memory asset status map. One might think that pre-caching everything before application execution would be optimal, but then we're back at the case of having way more assets than you need for a game like UfoPilot2 (90mb instead of 16mb).
Reading this back, the algorithm for checking caches can be improved for the launcher case, since we are looking to see if actual filenames are present on disk, not files named as the entity tag: