-- zipFS has several limitations...
--- in a general way, seekg doesn't work... so getting a file's size needs to be done through JFileSystem.
--- getLine on files open with zipFS doesn't work so great. Not sure if it is a normal issue because files are open in binary or not... JFileSystem therefore offers a "readIntoString" function that needs to be used instead of the usual "getline" technique. However getLine can then be used on a stream connected to the string.
-- tested on Windows and PSP, I also made sure android still works, but haven't tested zip support on Android.
-- I tried to maintain backwards compatibility, but this might break on some platforms, if I broke some platforms and you can't find a way to fix them, please contact me and we'll figure something out
-- This removes wagic::ifstream. I didn't reimplement the securities that were involved in this, apologies for that. Might be useful to reimplement such securities in JFileSystem
-- I haven't tested options/profiles in a deep way, it is possible I broke that.
1) prevent the textureQuad map from containing references to cards when we're returning a temp (card backdrop) image in the threaded version.
2) we had two copies of the pspicon image in the cache (ie the regular & managed). Changed the reference in the DeckMenu code to use the managed instance.
3) Fixed a problem with a default constructor param that would set RESOURCE_LOCKED on certain images. I noticed that the 'NEW' image was locking itself in the cache because no resource modes was being passed in, so the default constructor param was setting it to locked. This seems like a bad idea - if you're not explicitly locking a resource in the call, that shouldn't be the default behaviour.
Also added an assert safeguard to catch the possibility of 'overlocking' an image resource, as this probably means the image fetching routine is being abused somehow.
Turned on the threaded card fetching code for win/linux. PSP runs unthreaded. There's an easy toggle for switching which mode the app runs in: check out WResourceManager's constructor.
To fully appreciate the difference, try going into the deck editor without these changes, and use the arrow keys to navigate around (esp. up/down, as it loads 7 cards at a time). Then try again with these mods, you'll see the cards flicker briefly to the back card image and then load as they scroll onto the screen.
This btw points out another circular dependancy between the texture and the JQuad - a texture owns a bunch of JQuads, yet the renderer uses JQuads and always assumes that the texture is valid. We're going to need to add more defensiveness to JGE to protect against this.
Other changes in this check-in: WResourceManager doesn't derive from JResourceManager anymore. It actually didn't require anything from the base, so I killed the dependency. Also cleaned up the notion of a WTrackedQuad in the WCachedResource - it didn't need a separate class, just a better container.
I've build this & tested against PSP, win, linux, QT (linux). I haven't tried against iOS and QT Win, or Maemo. If these other platforms are broken, I apologize in advance! - I'm hoping it should be fairly simple to put them back into play.
finishing up my reformatting of the source from November/December following the guidelines that were posted.
some extra things I added:
* Any empty virtual declarations were kept to one line.
* Enums were split up into separate lines to promote uniformity across all headers. ( each header file had a different style for enums)
-Attempt at fixing issue 538. This is a wild guess, but basically 1 000 000 bytes is NOT enough to decompress a 512*512*4 texture (32bits png). We need at least 1MB (1024*1024) AND some additional space for the decompression process. Based on that, bumped the 1 000 000 (minimum contiguous ram necessary to start decompressing a picture) to 1 500 000. I've had pretty good results so far but this needs confirmation
- GameObserver <- changes from Win to *nix carriage returns.
- fixed the debug framerate calculation / drawing code (consolidated duplicated functions & variables mDelta vs a win-only mDeltaTime, no reason to have two), repositioned it to draw on the bottom left (it was colliding with other debug text for the cache info).
- for debug testing purposes, added a #define override to force the image cache to work with less memory (I've picked 8 megs, which is what the psp seems to use - the normal default on win/linux is 20 megs).
(I did a cursory check to make sure chinese still displays correctly - at a glance, I'm seeing what looks correct to someone who doesn't read the language :) )
I'm also noticing that the GetQuad(int) variant never seems to get hit, so I suspect that the ID lookup map is redundant. I left it alone as the JResourceManager base class forces the need for the function; I need to spend more time looking at just how much of JResourceManager we actually use at this point.
-reduced SFX Quality. This seems to fix the Bug with samples getting silent on the PSP (issue 112), and improves loading times. If you think this is unacceptable, let's discuss it
- AI now should play fetchlands' ability, although still not "efficiently"
- Fix a bug where the game would crash if a card has a valid alias to a hardcoded card that is in no _cards.dat (starwars mod cards aliasing a MTG card)
This needs some impact testing, to be certain it doesn't noticeably slow anything down. In a future version I'd like to memoize checking for zip file existence in each "Res/sets/XXX/" directory once during boot, and in the themed directory whenever switching themes.
* Removed all calls to Release(JQuad*).
* Updated flatten(). Prior flatten was buggy beyond belief.
* Done some extensive testing, but if this causes more trouble than it fixes, we'll have to revert. It's too close to release time.
* The alternate theme is just useful for testing themes, but please comment on the changes to the main theme.
* Options reading and initializing separated from UI.
* Options UI abstracted a bit, made more easily extensible.
* With some extension, the options UI could be used as the basis for more complicated themes, if we load a file and use it to do stuff like GameStateOptions::Start()
- fix issue 65 (quads when no image load slowly in shop/deck editor)
- Possibly fix issue 92, please let me know if it reproduces
- Fix issue 97 (Deck editor: weird behavior of deck display)
- Fix issue 39 - please verify
- Issue 56 can probably be closed as well
- Fix issue 86
Exception handling in the cache means that Wagic runs slightly, but noticeably slower. However, it also means that the cache works the way it was intended to, and should prevent all sorts of awkward errors I introduced because things were half-allocated or worse.
I've learned my lesson from tonight, and once I get this stuff put back together without exception handling I think I'll self-impose a manditory 24 hours debugging time before any commits. Sorry for the inconvenience.
If for some reason you can't compile with exceptions, r862 should be completely without them. Again, wow, this was stupid-- I hadn't even realized exceptions were off for like 24 hours, as the compiler wasn't producing warnings.
Apologies,
—Jeck
* Restored OptionItem saving fix.
* Booster duplicate replacement was picking random cards from the wrong set. My fault, but I've fixed it :)
* The problem with cache had nothing to do with memory fragmentation, but I've switched to an array rather than map<> just in case. The actual issue was that in GameStateDeckViewer, I'd given the cache unlimited space, thinking that AttemptNew would recover from any bad_allocs. Unfortunately, the image loading routines and similar stuff called by various implementations of WResource::Attempt() could fail halfway through, leaking memory.
The temporary solution is to set a proper limit (8000000 px, more or less) and (in case we still run out of memory) test to make certain CACHE_SPACE_RESERVED can be malloc'd/free'd. The proper solution would be to keep byte-perfect records of memory used (right now we're kinda fuzzy-- we track pixels per image and bytes per sound, but not the space for jquads or other incidentals) instead of testing a malloc, and potentially cleaning up all calls inside of Attempt() so they fail without leaks. That's what I'm working on now.
Still, it's nice to have identified the problem. This version of the cache should be fully functional, it's just a bit inelegant.
* mtgid now defaults to 0.
* TextScroller will not update when empty.
* Cache now moves WCachedResources we're finished with to a garbage pool for later use (to reduce memory fragmentation).
* Demo still crashes... but I'm thinking that has to do with fragmentation, not a leak?
* Numerous cache fixes, reduced filesystem access.
* Cache fails very gracefully.
* Cache is now a templated class, with individual caches per data-type.
* Much easier to extend.
* Extensively debugged. Try compiling with -DDEBUG_CACHE.
* Caches limits can be set on a per-item basis.
* hgeParticleSystemInfo are now cached, mana particles now fall back to defaults.
* Samples are not cached, but track filesystem misses using the cache backbone.
* Avatars are cached. Default baka avatar is now baka.jpg, to prevent collision with player.
A note on the retrieval types:
RETRIEVE_MANAGE puts a resource into a seperate, managed resource list.
Managed resources are guarenteed valid for the lifetime of the program.
Retrieving a managed quad promotes the associated texture to managed. Don't do that by mistake.
Calls to Resources.Refresh() will attempt to reload managed resources in place.
RETRIVE_LOCK (and by extension, RETRIEVE_VRAM), returns a resource after locking it.
A resource may have many locks, and remains in cache until they are all released.
If the resource is managed, it returns it unmodified.
A note on quads:
Unlike all other RetrieveWhatever() functions, the default behavior for RetrieveQuad is RETRIEVE_LOCK. Worse, Release(JQuad*) is slow, and will /always/ release one lock from the associated texture.
There's a long and complicated explanation for this, involving support for live relinking of textures to existing quads, but basically what it means is that we only use RetrieveQuad for quads we intend to store and later Release(). If a temporary quad is needed, the preferred method is to use NEW JQuad* and SAFE_DELETE with RetrieveTexture(). RetrieveTempQuad is also provided, but is only guaranteed until the next call to the cache.
Note that RetrieveCard has none of these problems.
