EHPC

[rolfstenholm]

Cool game, but what kind of compression are you using on the images in the game?

[Fujisawa4654]

Well since no one else has responded for a while, I'll give a little information about it.

First of all, all artwork was extracted from the Sega Saturn version of this game. The main problem was, the pictures were actually 15-bit pictures, so conversion to 16-bit was crappy at first. Rob spent time converting the pictures to.. uhh.. not look as crappy...

[rolfstenholm]

What I want to know is how to unpack the image data in EHPC? For instance how does the source code look for the unpacker ?

[Fujisawa4654]

That's out of my hands I'm afraid... Perhaps Rob will tell you but what I told you alredy is the best I can do.. without permission >_>

[Captain Southbird (EHOL Creator)]

The original Saturn images were uncompressed, only mixed up bits for encoding.  But naturally for internet downloading's sake I wanted a compression format.  Early versions simply used the old PCX format, but I found much better compression using a generic implementation of the opensource ZLib library.  (The same basic compression algorithm used for the PNG format.)  However, the image format itself is not PNG, but a simple format that merely stored the image dimensions and ZLib compression data.

Finally for neatness sake all of the external data files are packed in a simple "glob" format I made up a long time ago, known as BOBGLOB.

If you're really interested in tearing the game apart for some reason, I'll be glad to go into this further.  But first at least tell me why.  

[rolfstenholm]

I wanted to see how much work it is to make a java version running ehpc.

[Captain Southbird (EHOL Creator)]

Hmm... well, it'd still be a very large Java app no matter what you do.  Although I suppose Java only has to download things as it runs so it wouldn't be all at once.  But I don't know nearly enough about Java to even guess exactly how this would all work.  But the little I do know suggests that it could probably be done.

Can a Java app download a script data file into some sort of memory and process it?  That's basically where I think you'd take it.  You'd download the EHPC script for a given chapter, then the Java app would download graphics as needed for any given moment.  I assume it can cache things it has already loaded as well?

[rolfstenholm]

Java is not necessarily run in a browser, it can be run as standalone although doing both is not that difficult. It is possible to stream all data including the EHPC scripts on demand. I doub caching the data helps much, but i guess it depends on the browser and operating system. Secondly the ehpc files and the rest of the files could be compressed using zip. The app running the ehpc scripts is not very difficult to write especially if some screen effects are not implemented.

The trouble with compressed data is what decompressor to use. To decompress the data in EHPC file saturn.art I would need to know exactly what ZIP version you compressed this with, and what byte the compressed file starts and stops at, in other words if there are any padding bytes.

[Captain Southbird (EHOL Creator)]

SATURN.ART is merely a group of files into one; it by itself is not compressed by any means.  I never wrote an extractor for it, so you're free to try yourself.

It follows this order:

Header:
7 bytes: "BOBGLOB"
2 bytes: Number of entries

First data block (file name and file info), repeated for as many entries as was defined:
8 bytes: Filename
4 bytes: Location in glob
4 bytes: Length of data


The filename data is ALWAYS exactly 8 bytes.  NULL values are used to fill unused characters, but a NULL terminator is not necessarily present.

I.E.  The name "_CRED00A" is exactly eight characters long.  The next byte after the character 'A' is the first byte of the four that define the location.  The name "BIN00" will be followed by three zero-value bytes to extend it to the full eight character space before reaching the first byte of the location.

Since BOBGLOB stores all individual file data in one long string, you have to seek to the starting point first (which the location bytes are used for) and then read up to the length specified.

[rolfstenholm]

Something that puzzles me with your format is that the bytes numbered 7-8 (0 first) are 802. Checking the first entry byte position and the names it appears as if this value should be 918. Is there an error in the length field in the saturn.art file ?

First few bytes with values
BOBGLOB   802
_CRED00   14697  4181
_CRED00A  18878  2411
_CRED01   21289  20755

[Captain Southbird (EHOL Creator)]

Bytes 7 and 8 ought to register 917, which is the total entry count.  I have just verified this manually using EDIT and reading the bytes 149 and 3, where of course: 149 + (3 * 2⁸) = 917

Your file sizes are correct (the second value following the names) and I'll assume the jump-to value is correct, though that's not easy to verify arbitrarily.  (At least, doing the same thing I've verified both numbers manually, although since I have the RAW files I can also check them for accuracy, and they are correct.)

So you may want to check around whatever is supposed to gather the entry coount value.

[rolfstenholm]

Thanks that info was helpful.
How are images coded ?
As far as i can tell they seem to be 2 byte encoded.
Are the images encoded big endnian (first bit most significant) or small endnian ? (first bit least significant), and are the colors BGR,RGB ? and what bit length is each color ?
They appear to be roughly 5 bit each but is hard to tell..

[Captain Southbird (EHOL Creator)]

As mentioned previously, the pictures are ZLib encoded.  The compression algorithm is not my own, so I cannot describe how it works.  But at least I can say that the images themselves go in as 16-bit HiColor images, and that's what is compressed.

The format of the image is simple; 2B width, 2B height, 4B ID ("HICO"), and the rest is ZLib data of some varying size.

Since I have no idea what you're programming abilities are/utilities you have available, I'll simply give you the quick utility I wrote to make these in the first place.  I included the source if you want to see it interface with ZLib and all, but I apologize that it's very sloppy quick coding.  This is one of those little things I never figured would be seen by public eye and just needed it quickly.  

http://www.el-hazardonline.net/download_temp/HiLumper.zip

To decompress any of the pictures, simply use the included binary as such:
HILUMPER IMAGENAME OUTPUT.BMP 2

Where "IMAGENAME" is whatever you called the image and OUTPUT.BMP can be substituted as any name .BMP.  The "2" tells the utility to perform a decompression.  A "0" tells the utility to perform a compression, so you could also create the graphics files.  The "1" mode, "Compress Sprite", is actually for a completely different project I had been working on when I made the format in the first place, so you can ignore it as it serves no purpose here.

Note that not EVERYTHING you get out of SATURN.ART are these graphic files... some are sound and some are MIDIs.  Generally anything named with an underscore, using the format ELx_yy, starting with an F, or appears to be Japanese phonetics -- such as "SOUKUU" -- is generally a graphic.

[rolfstenholm]

I managed to decode the images. Works nicely now. Founn all the "secret" images in saturn.art file.
How does the encoding of the dialogs work ?
Are the bytes used in the dialogs positional offsets into the images _fonta,
_fontb, _fontc ? An what is does the chaptern.epc, chapterx.epc, chapterx.epc file do ?

A tip for programing:
Check out the bit syntax in C, it makes it easier to grab the lowest bits in an int, for instance
myUnsignedChar = myInt & 0xFF;
Does the trick.

[Captain Southbird (EHOL Creator)]

How does the encoding of the dialogs work ?
Are the bytes used in the dialogs positional offsets into the images _fonta,
_fontb, _fontc ?

Yeah, all of the ".wth" files contain single bytes specifying the WidTH of each character in each respective font file.

An what is does the chaptern.epc, chapterx.epc, chapterx.epc file do ?

There's where everything happens.  The EPC's are "El-Hazard PC" game scripts.  All scripts start off with a magic number that translates into ASCII as "EHPC."  After that it simply contains an array of command bytes with parameters.  Commands are one byte wide and parameters are variant.  There are 39 commands.

Here's the header file from the script compiler:
http://bobstuffs.no-ip.com/El-Hazard/temp/bob/c_commands.h

The comments should generally tell you about what types are and everything.  In the "script_parameters" array, they are strings that were used by the compiler to break down my plain text script files into binaries.  

Anyway, the first command, BACKGROUND, is matched to a string that reads "210".  That is, 2 parameters, and their types are 1 (a string, filename to be exact) and 0 (a 16-bit number).  All types are noted in a comment in that .h file.  

All strings are NULL terminated and general numericals are 16-bit.  Parameter type 6, menu string, is actually just another NULL terminated string, it's only looked at specially by the script compiler but otherwise it's just a string.

Parameter type 4, dialog string, is deliberately altered so it's unreadable under a normal editor.  (I.E. To prevent spoilers or cheaters!)  Specifically, each character of the string has the following operation performed on it:
corrupt_char = 127-charofstring;
So to restore dialog strings, you must perform that operation again to get the original character back.

(E.g. 127 - 5 = 122; 127 - 122 = 5)

Parameter type 2 sets left or right position on dialog boxes by how many "blocks" (16 pixels wide I believe) over the picture should be from the edge of the screen.  If bit 128 is set on the byte, it's coming from the right side and the lower bits still specify the block count.

Parameter type 3 is the same as type 2, only it's specifying top and bottom block positions, where bit 128 specifies coming from the bottom.

Parameter type 5 is merely a special byte-sized number for screen effects.  0 = Wiggle screen, 1 = White Flash, 2 = Fade to white, 3 = Fade to black.

Parameter type 7 specifies a map operation which responds to the other parameters sent to the command:
0 = Change map display by first paramter:
first parameter values: 0 = Shut off map mode (BACKGROUND command does this automatically), 1 = Do the large world map, 2 = Do the Ura Map

1 = Store a cursor coordinate pair for a menu selection (for large world map, where the blue cursor hops to hover over a destination); first parameter sets which menu choice this coordinate pair is to be assigned to, and the following two parameters are the screen coordinates

2 = Make Map Makoto sprite walk to current blue cursor position (no parameters)

3 = Forcefully set Map Makoto sprite to some screen location X and Y

4 = Set Map Jinnai's position same as above

5 = Reset the Bugrom spawning amount

6 = Specify a number of a label to jump to when time runs out on the map


This can probably be confusing as hell at first glance.  Here's the raw TXT pre-compiled script for Chapter 4, which uses pretty much all of the commands:

http://bobstuffs.no-ip.com/El-Hazard/temp/bob/chapter4.txt

A tip for programing:
Check out the bit syntax in C, it makes it easier to grab the lowest bits in an int, for instance
myUnsignedChar = myInt & 0xFF;
Does the trick.

Well, thanks for the tip, but I am quite familiar with all bit operations.  (If questionable content was found in my silly HICO converter, remember I said I wrote that thing really quickly and sloppily!)  After all, GBA programming requires changing hardware registers at bit level almost regularly.  In C:

REG_DISPSTAT |= 0x8;

SlideshowPrint(8, 2  + (dialog_params & 0x1 ? 120 : 0), -1, dialog_texts[0]);

byte_length = *starting_pos + (*(starting_pos+1)<<8) + (*(starting_pos+2)<<16) + (*(starting_pos+3)<<24);


Or in the code components that must be GBA ARM7 assembler:

@ REG_WIN0V = ( 0 or 120 << 8 ) | ( 39 or 159 );
tst            r4, #1            @ If bit 1 is set, do it on bottom
movne      r1, #0x7800 @ BOTTOM OF SCREEN settings -- (120 << 8) | 159 = 0x789F
orrne      r1, r1, #0x9F
moveq      r1, #39            @ TOP OF SCREEN settings --
strh      r1, [r0, #0x44]

... so yeah.  :P