Dangerous Dave modding

Tools, assembly, and file formats.
User avatar
Crazy Daver
Posts: 8
Joined: Mon Mar 07, 2011 12:21 am
Location: Russia
Contact:

Dangerous Dave modding

Post by Crazy Daver » Wed Apr 06, 2011 7:00 am

I'm creating programs for the modification of Dangerous Dave 2, and the file structure of Keen's and Dave's similar. I have several questions about the structures of files and compression algorithm.

1) How to uncompress files compressed by Huffman (sprites and the title screen)? I understand the algorithm itself. I do not understand how to build a tree of data, recorded at the beginning of the file, then to correctly decode the file. In what order are the elements of the tree stored in a file?

Can anybody on the example of a file to show how these data to build a tree?

2) Unlzexe says Dave 2 exe-file is not compressed by this algorithm. Where can I find the unpacked exe-file Dangerous Dave 2?

3) What is the structure of information about the colors in the title screen (for example, in intro.dd2, in which the color 'dark blue' is used as a transparent color)?

4) Where in the exe-file to change the binding of the doors to the numbers of tiles? If you simply change the value of tile in tileinfo at the door, it will not work.

5) Where in the exe-file are sprites death of Dave? What structure of information about the colors?

6) Where in the exe-file contains characters in the font? What structure of information about the colors?

levellass
Posts: 2932
Joined: Wed Oct 11, 2006 12:03 pm
Location: Ngaruawahia New Zealand

Post by levellass » Wed Apr 06, 2011 10:04 am

Welcome, glad you solved the problem with your account and nice avatar!

Keengraph can modify Dave's graphics, which I believe are the only files that use the huffman compression, the files it produces on import have huffman stuff, but are not compressed. (After a certain point they can be read as raw data.)

The huffman algorithm is rather odd to explain, though I can do this and indeed have some files about this that used to be up on the patch index.

Dave is indeed LZEXE compressed, maybe you have one that is already decompressed. It will be 215'536 bytes long. Keengraph checks for this.

There is no 'info' for the colors in the intro as such, any pixel that is EGA plane 0 only (Dark blue) is clear. This is difficult to change.

What is used for doors (Door opening?) depends on tiles placed in the level I believe, not the tile properties themselves.

The death BITMAPS are scattered all over the exe and are basically 'Do the death thing starting with bitmap x'; basically you can change the start bitmap of a sequence but changing the number of frames say affects all death sequences.

The font, stored as a monochrome (8 pixels per bit) is stored at $106B0 - $114EF in the dumped executable data. Keengraph can extract, but not import this I believe. It is two color only.


The setup to edit graphics, levels, sounds and patches is here: http://dl.dropbox.com/u/3940020/Dave%202%20ed.zip

Mink seems to be unwilling to expand Mindbelt to handle Dave 2, it's very sad. If you have any questions about how to use any of the utilities here, just ask.

Next step, pester Mink so you get a smoother level editor. Dave 3 and 4 editing is also possible.

User avatar
Crazy Daver
Posts: 8
Joined: Mon Mar 07, 2011 12:21 am
Location: Russia
Contact:

Dangerous Dave (3,4) level editor

Post by Crazy Daver » Thu Apr 07, 2011 7:27 am

Thank you for the program. Do you have graphics and similar programs for Dave 3 and Dave 4?

Now I make a level editor for Dave 2 and I want to make it possible to edit levels for Dave 3 and 4. I read about file formats. It seems that the compression algorithms (Carmack and Huffman) is the most difficult part of editing levels. Do I understand correctly?

levellass
Posts: 2932
Joined: Wed Oct 11, 2006 12:03 pm
Location: Ngaruawahia New Zealand

Post by levellass » Thu Apr 07, 2011 10:14 pm

Well as I said, we *already* have a level editor for Dave 2, we just need Mink, the program's creator, to make a few tweaks so the levels load directly. We also have one (Actually TWO, one of them being TED5, the original level editor!) for 3&4, but again we need to port TOM to Dave so it can work with the levels directly.

For Dave 3 it's possible to edit the graphics, sounds (Some of them!) and levels. Dave 4 needs a patch program made up. (5 minutes work for Mink, but you know how he is.) It would be nice to get some tile editing, etc set up for Dave, and I made a topic on the PCKF about this.

User avatar
Crazy Daver
Posts: 8
Joined: Mon Mar 07, 2011 12:21 am
Location: Russia
Contact:

Post by Crazy Daver » Fri Apr 08, 2011 9:46 am

I understand. But I still want to make my editor. He is already working and he can edit levels for Dave 2. Only need to add a few features for convenience in editing. Still, I want to do so that he could edit the levels of Dave 3 and 4 and possibly Dave 1.

What should I do to edit the levels of Dave 3 and 4? Which files need to decompress and what compression algorithms they use, what files need to cut from the exe-file and what else should I do? Can you write a sequence of steps that need to do to go directly to edit the levels?

And you have already cut the sprites and tiles from Dave 3 and 4?

levellass
Posts: 2932
Joined: Wed Oct 11, 2006 12:03 pm
Location: Ngaruawahia New Zealand

Post by levellass » Sat Apr 09, 2011 1:35 am

Dave 4 can be edited, but won't work until Mink makes us a Dave4Patch. Dave 3 is here: http://dl.dropbox.com/u/3940020/DAVE%203.zip

You can edit the levels by running 'runted.bat' (IN DOSBOX!' to bring up the original Dave 3 level editor. You must save (Alt+S) but NOT carmackize levels before exiting. You will get a 'converting' message, then you can play your edited levels using 'dave3.pat' (You will notice I make a one tile edit in level 1)

While the graphics are extractable, there is a small bug that means you CANNOT import them, I am busy tracking down why this is. (You can edit the graphics and wait for me to fix the bug.) We cannot edit the big screen graphics (Like Warning: Violence ahead) yet, but I am adding that too. Sounds of course can be edited as well. (The PC\Adlib, not the disney\digital.)

Again, hopefully TOM can be made to work with Dave 3 and 4 soon enough.

User avatar
Crazy Daver
Posts: 8
Joined: Mon Mar 07, 2011 12:21 am
Location: Russia
Contact:

Post by Crazy Daver » Sat Apr 09, 2011 4:23 am

Why Dave 4 may be edited, but will not work? As far as I understood from the description, the program DD*PATCH.EXE created solely for the convenience. Instead of changing something in the exe-file or the original game files, it loads the new files directly to memory before the execution of the program.

Thanks for the graphics from Dave 3. Do you have sprites with alpha channel?
Again, hopefully TOM can be made to work with Dave 3 and 4 soon enough.
In any case, the two editors are better than one.
I have a few questions about editing:
To edit only need three files: MAPDICT.DAV, MAPHEAD.DAV and GAMEMAPS.DAV, if the editor to load graphics from the cut sprites, rather than directly from EGAGRAPH.DAV. About MAPHEAD.DAV, it seems, everything is clear. In MAPDICT.DAV stored Huffman dictionary. He needed to unpack the whole gamemaps.dav or any part of it? The levels are stored in GAMEMAPS.DAV. What are the algorithms that the file is compressed? In what sequence it needs to decompress?

levellass
Posts: 2932
Joined: Wed Oct 11, 2006 12:03 pm
Location: Ngaruawahia New Zealand

Post by levellass » Mon Apr 11, 2011 12:04 am

Hmmmn, wish your english was a bit better.

Dave 4 can't be patched without Mink making a patch program for it, but we can edit sounds, levels and graphics for it. (But not USE them.)

I don't know what an alpha channel is; keengraph, like modkeen works with windows bitmaps only. The graphics there are the ones it exports by itself.


TED5 works with the game files, EGAHEAD, EGAGRAPH and EGADCT to load graphics (The same one as the game uses.) and edits levels with MAPTEMP and MAPTHEAD. The converter makes the GAMEMAPS, MAPDCT and MAPHEAD.

Unlike Keen 4-6, but like Keen Dreams (Which it is based on.) the maps use RLEW and then huffman compression. My converter doesn't bother with the huffman (The MAPDICT file basically says 'Do nothing') so it's only RLEW compressed, like the MAPTEMP file made by TED5. You CAN do huffman if you like, but I think it's a waste of time.

The compression is RLEW first, THEN huffman.

The converter works by adding four bytes to all the compressed stuff, all that's needed for 'huffman compression' when you're lazy. Each level is stored separate and in four parts (Three planes and the header) and so can be loaded separately, even copied from other GAMEMAPS files, but you must always end up with all of them saved to one GAMEMAPS file.

User avatar
Crazy Daver
Posts: 8
Joined: Mon Mar 07, 2011 12:21 am
Location: Russia
Contact:

Post by Crazy Daver » Mon Apr 11, 2011 9:25 am

The second part of the picture is the alpha channel. (http://s004.radikal.ru/i206/1104/c5/f17648c4ed33.bmp)

To decompress gamemaps.dav, using Huffman, I need to know the size of the decompressed file gamemaps.dav.
How to determine its size?

Where in the file gamemaps.dav begins packed information and where it ends (HUFF)?

levellass
Posts: 2932
Joined: Wed Oct 11, 2006 12:03 pm
Location: Ngaruawahia New Zealand

Post by levellass » Tue Apr 12, 2011 1:59 am

Keengraph uses 256 color bitmaps, the alpha channel is part of the sprite, colors 0-15 are used for masked colors (Black in your alpha picture) while colors 17-31 are used for unmasked colors (White on your alpha picture.) Color 16 and all other colors (32-255) are transparent. I did this because it is easier than doing all those masks, and nobody makes see-through sprites very often.

http://www.shikadi.net/moddingwiki/GameMaps_Format

The gamemaps file is in pieces, each piece is huffman compressed. The HEAD file points to the level headers, these all decompress to $26 bytes long. Each header points to the three parts of the level, background, foreground and sprite layers and their compressed size. The first 4 bytes of each is the decompressed size after huffman.

User avatar
Crazy Daver
Posts: 8
Joined: Mon Mar 07, 2011 12:21 am
Location: Russia
Contact:

Post by Crazy Daver » Thu Apr 14, 2011 6:33 pm

Which file contains information about the tiles, masks and icons that are not used in the Dave 3 (black square in place of tiles in the editor TED5)?
What is stored in files SHP0*.DAV of Dave 4?
Do you have a graphics of Dave 1?

levellass
Posts: 2932
Joined: Wed Oct 11, 2006 12:03 pm
Location: Ngaruawahia New Zealand

Post by levellass » Fri Apr 15, 2011 5:48 am

The unused tiles are just that, unused, you can use them when making new graphics. Some of the unused icons are blank though.

Dave 1 has three types of graphics, CGA (4 color) EGA (16 color) and VGA (256 color) Only the EGA are external and can be edited at the moment. Malvineous over at the modding wiki is working on a Dave 1 editor, though if this takes too long I suppose it's possible to make one ourselves. (Note most of the graphics and the levels are in the EXE file and thus need to be patched.)

I do not know what is in the SHP files, only that they are not Softlib compressed. I'm guessing the names mean 'shape', but what those are I don't know. They're not graphics or levels or sounds or screens.

User avatar
Crazy Daver
Posts: 8
Joined: Mon Mar 07, 2011 12:21 am
Location: Russia
Contact:

Post by Crazy Daver » Thu Feb 09, 2012 10:41 am

I have a question about the file EGAGRAPH of DD3.

EGAHEAD store 4-byte offset for file parts EGAGRAPH. (http://s005.radikal.ru/i209/1202/9c/0eeb59e16fd9.png)

EGAGRAPH stores image files. (http://s004.radikal.ru/i206/1202/7b/9dc42948abe9.png)
The first four bytes of each part (in the rectangle) is the size of the uncompressed file.
The remaining bytes (underlined) is a file that is compressed using the Huffman algorithm.

The first file is unpacked fine, but the size of the unpacked files become too large (even impossible), starting from the second file.

Where I made a mistake?

levellass
Posts: 2932
Joined: Wed Oct 11, 2006 12:03 pm
Location: Ngaruawahia New Zealand

Post by levellass » Fri Feb 10, 2012 9:56 am

You have the wrong segment in dave.exe; what you are looking at is AUDIOHED.DD3 not EGAHEAD.DD3. In the patch dump I have it is at $1FDA0 and starts with the string $00 $00 $00 $00 - $D5 $00 $00 $00 $7B $0B $00 $00


Remember now, keenlike executables may have two or even three header files in them! I had to do some tricks when getting Keengraph to work with Dave.

User avatar
Napalm
Posts: 19
Joined: Sat Oct 27, 2007 6:49 am
Location: UK

Post by Napalm » Fri Feb 17, 2012 1:33 pm

Crazy Daver,
You should check out this page: http://www.shikadi.net/moddingwiki/Huffman_Compression my QBasic code is easy to understand.

If I remember correctly once you decompress a DD2 title screen you will be left with 4 planes of 1bit data. These are the 4 color planes of 16 color EGA screen mode. The reason this is stored in this format is because its faster to load and display each plane rather than switching planes every time you need to draw a pixel. You simply switch to 4 planes overall and do a block copy of the pixel data into display memory.

If you want, I can write a quick app that will decompress a DD2 screen into a PNG.


Napalm

Post Reply