Keen 4-6 Action Format

Tools, assembly, and file formats.
gerstrong
Posts: 63
Joined: Sun Jan 25, 2009 3:21 pm

Keen 4-6 Action Format

Post by gerstrong »

Hello fellows,

I have a question about the Action Format. How can I get to read that information?

I have read the wiki article about it, but I don't get it, how they mean it with the location. What does that mean, if it's not an offset?

Do I need a special formula to read to read that Action Format Data?

I'm one of the developers of CG, and I want to make it read Action format so we can in future also support the Keen galaxy mods and have a code near to the original engine
lemm
Posts: 554
Joined: Sun Jul 05, 2009 12:32 pm

Post by lemm »

I'm assuming you're talking about the last four fields of the action format.

Code: Select all


// from andy's keen4 disassembly

struct ACTION;
typedef struct ACTION {
	int right_chunk;
	int left_chunk;
	int int2, int3, int4;
	int delay;
	int int6, int7;
	void far (*move_proc)( OBJECT *obj );
	void far (*collide_proc)( OBJECT *obj1, OBJECT *obj2 );
	void far (*proc3)();
	struct ACTION *next;
} ACTION;
The Behaviour, Sprites, and Tiles are all far function pointers, (4 bytes, segment:offset ). The functions handle sprite behaviour, sprite-sprite collision, and sprite-tile collision respectively.

The final field is a near pointer, which points to the start of the next action that the sprite will process. Because all actions are located in the DSEG of the executable, there is no need for a segment to be defined, unlike the 3 function pointers.
levellass
Posts: 3001
Joined: Wed Oct 11, 2006 12:03 pm
Location: Ngaruawahia New Zealand

Post by levellass »

DSEG? What is this DSEG?
lemm
Posts: 554
Joined: Sun Jul 05, 2009 12:32 pm

Post by lemm »

data segment
gerstrong
Posts: 63
Joined: Sun Jan 25, 2009 3:21 pm

Post by gerstrong »

No, I really mean, how do I get that data, at what adress do I get access to that structure.

Let's say, I want to know the sprite picture number of keen shooting, when triggering an action. How can I read that number of picture? All sprite images and bitmaps and so on I think are numbered somehow, I really would like to read those number. I think Action format contains that information.

Can you tell me at what segment I can read that information?

The last four pointer, seem to store some code I will have to programm myself anyway I imagine. I don't think I can use it for CG...
lemm
Posts: 554
Joined: Sun Jul 05, 2009 12:32 pm

Post by lemm »

See http://www.keenmodding.org/viewtopic.php?t=906

In the actions, as the left_chunk and right_chunk contains the index of the chunk (i.e. sprite frame) displayed when the sprite is facing left or right, respectively.

In keen4, for example, the first sprite frame (1SPR0000.BMP) is chunk 124.

With respect to actual offsets within the EGA*.CK4 file, take a look at http://files.keenmodding.org/mobydoc.txt, but I'm guessing this is not what you mean if you can already read the game files.

If you are talking about where the actions are located within the actual executable, then take a look in the patch index:

http://levellord.rewound.net/Index/Keen ... 0_List.TXT

The location of the action in the executable (not counting .exe header) is $2EE70 + whatever value you see in the table (since the data segment starts at $2EE70).
gerstrong
Posts: 63
Joined: Sun Jan 25, 2009 3:21 pm

Post by gerstrong »

Yeah, thanks, that is what I was looking for. I think for future modding it is better loading those rather than hardcoding by ourself.

Thanks again.

Hmm, the data segment location. Is it the same in all Episodes?
gerstrong
Posts: 63
Joined: Sun Jan 25, 2009 3:21 pm

Post by gerstrong »

Sorry, I still have some questions.

The Datasegment start is clear, there we have the data, but at this segment, what does really start of data?

Is it the action_format itself or also other data before?

In action.doc I saw the lowest address Az.08x I think it is, which I think means, Dseg_pos + $08xx the get to that data-block of action format.

The data also seem partly bundled into blocks (keens actions for example, makes sense), although I'm not really sure about it.

Can I get all the action format data by just reading from the dseg offset to the end, whereever this is?

Or are they really interspersed as they tell in the keenwiki, and if so, is there a known algorithm to get that az hex number which is written in action.doc?

I really would like to read them all and bundle them into one big vector of structures, or dynamic array.

Because, if we have to hardcode those hexnumbers, to get every action-format, and that for every episode, I'm not sure, if it's really worth. Especially because I'm very unsure, how many modders really change that action-format.
levellass
Posts: 3001
Joined: Wed Oct 11, 2006 12:03 pm
Location: Ngaruawahia New Zealand

Post by levellass »

Hmm, the data segment location. Is it the same in all Episodes?
Heck no, the offsets for all the Keens are here: http://www.shikadi.net/keenwiki/Patch:Text_patches (First table.)

The Datasegment start is clear, there we have the data, but at this segment, what does really start of data
?

Um, they haven't it then because?

If you're asking what type of data is stored in this segment, mostly text, sprite actions and tables of values for stuff (Such as the sprites to use for various things on the map)

If you're asking where the data starts, it starts at the start of the segment, in theory. Think of it as a big box, you can put stuff anywhere in the box, but that doesn't mean that the box will be filled completely or in an orderly fashion.

The start of the segment typically has something about Borland C++ in it, as far as I know this isn't used by anything, but it marks the start oft he segment.

If you're asking about the segment being divided up, this is only due to lazyness on the programmer's parts I think. You see stuff like Keen's actions being split in two by some stuff about keyholders or shots. The actual arrangement of stuff is just how it turned out, you can place and rearrange data however you want, (and neater too, but that's a lot of patching just for neat code.)

In summary, the segment is a disorganized mess.

Can I get all the action format data by just reading from the dseg offset to the end, whereever this is?
Yes, in theory. However you'd need some way to tell action data from all the other stuff, and there's no rules for where an action can be placed. In TUIT several new actions are placed in all sorts of convenient places.

What the game does is more sleek, it doesn't read an action until a sprite behavior or other action tells it to (By giving it the action location.) Typically it starts with the sprite spawn code which will give the sprite's first action. (This is why we get errors like uncached sprite, the game didn't know it needed to have that sprite because it didn't read all the actions before starting the level.)

Or are they really interspersed as they tell in the keenwiki, and if so, is there a known algorithm to get that az hex number which is written in action.doc?
Interspersed? They're all mixed up, generally in large blobks, but not always. I have a QBASIC program, SPR33 that can take all the sprite actions from Keen 4-6 and Dreams and dump them into a file, ACTIONS.CK* I use this when looking at sprite stuff.

Because, if we have to hardcode those hexnumbers, to get every action-format, and that for every episode, I'm not sure, if it's really worth. Especially because I'm very unsure, how many modders really change that action-format.
Depends how you're going to do it. Hard-coding will not be compatible with mods as they don't have to use the default values, especially when adding 'new' actions. I suggest doing it like the game does, reading an action when directed. If you know the episode's segment offset (Or just search the executable for the telltale start of the segment.) you can read any action you're given the location of.

If you want to be more sneaky you could have all the required actions read and stored in some manner either on game startup or on level startup by reading the spawning code, getting all the actions it refers to, all the actions THEY refer to and so on. (As well as all actions referenced in the action's behaviors and sprite collision.)
gerstrong
Posts: 63
Joined: Sun Jan 25, 2009 3:21 pm

Post by gerstrong »

levellass wrote:
Hmm, the data segment location. Is it the same in all Episodes?
Heck no, the offsets for all the Keens are here: http://www.shikadi.net/keenwiki/Patch:Text_patches (First table.)

The Datasegment start is clear, there we have the data, but at this segment, what does really start of data
?

Um, they haven't it then because?

If you're asking what type of data is stored in this segment, mostly text, sprite actions and tables of values for stuff (Such as the sprites to use for various things on the map)

If you're asking where the data starts, it starts at the start of the segment, in theory. Think of it as a big box, you can put stuff anywhere in the box, but that doesn't mean that the box will be filled completely or in an orderly fashion.

The start of the segment typically has something about Borland C++ in it, as far as I know this isn't used by anything, but it marks the start oft he segment.

If you're asking about the segment being divided up, this is only due to lazyness on the programmer's parts I think. You see stuff like Keen's actions being split in two by some stuff about keyholders or shots. The actual arrangement of stuff is just how it turned out, you can place and rearrange data however you want, (and neater too, but that's a lot of patching just for neat code.)

In summary, the segment is a disorganized mess.

Can I get all the action format data by just reading from the dseg offset to the end, whereever this is?
Yes, in theory. However you'd need some way to tell action data from all the other stuff, and there's no rules for where an action can be placed. In TUIT several new actions are placed in all sorts of convenient places.

What the game does is more sleek, it doesn't read an action until a sprite behavior or other action tells it to (By giving it the action location.) Typically it starts with the sprite spawn code which will give the sprite's first action. (This is why we get errors like uncached sprite, the game didn't know it needed to have that sprite because it didn't read all the actions before starting the level.)

Or are they really interspersed as they tell in the keenwiki, and if so, is there a known algorithm to get that az hex number which is written in action.doc?
Interspersed? They're all mixed up, generally in large blobks, but not always. I have a QBASIC program, SPR33 that can take all the sprite actions from Keen 4-6 and Dreams and dump them into a file, ACTIONS.CK* I use this when looking at sprite stuff.

Because, if we have to hardcode those hexnumbers, to get every action-format, and that for every episode, I'm not sure, if it's really worth. Especially because I'm very unsure, how many modders really change that action-format.
Depends how you're going to do it. Hard-coding will not be compatible with mods as they don't have to use the default values, especially when adding 'new' actions. I suggest doing it like the game does, reading an action when directed. If you know the episode's segment offset (Or just search the executable for the telltale start of the segment.) you can read any action you're given the location of.

If you want to be more sneaky you could have all the required actions read and stored in some manner either on game startup or on level startup by reading the spawning code, getting all the actions it refers to, all the actions THEY refer to and so on. (As well as all actions referenced in the action's behaviors and sprite collision.)

Ahh okay, I get it. It's the data-segment with all the stuff, good to know. I want to read that action format correctly and completely and definetely agree, that it will be incompatible having hard coded stuff.

So you say, some are bundled into blocks and some others aren't. Are you sure about it?

I have been looking into the action.doc, and they seem to be bundled, but maybe some addresses are unknown.

how do you bundle them together with your qbasic program?

Do we have the case like action-format block1, then other data, then action-format 2, and so on, or is it a fact, that not all action-formats are known, but they are bundled together at a place.

I really want to have CG reading them out, especially for the mods.

Thanks again for your information. I really don't know much about Keen Galaxy but I'm learning and you have helped me a lot so far...
levellass
Posts: 3001
Joined: Wed Oct 11, 2006 12:03 pm
Location: Ngaruawahia New Zealand

Post by levellass »

I was going to post another program, but I found this while browsing my folders, and it works pretty good. It is a basic program that scans the DSeg of ANY Keen Galaxy engine programs looking for actions. It turns up more than a few false positives of course, but these can be weeded out, the basic (heh) code suffices for me. The code below is set for Keen 4, all you need to change is the executable name (Opened as #1) and the location of the Dseg (x, here set as 192112, as I've said, it's not hard to scan an executable to find the start of the Dseg as well.)

Code: Select all

#QBASIC scan thing
DIM SHARED x2 AS STRING * 2
DIM SHARED x6 AS STRING * 6
DIM SHARED x30 AS STRING * 30

OPEN "DUMP4.BEH" FOR BINARY AS #1
OPEN "OUTPUT.BIN" FOR BINARY AS #2

x = 192112 ' Start of Dseg

FOR l1 = x + 1000 TO x + 17000
 GET #1, l1, x6
 IF CVI(LEFT$(x6, 2)) < 4 AND CVI(MID$(x6, 2, 2)) < 2 AND CVI(RIGHT$(x6, 2)) < 2 THEN
  GET #1, l1 - 2, x2
  SELECT CASE CVI(x2)
   CASE 100 TO 600, 0, -1
   GET #1, l1 - 2, x2
   SELECT CASE CVI(x2)
    CASE 100 TO 600, 0, -1
     GET #1, l1 - 4, x30
     IF x30 <> STRING$(30, 0) THEN PUT #2, l1 - x, x30 ' Don't place blank strings
     l1 = l1 + 29 ' NOT 30 since loop adds 1!
   END SELECT
  END SELECT
 END IF
1 NEXT l1
CLOSE
END
What's outputted is the actions in a binary file, with each action located at its AZ address. Of course actions can be dealt with as they're found.

In pseudo code this is what happens:
Open executable
Scan executable byte-by-byte from Dseg start to end of file(ish)
IF we find a string of 6 bytes where:
First word < 4
Second and thrid words are < 2
THEN We may have an action
Get two words behind this string
IF they are both
Between 100-600
0
-1 ($FFFF)
THEN This looks like a 'proper' action and we put it in OUTPUT
False positives tend to be segments of tables in the Dseg or bits of text. It's also possible that actions will be located in the first 1000 or further than 17000 bytes into the Dseg (Which the program doesn't scan) but this isn't the case for any Galaxy game and are highly unlikely to occur even in mods.

The severity of false positives depends on what you do with them. Attempting to get information out of them results in gibberish (But will quickly eliminate them if you LOOK for gibberish and discard any 'actions' that have it.) if you use actions like the game does (only using them when their address is called) all false positives will of course, be eliminated.

Using the action addresses it is also possible to scan the exe for calls to them, this is how I made a lot of patches.
gerstrong
Posts: 63
Joined: Sun Jan 25, 2009 3:21 pm

Post by gerstrong »

Thanks levellass! This is really great and if it will read all action formats we need, I will take that algorithm. The pseudo code made me understand it quite well.

I also will put your name in our source code as honours.

Thank you very much and also thanks to lemm for the explanation!!!

I'm still not sure, how you to bind those formats to for example have the same action format request if player shots, in every episode, because I imagine it's different, but I think we will find that out. The important thing, we can bundle them now into one array.

Thanks again.
gerstrong
Posts: 63
Joined: Sun Jan 25, 2009 3:21 pm

Post by gerstrong »

Hi Levellass, maybe I can help you optimizing that algorithm, if you want, of coure. I have some questions about it.

Why from 100-600? Wouldn't if be more convinient to go from 124-400? The Sprites images begin at that number 124 and end 400 (?), or did I confuse something here? well was just a thought.

Code: Select all

Open executable
Scan executable byte-by-byte from Dseg start to end of file(ish)
IF we find a string of 6 bytes where:
First word < 4
Second and thrid words are < 2
THEN We may have an action
Get two words behind this string
IF they are both
Between 100-600
0
-1 ($FFFF)
THEN This looks like a 'proper' action and we put it in OUTPUT

Code: Select all

n: Commander Keen Episode 4 (Version 1.40) was detected.
n: 1190 Action Format(s) read
I got that output. 1190 Action Formats. Gee, is that number correct or how many do you read out with your algorithm?

Anyway your algorithm is awesome!!
levellass
Posts: 3001
Joined: Wed Oct 11, 2006 12:03 pm
Location: Ngaruawahia New Zealand

Post by levellass »

Why from 100-600? Wouldn't if be more convinient to go from 124-400? The Sprites images begin at that number 124 and end 400 (?), or did I confuse something here? well was just a thought.
In Keen 4, yes. However other games have other chunk values for sprites. I could of course have hard-coded values for each episode, but as the code was general rather than specific I decided against it.
I got that output. 1190 Action Formats. Gee, is that number correct or how many do you read out with your algorithm?
Strange, I got 367 when I run it on Keen 4, including a few false positives.
gerstrong
Posts: 63
Joined: Sun Jan 25, 2009 3:21 pm

Post by gerstrong »

Yeah, something was wrong here. I really didn't get that algorithm to be honest. I tried to implement it in CG but it didn't work and I also discovered it's not very useful to bundle the actions either, because the next tile adresses needs to be calculated. It would be more code.

The only I want to hardcode is the relative address of every sprite starting from the start dseg. I can read the rest of it, and CG is reading them pretty well that way. I also was able to get the proper animations as they have to be.

I don't know, if that hardcoding method is a good idea, but having another enemy added in Keen 4, would mean we need something different...

I think that way, Mods can rely on it pretty well.

Btw. the cache problem won't be in CG because all graphics in CG are loading as soon as the game starts.
Post Reply