Chocolate Keen (a "port" of Commander Keen 1-3)

Tools, assembly, and file formats.
NY00123
Posts: 85
Joined: Thu Sep 24, 2009 8:03 am

Chocolate Keen (a "port" of Commander Keen 1-3)

Post by NY00123 »

Copying from pckf.com for the ones who get more chances to see it here and others...

UPDATE (Dec 24th, 2016): Edited download links since Dropbox' "Public" folder is expected to become private by March 2017.

UPDATE (May 12th, 2013): Support for Keen 3 has just been added, meaning Chocolate Keen has full support for the Invasion of the Vorticons trilogy. Thanks again to Lemm for his assistance!

UPDATE (April 11th, 2013): Support for Keen 2 has just been added, hence the project is renamed "Chocolate Keen" now with no numbers involved. Thanks to Lemm for Keen 2 support!

Links for the most current version:
- Windows executable, packed with Keen 1: https://www.dropbox.com/s/5e1sqvbjzq1i7 ... 9.zip?dl=0
- Source code (for GNU/Linux, Windows and maybe more), again packed with a Keen 1 "test" folder: https://www.dropbox.com/s/14ouagaqamc1i ... r.bz2?dl=0

Links for an earlier version (without the launcher and cfg file, looks for some game data files in another place):
- Windows executable, packed with Keen 1: https://www.dropbox.com/s/b16l3l37cegp9 ... 5.zip?dl=0
- Source code (with Keen 1): https://www.dropbox.com/s/ejohb3d7nlegb ... r.bz2?dl=0

Original post contents (with a few edits):

Hey all,

Let me first add that the following would not be possible without the great work of QuantumG and Lemm.
Basically, what I am talking about is a "port" of Commander Keen 1 to modern platforms, using SDL (and SDL_mixer). I cannot say it is complete, but you should be able to play through all levels, at least. I also can't promise it's stable, though! Happy (Actually, that can *never* be promised...)

UPDATE (November 23th, 2012): I think that we now have a complete "port" of Commander Keen 1 v1.31. A few parts of the original code may not be ported in case they are never actually used (with no modification applied).

Please see the included README.TXT file for more details, along with things that may or may not be supported.

A short list of points describing the way this has been done:
- First, QuantomG figured out a lot of stuff about Keen 1 itself. Here is a link to a version of the file posted by him: http://www.quantumg.net/keen1.c.txt
Basically, it is C code which actually seems to have an implementation of x86 Assembly code.
- Secondly, Lemm came out, got the Keen 1 exe in ASM form and used QuantumG's work in order to give names to stuff.
- Finally, I have "ported" Lemm's work to a mixed C/C++ app using SDL. A few uncompression routines from Commander Genius have been used to the rescue.

Chances are that I won't be working on it any time soon. If someone wants to step up and improve it, be warned that a lot of the work, well... Either something is not so trivial (probably not so much now), or it's trivial but also "robotic" in a way...

P.S. About the name, I've thought that it makes sense since I aim to replicate vanilla Keen1 (including compatibility with files like SCORES.CK1). I've had other thoughts and the name *may* change later. For instance, if someone actually gets to add support for Keen 2 or 3...

For now, though, it is called Chocolate Keen1. NOT Chocolate Keen, nor Chocolate Keen Vorticons, Chocolate KeenV or anything else. (Hey wait, Chocolate KeenV may refer to Keen 5...)
The current name is simply Chocolate Keen1.

Hope you have fun while trying this out!
Last edited by NY00123 on Sat Dec 24, 2016 8:38 am, edited 20 times in total.
NY00123
Posts: 85
Joined: Thu Sep 24, 2009 8:03 am

Post by NY00123 »

And here is an update. It has taken somewhat longer than I have hoped...should have been expected, though. Now assume for real it's the last update of mine for awhile!

*** See the first post in this thread in order to find download links for the last release.

Changelog

October 31th, 2012 (v0.4.0): A secondary release.
- Three numbers are used now for the version numbering; Although, I can't think of that many (theoretical) future changes right now...
- OpenGL support has been implemented. It can be disabled at compile time.
- Hopefully fixed a serious potential cause of several memory corruptions, even though these do not always occur, or they occur on very specific setups. Basically, the allocation of a quite LARGE object on the STACK, in contrary to (not-too-many) smaller ones, is a pretty BAD idea!!! Of course, it may be possible to increase the stack size, but it seems more portable to avoid this.
- A good reminder has come that, by default, SDL redirects text output to stdout.txt and stderr.txt on Windows. Most importantly, this means that the list of command line arguments is not seen if the game is loaded from a command prompt. So, now the arguments are simply given in the README.TXT file.
- Changes have been done to the command line arguments and how they're used. A few arguments are replacements of others, while the rest are actual additions.
- Fullscreen is now the default, along with a scale (zoom) level of 2.
- OpenGL is used as a default. If Chocolate Keen1 is compiled without OpenGL support, then Overlay is the default.
- The desktop resolution is chosen for the display dimensions by default. That role does not apply if a software surface is used for output, though.
- As a result of changes regarding the video mode (like fullscreen by default), the chocolate_keen1_full_overlay.bat and chocolate_keen1_full_overlay.sh scripts are not bundled anymore.
- Fixed the Keen death sequence in the case Keen falls to a pit (below the level's bottom).
- The correct overscan border color should now be used during gameplay. At least, it is probably the case with the vanilla unpatched Keen behaviors.
- When a raygun is collected, 5 additional charges are added to the current arsenal, rather than replacing it.
- Fixed a few more cases of potential memory corruptions. While on vanilla Keen such corruptions are found, we simply cannot accept it here. It has seemed to resulted in a few "mysterious" crashes with OpenGL and video mode resets (fullscreen versus windowed).
- A few more minor technical changes, here and there...
Last edited by NY00123 on Fri Jul 05, 2013 9:07 pm, edited 1 time in total.
levellass
Posts: 3001
Joined: Wed Oct 11, 2006 12:03 pm
Location: Ngaruawahia New Zealand

Post by levellass »

Aaah excellent; that raygun bug was very confusing.
NY00123
Posts: 85
Joined: Thu Sep 24, 2009 8:03 am

Post by NY00123 »

levellass wrote:Aaah excellent; that raygun bug was very confusing.
Nevertheless, it's far from over. You've recently spotted some problem with the garg jumps.

This results in a minor release with no much testing this time... let's hope it is actually working...

*** See the first post in this thread in order to find download links for the last release.

Changelog

November 3rd, 2012 (v0.4.1):
- Fixed a couple of bugs in regards to the garg, mainly with the jumps.
- The "chocolate_keen1_full_overlay" scripts are actually removed from the
source tarball now.
Last edited by NY00123 on Fri Jul 05, 2013 9:08 pm, edited 1 time in total.
NY00123
Posts: 85
Joined: Thu Sep 24, 2009 8:03 am

Post by NY00123 »

Guess what? There is another update! A few more things missing beforehand are now implemented. The most important one, though, is probably the Keen 1 finale sequence!

*** See the first post in this thread in order to find download links for the last release.

Changelog

November 11th, 2012 (v0.4.7):
- Scrolling text functionality is now fully implemented. The help text is still not there, though.
- The ability to show an image from a file is also implemented! This includes the Keen 2 and 3 previews within Keen 1.
- It has happened after all... The Keen 1 finale can now be watched!
- As a consequence, a minor bug in the image loading routine has been spotted while originally implemented. Upon fixing it, though, it has resulted in a crash... Why? Well, on vanilla Keen the image is temporarily loaded to the map data (!!), but the fix wants to update the screen. This includes an update to the on-screen map tiles. So, as a workaround/fix, no bogus tile update is done at all now.
- Switches for lights, bridges and tantalus rays are now implemented. While out of all absolutely unmodified Keens these are found on Keen 2 only, relevant code is also found on Keen 1. The light switches can easily be tested by modifying a level. For the two other switches, tile types need to be adjusted. Note that while the Tantalus Ray switch appears to show the sequence with the screen shaking as expected, it does not result in a "game over" like on Keen 2. Rather, it returns to the world map with a black screen, being the result of a "fade out" without a following "fade in". It is the case in the original DOS version of Keen 1 with minor modifications to the tile properties, though.
- An ice cube shot from a cannon is now split into four smaller cubettes when it hits something (i.e. the original behavior), instead of two.
- An unpacked KEEN1.EXE file can actually be used now.
- A workaround for OpenGL border glitches which may appear on window/fullscreen mode switching within X11 is now in effect. At least, it works locally on Linux with X11. On Windows it may not occur since SDL resets the OpenGL context upon video mode switching, while it is not done on Linux with X11 (apparently).
- A few more minor fixes/changes.

*** Known issue: The timing may be less true to the real thing (if not to the DOSBox way), especially in the finale sequence.

P.S. I've just spotted a couple of typos in the changelog *after* uploading the archives... great. This is one of the few cases where a spell checking within the text editor could be handful. At least I have that here on the forums!
Last edited by NY00123 on Fri Jul 05, 2013 9:08 pm, edited 1 time in total.
levellass
Posts: 3001
Joined: Wed Oct 11, 2006 12:03 pm
Location: Ngaruawahia New Zealand

Post by levellass »

Huzzah! We are now much closer to full functionality.
Dr. Kylstein
Posts: 120
Joined: Wed Dec 16, 2009 5:20 pm

Post by Dr. Kylstein »

NY00123 wrote: - As a consequence, a minor bug in the image loading routine has been spotted while originally implemented. Upon fixing it, though, it has resulted in a crash... Why? Well, on vanilla Keen the image is temporarily loaded to the map data (!!), but the fix wants to update the screen. This includes an update to the on-screen map tiles. So, as a workaround/fix, no bogus tile update is done at all now.
I once tried to make a patch that would use full-screen images for titles and menus. Now finally I know why that didn't work. Do you think there's a way to back-port that fix?
NY00123
Posts: 85
Joined: Thu Sep 24, 2009 8:03 am

Post by NY00123 »

Dr. Kylstein wrote:
NY00123 wrote: - As a consequence, a minor bug in the image loading routine has been spotted while originally implemented. Upon fixing it, though, it has resulted in a crash... Why? Well, on vanilla Keen the image is temporarily loaded to the map data (!!), but the fix wants to update the screen. This includes an update to the on-screen map tiles. So, as a workaround/fix, no bogus tile update is done at all now.
I once tried to make a patch that would use full-screen images for titles and menus. Now finally I know why that didn't work. Do you think there's a way to back-port that fix?
I think that you basically want that no tiles would be drawn to screen at this stage, right?
Well, the screen update function that I have mentioned (let's call it "draw screen") does some things. Most of the job is done by a "do drawing" function called by it, applying some tasks (where some are them by even more internal sub-functions). Here are some tasks to mention, if not all:
- Apply a page flip, and also pick the correction location in Video RAM for drawing in respect to map smooth scrolling.
- Do an adaptive tile refresh (i.e. ATR). Basically, some of the on-screen tiles have been queued for update before and are now processed. It is required for animating tiles, parts of tiles revealed after a sprite moves and possibly more.
- Process a queue of sprite draws.
- Process a queue of tile draws. This seems to be used only for the Keen-exiting-via-door sequence, as well opening a locked door.
- Process a queue of bitmap draws.
- Calling a variable "draw function". Usually this is a zero pointer and nothing is done. Otherwise, it can be a function responsible for some things, like drawing the menu. Another such alternative is "mural drawing", which consists of the bitmaps displayed in the menu screen but without the menu.
- Actually update the location in Video RAM used for displaying contents, and sync to vertical blank.

Hence, while I don't know how much code lines it requires, a quick hackish way I can think of is to not apply an ATR when you want to display the image.
For a better solution, you want to avoid from calling the "draw screen" function if you don't want any tile update.

One chance of a risky thing: If you ever call the "show image" function from within one of the variable draw functions (to be stored in the variable function pointer), this is expected to result in an "infinite" recursive loop! The reason is this:
- First you set that function pointer to something, like the "draw mural" function.
- Now you let "draw mural" load an image from a file.
- Hence, when "draw screen" is called (and hence "do drawing"), at some point "draw mural" would be called as a result.
- A call to the image load function would be done.
- And that results in another call to "draw screen"...

It should be possible to temporarily set the function pointer to NULL in order to prevent that.
Dr. Kylstein
Posts: 120
Joined: Wed Dec 16, 2009 5:20 pm

Post by Dr. Kylstein »

So for a main menu/title loop I'll be fine if I just don't call draw_screen? That makes sense. (ie: why didn't I think of that?) But what if I did it during a level? Is level corruption avoidable/correctable?
NY00123
Posts: 85
Joined: Thu Sep 24, 2009 8:03 am

Post by NY00123 »

Dr. Kylstein wrote:So for a main menu/title loop I'll be fine if I just don't call draw_screen? That makes sense. (ie: why didn't I think of that?) But what if I did it during a level? Is level corruption avoidable/correctable?
Basically, if you don't call it on the middle of game-play then most things would not be updated on screen at all, like a queue of sprite draws to process and scrolling based on the current player's location on the map.
Dr. Kylstein
Posts: 120
Joined: Wed Dec 16, 2009 5:20 pm

Post by Dr. Kylstein »

What I mean is, that when I had tried this before, once the image went away the level was a garbled mess and Keen got killed by something. I was trying to replace the status box with a full screen image.
NY00123
Posts: 85
Joined: Thu Sep 24, 2009 8:03 am

Post by NY00123 »

Dr. Kylstein wrote:What I mean is, that when I had tried this before, once the image went away the level was a garbled mess and Keen got killed by something. I was trying to replace the status box with a full screen image.
Ah, right. Well, I think that you'd want to use a different location in memory than the map data for storing the image before it gets copies to the video RAM. (By default one half of the map data is used for compressed data and the other for the uncompressed, more or less.)
Don't know if there is enough memory, though...
levellass
Posts: 3001
Joined: Wed Oct 11, 2006 12:03 pm
Location: Ngaruawahia New Zealand

Post by levellass »

Can you put a large bitmap into the graphics (320x200) then store the data there in memory?
NY00123
Posts: 85
Joined: Thu Sep 24, 2009 8:03 am

Post by NY00123 »

levellass wrote:Can you put a large bitmap into the graphics (320x200) then store the data there in memory?
I suppose it can be done. For some reason I've thought that reading data from the video RAM would be problematic due to the planar structure (rather than writing), but it may not be an issue.
In the original EXE, the file should be mapped to a different location in memory - somewhere in video RAM. However, I don't know how well it is going to work due to the planar stuff...

Of course, if it can be done then two pages (or more?) could be used: One for the compressed data and another one for the uncompressed one. Flicker may occur as a side-effect unless there is enough memory to cover that.
NY00123
Posts: 85
Joined: Thu Sep 24, 2009 8:03 am

Post by NY00123 »

A new version is available. There is very little which is missing now.

*** See the first post in this thread in order to find download links for the last release.

Changelog

November 14th, 2012 (v0.4.12):
- Dependency on SDL_mixer has been removed, and SDL is used for audio directly in this version.
- More shortcut keys are implemented! These consist of F1 for the help text, F2 for sound toggling and F3 for reconfiguring keyboard assignments.
It is a good time to note that the -nosound argument still works, although it makes a few things shorter since they usually wait for a sound playback to complete. The level completion sequence is an example.
- The "clear keys" functionality hasn't done a vital part of its job, hence toggling off the scorebox with the "Space" key hasn't worked as expected (basically displaying another one immediately). That is fixed now.
- Another keyboard related change:
The keyboard's "repeat rate" and "repeat delay" are now emulated, with specific values chosen for these. Hold a key for about half a second, and then multiple presses of the same key are going to be simulated (but not releases of it).
- Settings like keyboard assignments and sound toggling are now actually saved on quit! Truly, the bug was irrelevant before since none of these could be changed up to the current revision...
- Alright, Looks like the time measurement is more accurate now. Or at least, when displaying two instances of the finale sequence side by side: One by Chocolate Keen1 and the other from a captured DOSBox video, they actually tend to stay in sync way more than beforehand!
The fix? Well, it has already occurred before with the general game timer, and now again for the simulation of VGA vertical retrace: Force the usage of 64-bit values for calculations due to (relatively) large numbers involved...
A bit more info: The simulation needs to know the refresh rate the game should "think" it has. Online specs tell that, at least on a VGA card, 70.086Hz is the refresh rate for the 200-line EGA graphics mode used by Keen 1, while VLC reports a frame rate of 70.086304fps for the captured video from DOSBox.
For now the rate of 70.086Hz is simulated.
- On a relevant note, while it is easy to think the graphics mode on EGA and VGA cards looks identical on both, there are in fact a few differences. One of them is the refresh rate. It can now be selected using the -ega argument, picking a rate of 59.92Hz (rather than the default of 70.086Hz).
- There are a few additional minor fixes again.
Last edited by NY00123 on Fri Jul 05, 2013 9:09 pm, edited 1 time in total.
Post Reply