Chocolate Keen (a "port" of Commander Keen 1-3)

Tools, assembly, and file formats.
NY00123
Posts: 85
Joined: Thu Sep 24, 2009 8:03 am

Post by NY00123 »

And... these thoughts about resource usage have made me spot clear instances of memory leaks! They've been occurring with OpenGL rendering on a few kinds of "video mode" related events, like fullscreen toggles. Furthermore, just to be on the safe side I have made another (really minor) code modification related to retrieving some... information about the (host) video hardware.

*** See the first post in this thread to find download links for the last release.

Changelog

July 7th, 2013 (v0.8.4):
- Fixed a clear memory leak that may repeat while OpenGL is in use.
User avatar
ckguy
Posts: 465
Joined: Tue Oct 14, 2003 11:20 am
Location: Wakefield, RI, US
Contact:

Post by ckguy »

Oh wow I haven't been following this at all since like November. Keen 2 and 3 support, that's awesome!
NY00123
Posts: 85
Joined: Thu Sep 24, 2009 8:03 am

Post by NY00123 »

Thanks again for checking out Chocolate Keen!

============================================
A bit of a long post, part 1 (not much about Chocolate Keen, yet)
============================================

Now, what does anybody think can be a good way to test Chocolate Keen's goal of authenticity?

That's right, reproducing vanilla demo playbacks!
Oh, but wait, is there such a thing at all? Keen 1-3 don't have this feature.

Well, no problem. The original Keen 1-3 executables (at least for version 1.31 as released by Apogee Software) do contain some unused demo-related code. With some amount of patching, these can be made useful (at the cost of losing the ability to restore saved games).

See the following link for a topic about that, with variations of the patch for all Vorticons episodes and sample demo files: http://www.keenmodding.org/viewtopic.php?t=1749

The patch has also been uploaded to the KeenWiki and added to the following page: http://www.shikadi.net/keenwiki/Patch:Demo

============================================
A bit of a long post, part 2 (now about Chocolate Keen for sure)
============================================

Anyway, how is that related to Chocolate Keen again? Well, basically, the patch's behaviors can now be optionally replicated in Chocolate Keen itself, using the new -vorticonsdemomode command line argument. So you basically have the chance to check if a demo is reproduced in the exact same way on vanilla Keen and Chocolate Keen altogether.

As expected, this leads us to a new release.

Oh yeah, for anybody interested in the SDL 2.0 executable for Windows: If it doesn't work, try adding -opengl. Alternatively add -renderer, followed by opengl or software. An earlier issue regarding sound playback should be fixed, though, thanks to an update to SDL 2.0 itself.

*** See the first post in this thread to find download links for the last release.

Changelog

July 29th, 2013 (v0.8.6):
- The bundled SDL2.dll file has been updated, solving a minor sound bug
in case the SDL 2.0 path is chosen on the Windows platform.
- A new "Vorticons Keen demo mode" has been implemented. Basically, it lets
you record and playback demos, as do vanilla Keen 4-6 (although Keen 4-6
can't just playback new demos without importing these in some way). It has
also been implemented as a patch for vanilla Keen 1-3 a bit earlier, so these
can be used for finding NON-vanilla bugs in Chocolate Keen. It is actually
the case that the vanilla Keen executables (without patching) contain
some unused code handling demos. The patch takes advantage of that.
- As a consequence of this, a fix to a function used for setting "game ticks"
(a kind of a moment measured in internal time units) has been applied.
- Furthermore, a typo in a function responsible for calculating jump heights
(and more) has been fixed, again as a result of demo playback tests.
- On an unrelated note, a minor fix to the main menu code has been applied.
- A minor error in the vorticon walk function has also been fixed
(before implementing demo playback). It's again all about == vs = in C...
- Furthermore, yet another instance of invalid memory access and a possible
memory corruption in some drawing function has been fixed, reproduced during
the Keen 3 finale when an image loaded from a file to a chunk of memory
generally used for map data is to be displayed. To anybody wondering about
the usage of map data storage for an image, that IS vanilla behavior.
NY00123
Posts: 85
Joined: Thu Sep 24, 2009 8:03 am

Post by NY00123 »

In case somebody is not entirely sure what I'm talking about, you can simply check the following video:

https://docs.google.com/file/d/0B_K_B54 ... edit?pli=1
NY00123
Posts: 85
Joined: Thu Sep 24, 2009 8:03 am

Post by NY00123 »

The following update is more about changes to functions handling graphics (and a bit more) than new features (if any). These include some optimizations and relocations of workflow from the CPU side to the GPU.

*** See the first post in this thread to find download links for the last release.

Changelog

August 18th, 2013 (v0.8.8):
- While it probably won't matter for many users of Chocolate Keen, the
licensing terms have been updated, so the whole application is (re)licensed
under the GNU GPL v2 and any later version now. Previously, only code sections
taken off the Commander Genius project had been marked as such.
- Now that SDL 2.0.0 is released, the bundled file SDL2.dll has been updated
yet again.
- This release has several changes regarding graphics handling. In fact, some
of these are not far from a rewrite. While various drawing functions have not
really changed, the situation is different when it comes to (emulated) video
mode changes, fullscreen toggling and more. These changes include restructuring
of code and some optimizations. A few specific changes are to be listed here.
- If OpenGL output and bilinear interpolation are both requested, a framebuffer
object (or FBO) is used for doing a first-step zoom (optionally based on an
input to the -scalef argument) by a GPU, if supported. The older behavior of
software zooming is applied if this approach is unsupported or the -nofbo
argument is specified. Basically, the FBO lets one use GPU resources as an
alternative to system/CPU resources, especially with a large scaling factor
(say 3 or more).
- Similarly, if an SDL 2.0 build is run and the SDL 2.0 renderer is requested,
along with bilinear filtering, then a secondary texture is used as a render
target for hardware zoom.
- When text mode is in use (e.g. when quitting via an in-game dialog),
a copy of the textual contents stored in (emulated) EGA memory is cached.
This is done so actual graphical updates are done only when ones are required.
It results in a significant decrease in CPU usage.
- When SDL 2.0 is used and a software surface is requested (using -surface),
less system memory should be consumed, at least on Linux with X11. While it may
be possible to take advantage of hardware acceleration in some way even with
the -surface argument being specified (with the cost of more memory required),
it seems like this is far from useful. -opengl or -texture should be the way
to go for hardware acceleration anyway (with SDL 2.0).
- Although not exactly tested, Chocolate Keen tries to revert to a software
surface in a few cases of problems. There is probably more to do, though.
- A dedicated function used for printing a warning or error to console has
been added, used as an alternative to direct calls to printf and fprintf.
For now this is not more useful than before.
- A long standing sound bug has finally been fixed. It is the case that the bug
has not always been reproduced, but whenever it has, it would be noticeable
during the whole Chocolate Keen session, at least with specific sound effects.
It is basically the result of a few chunks of memory, used for storing sound
effects, not getting initialized.
- Another minor sound bug has been fixed, based on the difference between
a signed mathematical operation (multiplication by -1) and a seemingly
similar unsigned operation (with a clearly different meaning).

EDIT (December 14, 2013): Wow, that's a late minor edit done to the changelog, correcting a minor typo. Chances are there are more grammatical errors...
Last edited by NY00123 on Sat Dec 14, 2013 9:37 pm, edited 1 time in total.
NY00123
Posts: 85
Joined: Thu Sep 24, 2009 8:03 am

Post by NY00123 »

Hi all,

As some of you have possibly noticed, there hasn't been an update for some time. Truly, although it's not perfect, I'd say that Chocolate Keen is mostly complete in terms of support for Commander Keen 1-3 (v1.31) on Windows and GNU/Linux desktops at the least.

Truly, I have just uploaded a small update, celebrating 23 years since the day Keen 1 became available to the public (according to Wikipedia). The changes are mainly graphical, and you might notice that something is a bit different in regards to the overscan border.

*** See the first post in this thread to find download links for the last release.

Changelog

December 14th, 2013 (v0.8.9): A small update, a 23-years anniversary release.
- Proper handling of the emulated VGA graphical 320x200 mode, which is
double-scanned to 320x400, is now done. In practice, this has an effect on
the dimensions of the overscan border. Note that this may result in some bugs.
- Some overscan border dimensions have, at least hopefully, been corrected.
Although this is a small update, the change should be noticeable visually.
- The aspect ratio of 4:3 is applied on the visual contents *without* the
overscan border now, which differs from the situation in the previous versions.
Again, this may result in bugs.
- In addition to the -egacga argument, used for emulating the behaviors of the
EGA with a CGA monitor, a new -ega argument has been added. It serves the
purposes of -egacga, just with an EGA monitor.
- SDL2.dll has been updated again (to v2.0.1). This may be the last time such
an update is ever mentioned in the changelog.
- A few more fixes have been applied to the codebase. Furthermore, A few minor
fixes and other modifications have been applied to the README file.
NY00123
Posts: 85
Joined: Thu Sep 24, 2009 8:03 am

Post by NY00123 »

Another update is up now. Most chances are that someone is going to be interested in it if a desktop color depth differing from 32-bpp is preferred (at least for software rendering via the -surface argument). Note that if it's working for anybody with a 8-bpp indexed color format, it may simply be a matter of luck. Oh, and yeah, it's still the first week of 2014.

*** See the first post in this thread to find download links for the last release.

Changelog

January 5th, 2014 (v0.8.10):
- Chocolate Keen should now be ready to work on environments with desktop color
depths differing from 32-bit; Or at least, the situation is improved with
regard to that. A few specific tests show code changes are doing their job,
although this version has not exactly been tested with truly varying desktop
color depths. Note that a different color depth may be used internally.
- The command line argument -nofbo is replaced with -nooffscreen, which is
applicable with -texture if SDL 2.0 is used, in addition to -opengl.
- With -texture in use, off screen rendering is not a must if bilinear
filtering is desired. (The latter was, in fact, the only way in v0.8.6.)
- Some more optimizations for text mode emulation are now applied.
- Sorry, but the text mode contents appearing on loading are back, although
they shouldn't be visible for long.
- A few more technical changes, here and there. Again this includes the README.
NY00123
Posts: 85
Joined: Thu Sep 24, 2009 8:03 am

Post by NY00123 »

In order to celebrate Keen Day... oh, two months too early!

Well, anyway, there is another update, bringing several interface-related changes. Although it should still be assumed that Windows and Linux desktops are the only platforms to be supported, it may still be desired by some random individual to make Chocolate Keen run in an environment with no full/laptop-sized keyboard. Even without that, I guess it'd be nice to have some change.

Hence, I'm introducing the Chocolate Keen launcher:

Image

This is a new GUI which is embedded into the Chocolate Keen executable and is drawn with SDL in a manner not that different from the game itself (for the sake of portability). Settings that could previously be changed only via the command prompt can now be modified with this. Furthermore, the last chosen settings should be saved to a configuration file named chocolate-keen.cfg on a proper quit. The command line can still be used for modifying settings and skipping the launcher, although the exact arguments differ now.

For anybody migrating from earlier versions

Please note that a copy of Keen 1 v1.31 is located at GAMEDATA\KEEN1 (or GAMEDATA/KEEN1) with the last release. If anybody wants to be able to start one of Keen 2 and Keen 3 (v1.31) then it should be found in a similar location, say GAMEDATA\KEEN2 for Keen 2.

Rewritten vanilla Keen files like saved games and the high scores table are still accessed from the same place as before (usually where the Chocolate Keen executable is located), though.

Be warned that these location may change in any new version that may (or may not) follow, so consider this a bit unstable. You can still grab version 0.8.10 for a more stable experience if you wish, though.

Known issue

- A mouse clamping bug may be spotted while using the launcher, especially after
toggling fullscreen on. It is apparently an SDL 1.2 bug, which is fixed in
SDL v1.2.15. So if an older version of SDL 1.2 is used, it is advised to
upgrade to that. As an alternative, Chocolate Keen can use SDL 2.0 as well.

*** See the first post in this thread to find download links for the last release.

Changelog

January 14th, 2014 (v0.9.0):
- A new launcher GUI has been added. It can assist while there is no access to
a full/laptop-sized keyboard. It can further assist in other ways, of course.
- Chocolate Keen specific settings are saved to chocolate-keen.cfg and are
remembered between launches.
- The usage of the command line arguments has changed, mostly mimicking the
cfg file format in a way, but also letting one skip the launcher.
- Chocolate Keen now looks for a Keen episode in a location like GAMEDATA/KEEN1
(GAMEDATA\KEEN1 on Windows).
- On the other hand, rewritable files from vanilla Keen, like SCORES.CK1, are
written to the *working directory* of the Chocolate Keen process (as before).
- A bug with text mode cursor blinking recently introduced is now fixed.
- Internal CGA/EGA/VGA font storage is modified, using 1 byte per pixel
instead of 1 bit.
NY00123
Posts: 85
Joined: Thu Sep 24, 2009 8:03 am

Post by NY00123 »

As you can see, there haven't been a lot of news on the topic. But I still have some kind of a "Keen month 2014" release.

The main update is a new OpenGL rendering path, using custom (but simple and common) shaders and powered by OpenGL 2.0 (or if you're sufficiently lucky and/or brave, OpenGL ES 2.0).

For more details check the README and/or the change log.

*** See the first post in this thread to find download links for the last release.

Changelog

March 22nd 2014 (v0.9.1):
- A new OpenGL rendering path has been added (unrelated to the SDL renderer),
powered by Open GL 2.0 / ES 2.0 and shader-based. As a consequence, it is now
possible to pick one of a few supported OpenGL versions, depending on the
environment and the Chocolate Keen executable: ES 2.0 (SDL 2.0 only), 2.0, 1.1.
- It should be noted that the OpenGL ES 2.0 path has not been tested in an
environment lacking support for desktop variants of OpenGL (say a phone).
- Some general improvements in OpenGL handling have been applied, mainly in the
way supported features of a given context are detected.
- For the few who want to experiment with this (and expect a crash or other
malformed behaviors), SDL2.dll has been updated to v2.0.3, so there is the
benefit of (theoretical) support for OpenGL ES 2 via the ANGLE library.
- If OpenGL output is chosen, with OpenGL ES 2.0 as the version, then
offscreen rendering is forced. Reason is that otherwise, a hackish alternative
involving glCopyTexSubImage2D would be in use, only it appears to be buggy on
one setup at the least (if supported at all).
- While a non-fullscreen window is not resizable, some improvements have been
done to take care of the case a window is resized anyway (possibly fullscreen).
In particular, less work should be done when a resize notification arrives.
- Fixed an infinite loop coming from a failure to get a desired fullscreen
resolution.
- Another such loop fixed, for the case a framebuffer configuration fails.
- The option "Force cut fullscreen" has been removed from the launcher UI,
although it can manually be toggled by editing the cfg file. In general, there
are *very* great chances it'll make things malfunction (if supported at all).
- Updated logging function (e.g., arguments should be printed as expected now).
- Fixed a few compilations warnings related to fprintf format string arguments.
levellass
Posts: 3001
Joined: Wed Oct 11, 2006 12:03 pm
Location: Ngaruawahia New Zealand

Post by levellass »

A nice polishing on the work.
NY00123
Posts: 85
Joined: Thu Sep 24, 2009 8:03 am

Post by NY00123 »

levellass wrote:A nice polishing on the work.
Thanks for your feedback, including the comments over at pckf.com!

As for today, it's currently Keen day 2014 plus two weeks, and another version is up.

Due to restructuring of some parts of the graphics related code, this may be buggy, but hopefully it should rather work better than before.

If there's any new feature, that's GPU palette cycling, which works like the EGA/VGA counterpart: By changing the contents of a small palette, the displayed contents are updated without modifying gfx data in system RAM from the CPU's side (although the GPU drivers may do so otherwise). Note that this feature requires direct usage of OpenGL 2.0 or OpenGL ES 2.0.

*** See the first post in this thread to find download links for the last release.

Changelog

March 28th 2014 (v0.9.2):
- GPU palette cycling has been added. Requires OpenGL 2.0 / ES 2.0 rendering
(not via an SDL renderer). It (very) roughly simulates the behaviors of EGA/VGA
adapters in mode 0Dh, where a palette of colors can be quickly modified with
no change to video memory data representing graphics.
This theoretically reduces (by a little margin) CPU usage, since the code
executed by the CPU doesn't need to handle the cycling and 8-bit -> RGBA
conversions on its own. Memory usage and bandwidth is also reduced (a bit),
since only 8-bit graphical data is stored and transferred to the graphics card
(or similar). A minor disadvantage is that the GLSL fragment shader program is
doing a bit more work.
Enabled by default (if GL 2.0 / ES 2.0 rendering is in use).
- A general restructuring of graphics handling, mainly related to window and
resource management (say creation/destruction and resizing), has been done.
This may result in unexpected bugs.
- As a consequence, with a bit of luck, it should be possible to modify more
settings from the launcher UI and apply the changes with no restart required.
- VSync may still be a problem in the OpenGL case, if SDL 1.2 is used.
- VSync for software rendering in SDL 1.2 has been added, but is totally
untested (and is, in fact, not available on Linux if an X11 server is used).
- ANGLE can be used, although a window recreation can still lead to a crash
(coming from a fullscreen toggle or getting out of the video settings submenu).
Hence, ANGLE is not in use by default.
However, the crash may occur on Wine only, and not on a real Windows setup, as
revealed by a simple (external) test case. Thanks to Lunick for checking this.
NY00123
Posts: 85
Joined: Thu Sep 24, 2009 8:03 am

Post by NY00123 »

Exactly a week has passed. Oh, and yeah, it's 404 day, I suppose.

While not significant again, at the least it should be possible to try out some abandoned mouse input mechanism now, with zero hex editing in the way.

The most noticeable change is probably the fact that the launcher UI is *not* displayed by default upon loading Chocolate Keen. I've decided that as long as there is a simple alternative way of showing the UI (say via a batch script adding a command line argument), it's probably better to be "closer to the roots" in that sense.

Most chances are the other noticeable change is the redesign of the launcher UI. Basically, I've realized that the previous codebase had probably been made to do more than necessary, leading to a bit too much thoughts about using it "correctly". At least I think the alternative codebase is somewhat simpler now, even if it can be improved.

As usual, the new release may have some new bugs introduced. Please check the last change log entry and/or readme in case of problems.

Image

*** See the first post in this thread to find download links for the last release.

Changelog

April 4th 2014 (v0.9.3):
- SDL 2.0 is the default SDL branch in use now. In particular,
chocolate-keen.exe is renamed chocolate-keen-sdl1_2.exe,
while chocolate-keen-sdl2.exe is renamed chocolate-keen.exe.
- As a side effect, a new input mapper file should be generated and edited, in
case a migration from an old SDL 1.2 exe to a recent SDL 2.0 exe is desired.
- "Back to the roots": As of v0.9.3, the launcher UI is *not* displayed on
startup by default. -showlauncher should be added as a command line argument
in order to gain access to the launcher. Alternatively, you can check out
chocolate-keen-launcher.bat (or chocolate-keen-sdl1_2-launcher.bat).
- If no command line argument is added, Chocolate Keen looks for a supported
Keen episode in a location like GAMEDATA/KEEN1. A specific episode can be
requested by adding -startkeen1, -startkeen2 or -startkeen3 as an argument.
- Realizing the code handling the launcher UI had been made to do more than
necessary (roughly being a super-simplistic GUI toolkit), it has basically
been replaced with (mostly) different code, used for launcher menus exclusively
as of this release. It can still be improved, but for now we have that.
- User video settings don't apply to the Launcher UI itself anymore. The exact
settings used for the UI are chosen to be internal safe defaults. For now they
depend on the SDL branch (1.2 vs 2.0).
- While the window is still not resizable during gameplay, with SDL 2.0 the
launcher UI window should be resizable as of this release. The main reason
the window is not permanently resizable is a kind of a technical difficulty.
- The setting "Don't forced (emulated) keyboard" has been replaced with a more
general setting, "First (emu) input device", allowing one to force (emulated)
mouse or joystick input on startup as additional options.
NY00123
Posts: 85
Joined: Thu Sep 24, 2009 8:03 am

Post by NY00123 »

Now, which feature has been considered missing since version 0.8.2?
Well, it may have some bugs and limitations, but a few snippets follow:

Image Image Image

Basically, the DOSBox-style input mapping originally introduced in v0.8.2 (that's June of 2013!) can now be configured via the launcher UI, as an alternative to manually editing a textual file.

Oh, and yeah, the keys (and other widgets) in the launcher and mapper UIs are as large as they currently are on purpose. I know it may be desired to have more elements on the display, but I can assure you there is at least one good reason for the current approach.

Known issue

- Troubles may arise if a single emulated event is mapped to two distinct
host events or more. This is the case, for instance, if physical key Left Ctrl
and a joystick button both trigger the emulation of Left Ctrl key press
in-game. Chances are the troubles are less significant if these are just
digital on/off switches which are in use (say non-analog keyboard keys and
joystick buttons), but once different host events are involved in some ways
(such as joystick axis motion or mouse motion) it may seem like certain buttons
or keys simply do no function at all, even if it's not the expected behavior.

*** See the first post in this thread to find download links for the last release.

Changelog

April 29th 2014 (v0.9.4):
- The launcher UI can be displayed in a fullscreen window again, by specifying
the command line argument -showfulllauncher (instead of -showlauncher).
Note that in the SDL 1.2 case it's just a small(er) sub-window in a fullscreen
window for now (assuming the fullscreen dimensions are greater than 640x480).
- One piece missing since v0.8.2 should finally be available:
A UI for input mapping. It is now possible to add/remove/edit mapper bindings
from the Chocolate Keen app itself (via the launcher). With that, there is no
need to fiddle with a file like chocolate-keen-mapper-sdl2_0.map manually.
- mapper_doc/how_to_use.txt has been updated to reflect this addition.
- As some words of a warning, this BREAKS compatibility with older mapper
files having too many bindings involving the exact same host side event.
Realistically, though, that hasn't really been usable from the very beginning
(think about a press on the "1" key leading to emulated presses on all keys
of a keyboard).
- Furthermore, there are very great chances that various mapper-related bugs
are hidden, here and there.
- In fact, a few such bugs dating back to v0.8.2 have been found and fixed.
For one, after writing the mapper state to a file, the file wasn't guaranteed
to be closed as expected. In addition, bindings for the mapper saving handler
(Ctrl+F8 by default) weren't getting properly loaded from the mapper file.
- As a consequence of the addition of the mapper UI, though,
this saving handler has been removed.
- Misc. (minor) UI adaptations and README file edits have been done.
NY00123
Posts: 85
Joined: Thu Sep 24, 2009 8:03 am

Re: Chocolate Keen (a "port" of Commander Keen 1-3)

Post by NY00123 »

It's been quite a while by now, but looks like someone has ported Chocolate Keen to Emscripten:

https://twitter.com/jamesfmackenzie/sta ... 1717592064
https://github.com/jamesfmackenzie/chocolatekeen

Just out of curiously, apparently there was a real problem here, possibly a crash? (There's also this old comment of mine.) https://github.com/jamesfmackenzie/choc ... ceb334c4a8
Post Reply