TED5 Replacement - KEDfW (Keen Editor for Windows)

Utilities for editing Keen:Galaxy and Keen:Dreams levels.
User avatar
Boeing747
Posts: 76
Joined: Fri Apr 16, 2004 11:24 am
Location: Germany
Contact:

Post by Boeing747 »

All done. Try it out! www.plsplayer.de/kedfw/KEDfW.rar
KeenRush
Patch Maker
Posts: 1988
Joined: Sun Aug 31, 2003 2:52 pm
Location: Sand Yego
Contact:

Post by KeenRush »

Wow, fast progress!

Gotta say, the latest version looks really nice, keep up the good code. :)
User avatar
Boeing747
Posts: 76
Joined: Fri Apr 16, 2004 11:24 am
Location: Germany
Contact:

Post by Boeing747 »

Only missing: Charmacizing and Compiling into GAMEMAPS.CKX :o(
KeenRush
Patch Maker
Posts: 1988
Joined: Sun Aug 31, 2003 2:52 pm
Location: Sand Yego
Contact:

Post by KeenRush »

Yeah, that'll be tricky one, indeed. You could ask adurdin for help when he has some spare time; he seems to be really busy at the moment.

This should explain the format (entirely, I guess), if you can understand it, that's pretty good:
http://www.spatang.com/cknews/apr7_02.txt
(Read 'Plumbing the Depths of Keen'..)
User avatar
Boeing747
Posts: 76
Joined: Fri Apr 16, 2004 11:24 am
Location: Germany
Contact:

Post by Boeing747 »

I'm reading it but unfortunately perhaps i need the compression algoritm
in VB6 source. But it is porsible that e.g.: adurdin makes an w32pe
exe that compresses the map within commandlinearguments.
I'm not so good in math. Please help me and give me a sample
to compress with RLE dword :o(. But i'll try translating this into VB6:

While there is input and the output buffer is not full:
Read datum (word)
If datum is 0xFEFE Then:
Read count (word)
Read datum (word)
Do count times:
Write datum (word)
Else:
Write datum (word)
Repeat.

AND

Set length to input length
Write length (dword)
Set count to one
Read lastdatum (word)
While there is input:
Read datum (word)
If datum equals lastdatum Then:
Increment count
Else:
If count >= 4 Or lastdatum is 0xFEFE Then:
Write 0xFEFE (word)
Write count (word)
Write lastdatum (word)
Else:
Do count times:
Write lastdatum (word)
Set count to one
Set lastdatum to datum
Repeat.

My questions: What is the outputbuffer? Size?
Read datum (word) is it the line in vB6: Get #1, , LongInput ?
Increment count: count = count - 1 or + 1??
dword = double or single? I think double!
KeenRush
Patch Maker
Posts: 1988
Joined: Sun Aug 31, 2003 2:52 pm
Location: Sand Yego
Contact:

Post by KeenRush »

A dword means double world, which is the same than using 'long' in VB when saving bytes. But those others, I have no idea, too bad. :(
User avatar
Boeing747
Posts: 76
Joined: Fri Apr 16, 2004 11:24 am
Location: Germany
Contact:

Post by Boeing747 »

I'd made a new project called RLE.
Using one Module "modMain" with sub main() in it.
sub Main() collects information file1 file2 -param
params: /s = silent mode
-d decompressing mode
Code of Module modMain:

Code: Select all

Command$ = """D:\Daten\Programs\Keen Projects\KEDfW\Graphics\Tiles\tiles.kdt"" ""D:\Daten\Programs\Keen Projects\KEDfW\Graphics\Tiles\ttttiles.bmp"" -d"

Code: Select all

Dim fn As String, fn2 As String, silent As Boolean

Sub Main()
cmd = Command$
If InStr(cmd, "/s") Then silent = True
If InStr(cmd, "-d") Then mode = 1
If cmd = "" Then
  If Not silent Then
    MsgBox "Error: No filenames were specifed!", vbCritical, "Warning": End
  Else
    End
  End If
End If
pos% = InStr(cmd, """")
If pos% = 0 Then GoTo noquot1
fn = Mid(cmd, pos% + 1)
pos% = InStr(fn, """")
fn = Mid(fn, 1, pos% - 1)
succeed1:
rest = Mid(cmd, pos% + 4)
pos% = InStr(rest, """")
If pos% = 0 Then GoTo noquot2
fn2 = Mid(rest, 1, pos% - 1)
succeed2:
'MsgBox """" & fn & """"
'MsgBox """" & fn2 & """"
Processing mode

Exit Sub
noquot1:
pos% = InStr(cmd, " ")
If pos% = 0 Then
  If Not silent Then
    MsgBox "Invalid parameter " & cmd, vbCritical, "Warning"
    Form1.Show
    End
  Else
    End
  End If
End If
fn = Mid(cmd, 1, pos% - 1)
GoTo succeed1
Exit Sub
noquot2:
pos% = InStr(cmd, " ")
If pos% = 0 Then
  If Not silent Then
    MsgBox "Invalid parameter " & cmd, vbCritical, "Warning"
    Form1.Show
    End
  Else
    End
  End If
End If
fn2 = Mid(cmd, pos% + 1)
GoTo succeed2
End Sub

Sub Processing(mode)
Select Case mode
  Case 0 'Compress
    Dim inp As Integer, lastdatum As Integer, datum As Integer, count As Long
    Open fn For Binary Access Read As #1
      Open fn2 For Binary Access Write As #2
        Put #2, , CLng(LOF(1))
        count = 1
        Get #1, , lastdatum
        Do
          Get #1, , datum
          If lastdatum = datum Then
            count = count + 1
          Else
            If count >= 4 Or lastdatum = &HFEFE Then
              Put #2, , CLng(&HFEFE)
              Put #2, , CLng(count)
              Put #2, , lastdatum
            Else
              count = count + 1
              Put #2, , lastdatum
            End If
          End If
          count = 1
          Get #1, , lastdatum
        Loop Until EOF(1)
      Close #1
    Close #2
  Case 1 'Decompress
    Dim co As Integer, flen As Long
    Open fn For Binary Access Read As #1
      Open fn2 For Binary Access Write As #2
        Get #1, , flen
        For I = 0 To flen
          Get #1, , datum
          If datum = &HFEFE Then
            Get #1, , co
            Get #1, , datum
            For n = 0 To co
              Put #2, , datum
            Next
          Else
            Put #2, , datum
          End If
        Next
      Close #1
    Close #2
End Select
End
End Sub
But it has a bug. After decompring the file has the double size of the
original file + 1kb. When I'm look in the file: On the start till the
comporessed size there are the same bytes as in the compressed file.
The rest are zerobytes. What's wrong? :o(
User avatar
Boeing747
Posts: 76
Joined: Fri Apr 16, 2004 11:24 am
Location: Germany
Contact:

Post by Boeing747 »

Finished. It works. But only with keen1 2 3 levels. other files starts
correctly but on the end they are corrupt. It could be a cause of word by
word working RLE compression, not byte for byte.
The whole code of my RLE module again:

Code: Select all

Dim fn As String, fn2 As String, silent As Boolean

Sub Main()
cmd = Command$
If InStr(cmd, "/s") Then silent = True
If InStr(cmd, "-d") Then mode = 1
If cmd = "" Then
  If Not silent Then
    MsgBox "Error: No filenames were specifed!", vbCritical, "Warning": End
  Else
    End
  End If
End If
pos% = InStr(cmd, """")
If pos% = 0 Then GoTo noquot1
fn = Mid(cmd, pos% + 1)
pos% = InStr(fn, """")
fn = Mid(fn, 1, pos% - 1)
succeed1:
rest = Mid(cmd, pos% + 4)
pos% = InStr(rest, """")
If pos% = 0 Then GoTo noquot2
fn2 = Mid(rest, 1, pos% - 1)
succeed2:
'MsgBox """" & fn & """"
'MsgBox """" & fn2 & """"
Processing mode

Exit Sub
noquot1:
pos% = InStr(cmd, " ")
If pos% = 0 Then
  If Not silent Then
    MsgBox "Invalid parameter " & cmd, vbCritical, "Warning"
    Form1.Show
    End
  Else
    End
  End If
End If
fn = Mid(cmd, 1, pos% - 1)
GoTo succeed1
Exit Sub
noquot2:
pos% = InStr(cmd, " ")
If pos% = 0 Then
  If Not silent Then
    MsgBox "Invalid parameter " & cmd, vbCritical, "Warning"
    Form1.Show
    End
  Else
    End
  End If
End If
fn2 = Mid(cmd, pos% + 1)
GoTo succeed2
End Sub

Function GetSS(ByVal sString As String, ByVal Space As String, ByVal Index As Long) As String
On Error Resume Next
Dim aArray() As String
aArray = Split(sString, Space)
GetSS = aArray(Index - 1)
Erase aArray()
End Function

Sub Processing(mode)
Select Case mode
  Case 0 'Compress
    Dim inp As Integer, lastdatum As Integer, datum As Integer, count As Integer
    Open fn For Binary Access Read As #1
      Open fn2 For Binary Access Write As #2
        Put #2, , CLng(LOF(1))
        count = 1
        Get #1, , lastdatum
        Do
          Get #1, , datum
          If lastdatum = datum Then
            count = count + 1
          Else
            If count >= 4 Or lastdatum = &HFEFE Then
              Put #2, , CInt(&HFEFE)
              Put #2, , count
              Put #2, , lastdatum
            Else
              For i = 0 To count - 1
                Put #2, , lastdatum
              Next
            End If
            count = 1
            lastdatum = datum
          End If
        Loop Until EOF(1)
      Close #1
    Close #2
  Case 1 'Decompress
    Dim co As Integer, flen As Long
    Open fn For Binary Access Read As #1
      Open fn2 For Binary Access Write As #2
        Get #1, , flen
        Do
          Get #1, , datum
          If datum = &HFEFE Then
            Get #1, , co
            Get #1, , datum
            For n = 0 To co - 1
              Put #2, , datum
            Next
          Else
            Put #2, , datum
          End If
        Loop Until EOF(1)
      Close #1
    Close #2
End Select
End
End Sub
The whole sourcecode of KEDfW and RLE:
http://www.plsplayer.de/kedfw/kedfwsrc.rar
User avatar
Boeing747
Posts: 76
Joined: Fri Apr 16, 2004 11:24 am
Location: Germany
Contact:

Post by Boeing747 »

Now i need a samplecode for CHARMACIZING and A Sourcecode for
the GAMEMAPS.CKe. I could not understand the format in this Text
but only a few points. GAMEMAPS.CKe is first Charmackized and
then recompressed with RLE, I understood. And Charmackizing
is a compression similar to RLE but it compresses a wordorder like
EF0C D0E3 EF0C D0E3 EF0C D0E3
Repeat words EF0C, D0E3 3 times.
But i can't mace an compression algorithm for this.
It would take a long time. The second problem is the
MAPHEAD file. But adurdin programmed CKePARCH.exe
My compiler could call this program.
I think only andy can help me with an examble for charmackize
like the examble in the text above for RLE. Better is: An examble
written in VB6 code.

I'll add cut copy and paste to my editor while
I'm waiting for the codes, I need as descriped above.

IMPORTANT NOTE
The KDL-format has changed.
I noticed that he Levelname have not to be longer than 16 bytes.
I thought 30 bytes. The Level header changed. To convert your
created levels to version KDLEVV1 change the 0 in the Header to 1
and temove the Bytes of the Name to fit 16 bytes. But the Icons
are now saved as words, not as bytes, because of the logical infos
like in ted5.
User avatar
adurdin
Site Founder
Posts: 549
Joined: Fri Aug 29, 2003 11:27 pm
Location: Edinburgh, Scotland
Contact:

Post by adurdin »

I didn't write CKPatch -- that was AdmiralBob. But anyway...

Two reasons why your RLE compression/decompression doesn't work on Keen 4, 5, and 6 files: one, you need to de-carmacize first, and two, the flag that marks an rle sequence is 0xABCD.

Anyway, as far as compression goes, you don't need to have a real carmacizing compression algorithm, you can fake it quite easily. Keep in mind that when KeenX.exe is trying to de-carmacise, it will only do something special if it finds a word with a high byte of 0xA7 or 0xA8. So to fool it, you need to make sure that these don't exist in the input. This is a simple as replacing occurrences of a word 0xA7nn or 0xA8mm with the three bytes 0x00 0xA7 0xnn or 0x00 0xA8 0xmm.
In other words, you'd be leaving the level data un-carmacized, but making KeenX.exe think it's carmacized.
In terms of VB code, it'd be something like this:

Code: Select all

...
Get #1, , worddata
If (worddata AND &HFF00) = &HA700 Or (worddata AND &HFF00) = &HA800 Then
    bytedata = (worddata AND &HFF)
    worddata = (worddata AND &HFF00)
    Put #2, , worddata
    Put #2, , bytedata
Else
    Put #2, , worddata
End If
...
You could also similarly fake the RLE compression (by writing out the words 0xFEFE 0x0001 0xFEFE to the output whenever an input word happens to be 0xFEFE).
User avatar
Boeing747
Posts: 76
Joined: Fri Apr 16, 2004 11:24 am
Location: Germany
Contact:

Post by Boeing747 »

That sounds good. But I have questions:
- What about the error message "MAP TOO TALL!".
A charmackizing fake would affect a large mapfile.
What would cause when the file becomes too large?

- I didn't understand well: GAMEMAPS.CKe is first compressed
RLE and then compressed Charmack? What must be done first and
second?

- I need a code for decharmackizing. I'd like to offer users to
load maps of original CK Episodemaps for modification or testing.

- INFO: I can test the faked charmackizing only after collecting solid
information about the mapformat. I didn't understand the planes.

Thanks for the information you gave me. I'll think about it.
Yesterday I thought my project is not finishable. But now I've go
more courage to complete my project.
KeenRush
Patch Maker
Posts: 1988
Joined: Sun Aug 31, 2003 2:52 pm
Location: Sand Yego
Contact:

Post by KeenRush »

I think it's first compressed with RLE, then with Carmacizing. In editor.
And then Keen exe does unCarmacizing, and then unRLE.
Thanks for the information you gave me. I'll think about it.
Yesterday I thought my project is not finishable. But now I've go
more courage to complete my project.
Heh, you've got this far so it's possible. :) Good luck, too bad I can't be more helpful. :(
User avatar
Boeing747
Posts: 76
Joined: Fri Apr 16, 2004 11:24 am
Location: Germany
Contact:

Post by Boeing747 »

It's very unlogical. First RLE then Charmack... RLE will make charmack
slover and the compressionratio will be bad.

To make a stable project I'd like follows:
- Full code for Charkmack/unCharmack
- Same for RLE
- Format specifications again. (How are the particulary maps stored?, ...)

Then I'm ready for a compiler soon.
User avatar
adurdin
Site Founder
Posts: 549
Joined: Fri Aug 29, 2003 11:27 pm
Location: Edinburgh, Scotland
Contact:

Post by adurdin »

Boeing747 wrote: That sounds good. But I have questions:
- What about the error message "MAP TOO TALL!".
A charmackizing fake would affect a large mapfile.
What would cause when the file becomes too large?
That error is caused by (among other things) the uncompressed size of the level being too big (to fit in memory). So if the data file is uncompressed, it won't make any difference.
- I didn't understand well: GAMEMAPS.CKe is first compressed
RLE and then compressed Charmack? What must be done first and
second?
To decompress the levels: first de-Carmacize, then de-RLE.
To compress the levels: first RLE, then Carmacize
To fake compressing the levels instead: first fake RLE, then fake Carmacize (although both of these could happen at the same time)
- I need a code for decharmackizing. I'd like to offer users to
load maps of original CK Episodemaps for modification or testing.
You can find C source code for carmack compression and RLE compression in the TED5 source code (from 3drealms website). You can find C source code for carmack decompression and RLE decompression in the Wolfenstein 3D source code (also from 3drealms website, IIRC).
AFAIK there is no VB source code for either algorithm available.
- INFO: I can test the faked charmackizing only after collecting solid
information about the mapformat. I didn't understand the planes.
Play around with using TED5 for a while, to get a feeling for the planes. Then re-read the "Plumbing the depths of Keen" article, and/or the (unfinished) keen file format doc, while looking at the gamemaps file in a hex editor. That's the best way to understand the map format.
It's very unlogical. First RLE then Charmack... RLE will make charmack
slover and the compressionratio will be bad.
No, it won't. The RLE will reduce long sequences (e.g. 11111111) into something short (e.g. {flag 8 x "1"}). This compresses the "empty space" that comprises most of the level fairly efficiently. The Carmack compression on the other hand only works on patterns repeated more than once (e.g. 12345671234568), changing them into something shorter (e.g. 1234567 {flag repeat first 6 of last 7} 8).
The carmack compression compresses the background layer of Keen most efficiently, because that's where the most pattern duplication is.
(Though why ID didn't use LZ or Huffman compression I don't know -- the former is best, and is used for graphics in Keen 1; the latter is used for graphics in Keen 4, 5, 6, D).
User avatar
Boeing747
Posts: 76
Joined: Fri Apr 16, 2004 11:24 am
Location: Germany
Contact:

Post by Boeing747 »

Thanks! The (unfinished) keen file format doc is better to understand.
I'll make a module with Type declarations.
But one question:
PlaneOffsets(0 to 2) As Integer is the same as:

Code: Select all

Type PlaneOffsets
  PlaneOffS1 As Integer
  PlaneOffS2 As Integer
  PlaneOffS3 As Integer
End Type
?

Thanks again.

How can I make this bug similar in VB6?

Code: Select all

  The presence of Junk is due to a bug in TED5.  Here is a simple equivalent this bug:

    char mapidstr[8] = "TED5v1.0";
    ...
    fwrite( mapidstr, strlen( mapidstr ), 1, file );

  As you can see, there is no terminating null in the array, so TED5 just keeps writing junk until it
  finds one in memory.

Code: Select all

Dim mapidstr As String * 8
mapidstr = "TED5v1.0"
Put #1, , mapidstr
But VB6 doesn't see this String as an 0 to 7 array.
Image
What are those "ABC-Words" in gamemaps?
and in this TED5 file, i could not see any junkbytes.

Here my first release of my TED5 Module:

Code: Select all

' Ted5 MODULE
' by Boeing747

' _______________________

Public Type GAMEMAPS_HEAD
  Signature As String * 8 ' Always set to "TED5v1.0"
  Junk() As Byte ' ???
End Type

Public Type PLANE_DATA
  Length As Integer
  DataArray() As Byte ' Charmack and RLE will worked out in _
                        a TEMPFILE this file will be loaded _
                        into the DATAARRAY!
End Type

Public Type GAMEMAPS_LEVEL
  Plane_CompressedData(0 To 2) As PLANE_DATA
  Plane_Offsets(0 To 2) As Long
  Plane_Lenghts(0 To 2) As Integer
  Width As Integer
  Height As Integer
  Name As String * 16
  HEAD_END As String * 4 ' Always set to "!ID!"
End Type
Post Reply