DM2_SKSAVE_Converters

Under construction. Last update: 2010/11/03 14:58 +0900

Overview

• Port sksave and skload procedure and subsequent procedures in DM2 PS/V to windows.
• Finally having a windows prot of DM2.

Development

2006.03.23

A screenshot of test tool. It is log reader. The log describes procedure-to-procedure call history. My special version of DOSBox generates it while playing the game.

Each tree item is a procedure. xxxx:yyyy means procedure's far pointer address in i386 real mode. It says where the procedure starts in computer memory then.

You can see some procedures have meaningful names such as "FIRE_BLIT_PICTURE". It is an analyzed procedure on its code working and brought results.

The above screenshots say many many (over 300) procedures should be converted from disasm codes to C/C++ compatible code. It is to partially reproduce the DM2 behavior, such as game load/save routine work. Disasm is brought by debug version of DOSBox. Many thanks to DOSBox community! It is very nice feature.

Currently, the code conversion is done by hand work. Human's brain is essential to do code conversion. Partially computer helps to generate formalized/normalized part such as conditional jump meaning such as "if AX != FFE0" against "cmp ax,FFE0", and "if ( AX | 0000 )==0" against "or ax,0000".

The following is disassembled source code of FIRE_BLIT_PICTURE procedure in unlzexec'd FIRE.exe.

FIRE_BLIT_PICTURE

Next pair of procedures should have been first successful conversion of this proj.

Disasm code
C/C++ compatible code

2006.03.25

Here is a sample of procedure call history tree.

A procedure call tree (partial, not entire)

CREDITS: source code html coloring and formatting is presented by htmlize.

2006.04.15

I'm developing the a conversion way which makes it easy to convert from ASM to C.

exp00240 conversion 0.1 sample

2006.04.17

I crafted a conversion tool with another thought. Just "I work for conversion, so shorten the total conversion time cost!"

exp00240 conversion 0.2 sample

2006.05.03

The progress report has moved to dedicated page. here is a progress paper.

2006.05.07

A procedure call tree updated. Investigating how to decode compressed rects chunk.

2006.05.14

A procedure call tree updated.

2006.05.21

A procedure call tree updated.

2006.05.27

A procedure call tree updated.

2006.06.10

A procedure call tree updated.

2006.06.17

A procedure call tree updated.

2006.06.18

A procedure call tree updated.

2006.06.22

A procedure call tree updated.

2006.07.01

A procedure call tree updated.

New: _0b36_129a, _1031_01d5, _1031_024c, _1031_030a, _1031_03f2, _1031_0a88, _1031_0c58, _1031_0d9e, _1031_10c8, _1031_111e, _1031_156f, _2e62_0670, HANDLE_UI_EVENT, PT_IN_RECT

2006.07.04

It is to shorten the conversion cost per procedure function. I continue to improve the helper tools to do it faster.

2006.07.07

A procedure call tree updated.

New: _0b36_1688, _2405_00ec, _2405_011f, _24a5_0e82, _24a5_0fb8, _24a5_1532, _2e62_00a3, _2e62_011b, _2e62_02ea, _2e62_0360, _2e62_0572, _2e62_05d4, _2e62_061d, _2e62_064a

2006.07.09

A procedure call tree updated.

New: _0cee_279d, _12b4_0141, _13ae_0009, _1c9a_3a6e, _2405_0029, _2405_005e, _2405_00c9, _2405_00e7, _2405_02e8, _2405_04f4, _24a5_000f, _24a5_0a06, _24a5_0ad2, _24a5_0b0f, _24a5_0c35, _2759_0155, _2c1d_06b5, _2c1d_073d, _2c1d_0889, _2c1d_0ea2, _2e62_0265, _2e62_03b5, _2e62_0cfa, _3929_07e1, _3929_0826, _3929_0914, _3929_0929, _3929_09fb, _3929_0b5a, _3929_0c37, _3a15_0f07, _3a15_0fa5, _3a15_38b6, _48ae_04b7, F_LMOD@, FILL_STR, FORMAT_SKSTR, GET_MAX_CHARGE, IS_MISCITEM_CURRENCY, RESET_SQUAD_DIR, SK_LTOA10, SK_STRCAT, SK_STRCPY
Removed:_0cd5_0018, _0cee_29b9

2006.07.12

A procedure call tree updated.

New: _1031_16a0, _24a5_1798, _2759_222e, _2759_274d, _2f3f_0000, _2f3f_04ea, _32cb_06cf, _443c_0004, _443c_0434, FIND_SPELL_BY_RUNES

2006.07.16

A procedure call tree updated.

New:_01b0_087a, _01b0_0d39, _075f_000b, _075f_1f33, _0b36_1757, _0cee_159b, _0cee_3202, _121e_0422, _13e4_0360, _1c9a_0cf7, _1c9a_103a, _1c9a_11bb, _24a5_10fb, _24a5_174d, _2759_0589, _2759_0602, _2759_0f39, _2759_1204, _2759_14c8, _2759_15d0, _29ee_0b4a, _2c1d_0b2c, _2c1d_0e23, _2c1d_0e57, _2c1d_0f67, _2c1d_111e, _2c1d_11bf, _2c1d_1cf3, _2c1d_1d64, _2c1d_1de2, _2fcf_183c, _2fcf_2da7, _443c_03f4, _443c_08ab, _48ae_0001, _48ae_0058, ADD_RUNE_TO_TAIL, ALLOC_NEW_CREATURE, DRINK_MISCITEM_WATER, ENGAGE_COMMAND, ENGAGE_X_TELEPORTER, FIND_HAND_WITH_EMPTY_FLASK, OVERSEE_RECORD, QUERY_CUR_CMDSTR_ENTRY, QUERY_GDAT_FOOD_VALUE_FROM_RECORD, REMOVE_RUNE_FROM_TAIL, SET_DESTINATION_OF_MINION_MAP
Removed:

2006.07.17

A procedure call tree updated.

Some procedures are removed due to refresh of exist procedure call history.

New:_0cee_2e09, _0cee_317f, _32cb_0287, ALLOC_UPPER_MEMORY, FIND_DISTINCTIVE_ITEM_ON_TILE, FIRE_BOOTSTRAP, GET_WALL_TILE_ANYITEM_RECORD, IBMIO_BOOTSTRAP, IS_WALL_ORNATE_SPRING, MOVE_RECORD_AT_WALL
Removed:_01b0_087a, _01b0_0d39, _0cee_279d, _2066_1ea3, _2066_1ec9, _2066_1f37, _2066_1fab, _24a5_0a06, _24a5_0ad2, _24a5_0b0f, _24a5_10fb, _24a5_1532, _24a5_174d, _2759_0155, _2759_15d0, _29ee_0b4a, _2f3f_0000, _2f3f_04ea, _3a15_0fa5, _443c_03f4, _443c_04b6, _443c_08ab, _48ae_04b7, DRINK_MISCITEM_WATER, FIND_HAND_WITH_EMPTY_FLASK, FIRE_MOUSE_SET_POS, IBMIO_MOUSE_SET_POS, IS_MISCITEM_CURRENCY, QUERY_GDAT_FOOD_VALUE_FROM_RECORD, REMOVE_RUNE_FROM_TAIL, ROTATE_RECORD_BY_TELEPORTER, RUN_ACTUATOR_TICK_GENERATOR

2006.08.06

Generated a per segment address procedure sort.

2006.09.12

New:_01b0_04e4, _075f_084d, _075f_1791, _075f_1bc2, _0cd5_0039, _0cee_3275, _1031_06a5, _12b4_023f, _12b4_0300, _12b4_0881, _12b4_0953, _12b4_099e, _12b4_0bf1, _13e4_0004, _13e4_0401, _14cd_0802, _19f0_000a, _19f0_0081, _19f0_00b8, _19f0_01d6, _19f0_0207, _19f0_045a, _19f0_04bf, _19f0_050f, _19f0_0547, _19f0_0559, _19f0_05e8, _19f0_0891, _19f0_0d10, _19f0_13aa, _19f0_1511, _19f0_1522, _1c9a_0006, _1c9a_0551, _1c9a_0598, _1c9a_0648, _1c9a_0e14, _1c9a_19d4, _1c9a_1a48, _1c9a_1b16, _2066_32bb, _24a5_036a, _2c1d_028c, _2c1d_105f, _2fcf_0c7d, _476d_050e
 

2006.11.04

Converted the depends procedures started from GAME_LOAD procedure. Now converting the fire_main procedure.

2007.01.03

Experimental procedure-to-procedure call tree graph available: here is a graph paper

New:_00eb_05c7, _00eb_0bc4, _01b0_08b6, _01b0_0e80, _01b0_1be8, _01b0_1ed2, _01b0_1ffc, _01b0_20ca, _01b0_20ef, _01b0_2b1b, _075f_11ba, _075f_1c74, _0b36_020e, _1031_07d6, _13ae_005c, _13e4_01a3, _13e4_0982, _13e4_0cf2, _14cd_0009, _14cd_0067, _14cd_0276, _14cd_0389, _14cd_0457, _14cd_0550, _14cd_062e, _14cd_0684, _14cd_0815, _14cd_08f5, _14cd_09e2, _14cd_0f0a, _14cd_0f3c, _14cd_102e, _14cd_10d2, _14cd_1316, _14cd_18cc, _14cd_18f2, _14cd_19a4, _14cd_19c2, _14cd_1a3c, _14cd_1a5a, _14cd_1a78, _14cd_1b74, _14cd_1bac, _14cd_1c45, _14cd_1c63, _14cd_1c8d, _14cd_1cec, _14cd_1d42, _14cd_1eec, _14cd_1f8b, _14cd_1ff3, _14cd_2162, _14cd_221a, _14cd_226d, _14cd_22a3, _14cd_232b, _14cd_2398, _14cd_240e, _14cd_248d, _14cd_25b8, _14cd_2662, _14cd_274f, _14cd_2807, _14cd_2886, _14cd_28d2, _14cd_2c23, _14cd_2e6e, _14cd_2f07, _14cd_3087, _14cd_3139, _14cd_32a4, _14cd_33e1, _14cd_3474, _14cd_34d3, _14cd_36b3, _14cd_3aee, _1887_00f8, _1887_0205, _1887_0239, _1887_05aa, _1887_05f3, _1887_150c, _1c9a_0732, _1c9a_078b, _1c9a_166f, _1c9a_37be, _1c9a_381c, _1c9a_38a8, _1c9a_3c30, _2405_0009, _2481_0002, _2481_007d, _2636_03d4, _32cb_0008, _38c8_00c8, _38c8_0104, _38c8_0109, _38c8_0224, _38c8_04aa, _3929_0e16, _3a15_072f, _3a15_0763, _3a15_2ca8, _3a15_3a9e, _3e74_0004, _3e74_04d8, _3e74_059e, _3e74_07b2, _3e74_0820, _3e74_1175, _3e74_1330, _3e74_16ed, _3e74_1d5f, _3e74_2162, _3e74_216a, _3e74_22cf, _3e74_2439, _3e74_24b8, _3e74_2641, _3e74_2ec7, _3e74_5788, _443c_0380, _443c_067a, _44c8_1baf, _470a_0003, _4726_02f7, _4726_03b2, _476d_018a, _476d_02e0, _476d_04f4, _47eb_0003, _482b_0004, _4937_0320, _crt_brk, _crt_coreleft, _crt_CreateHeap, _crt_farcoreleft, _crt_farmalloc, _crt_getvect, _crt_normalize, _crt_setblock, ADD_COIN_TO_WALLET, EMS_ALLOC_POOL , FIND_TILE_ACTUATOR, FIRE_main, GET_FILE_POS_6, GET_FILE_SIZE, KANJI_FONT_LOAD, OPERATE_PIT_TELE_TILE, WOUND_CREATURE
Removed:_13ae_02a3, IDLE

2007.01.07

here is a graph paper and here is a progress paper updated.

New:_075f_18fd, _13e4_071b, _13e4_0806, _13e4_08ff, _1887_09ab, _1887_0ce1, _1887_0d33, _1887_0df7, _1887_0e19, _1887_0eca, _1887_0f80, _1887_0fb9, _1887_10db, _3a15_07b4, _3a15_0acd, _3a15_10dc, _3a15_12b8, _3a15_13cb, _3a15_16ec, _3a15_1da8, _3a15_1ea8, _3a15_220c, _4937_028a, QUERY_DOOR_STRENGTH
Removed:
 

2007.02.12

Today I succeeded to start debugging DM2_SKSAVE_Converter on Visual Studio NET!

It is time to find and fix lying errors with running code. The converted source code has 57,054 lines for now.

My working processes:

• Convert minimal functions to make compiler/linker successful → Ok
• Prepare minimal variable declaration and implementation → Ok
• Tweak Skullkeep memory allocator and memory pool functionality → Ok
• Tweak Skullkeep GRAHICS.dat loader → Ok
• Tweak raw entry loader and locator → Ok
• Tweak set of rectangle loader and memory compressor → Already ok
• Tweak Kanji font loader → OK
• Implement some missing BitBlt functions → WIP
• ...
• Implement OS dependent UI features: keyboard, mouse, display.

Finally it got on mini frame.

2007.02.20

Here are some comments on my conversion work.

• The source code is too large to manage:
  • I have appended converted code bits on demand. It is stored in one source file "A.cpp".
  • "A.cpp" grows to 1.24MBytes (59,019 lines) now.
  • The source code editor (Visual Studio .NET) works slowly due to heavy file size. IMHO It may take 1.5 times to repair the problem code bit in "A.cpp" when I notice any errors.
  • It takes 3 seconds for compile/linker process in best case. Of course, it is within commonsense time cost.
• The quality of converted code bits:
  • Though the majority parts are correctly converted, sometimes the minor part contains critical errors. Mainly 4 patterns or more
    (1)wrong conditional jump conversion
    (2)incorrect order of parameters
    (3)lack to convert
    (4)lack of variable initialization.
  • The errors in minor part may (1)collapse stable status of running program (2)take side effects in another correct code bit rather than the wrong code bit.
• Find the errors how:
  • Insert assertion code as possible as many. Place the check code at back and forward of common purpose procedure invocation such as memory allocation.

The below is comparison sample. The left view is before edit (wrong program). The right view is after edit (fixed program). It shows that the order of function call parameter was incorrect.

The below one is also comparison sample. The wrong comparison code "si > bp0a" is corrected to right code "si >= bp0a". As the side effect of wrong conversion, this code breaks data memory widely, then rest process of running program brought unexpected results. Finally running code has been caught by Windows.
This code bit is part of DECODE_IMG3_OVERLAY procedure.

The below one says "bp01 += 2" was lack on left code bit.

It is effective to specify what is acceptable condition. For example, memory allocation checks the buffer size: must be greater equal than 0 bytes and less than 60,000 bytes.
The validation works only on executable program built on "debug build". All the validation code is gone on "release build". So the validation code does not affect the quality of executable program built for final release.

Now it implements mouse support. It can start the game and increase the game tick now. The message is displayed in Japanese because this port is based on DM2 PS/V version.
That language difficulty will be solved on near future effectively.

It still has numerous problems to play the game on usual way. The game often freezes rather than program abort on usual play.

2007.02.25

It gets better by doing bug fix.

I often proceed comparison behavior both DM2 PS/V on DOSBox emulator and ported DM2_SKSAVE_Converter by running simultaneously when I cannot find what is bug easily. This is very terrible work!

Next one shows proceeding comparison by output numerous debug information on console window. The red mark shows just one difference which I want to know.
The left console window is attached to DM2_SKSAVE_Converter. The right notepad shows output of DOSBox. I injected some debug codes into DOSBox for this cheat.

The mistake was very easy. However it costs so much time to locate them.

2007.07.26

Accidentally I knew my procedure sort page is hit when you google with "F_PADD" keyword. So I decided write some of what they are, with C languange and the lying x86 machine language manner.

The following procedures are provided by Borland's Turbo C++ compiler for MS-DOS programs. Their procedures are specific for x86 and 16-bits MS-DOS mode, to perform effective pointer operation. That characteristic is called as A20 address line mode Real-address mode model. (You can check it at "3.3. MEMORY ORGANIZATION" in Intel Architecture Software Developer's Manual PDF document.)

* F_xxx is far address procedure. N_xxx is near address procedure.

F_PADD, N_PADD

PADD

Input: DX:AX ... your far pointer
Input: CX:BX ... your offset for add operation. 32 bits signed integer
Output: DX:AX ... normalized result far pointer

Sample source code bit to use that procedure.

Disassembled by Turbo Debugger

F_PSUB, N_PSUB

PSUB

Input: DX:AX ... your far pointer.
Input: CX:BX ... your offset for subtract operation. 32 bits signed integer.
Output: DX:AX ... normalized result far pointer.

F_PADA, N_PADA

PADA

Input:          DX:AX  ... far pointer to your far pointer variable.
Input: far ptr [DX:AX] ... your far pointer variable.
Input:          CX:BX  ... amount for add operation. 32 bits signed integer.
Output:          DX:AX  ... normalized result far pointer.
Output: far ptr [DX:AX] ... normalized result far pointer.

F_PINA, N_PINA

PINA

Input:          ES:BX  ... far pointer to your far pointer variable.
Input: far ptr [ES:BX] ... your far pointer variable.
Input:    AX ... amount to increment. 16 bits unsigned integer.
Output:          ES:BX  ... result far pointer before increment operation.
Output: far ptr [ES:BX] ... result far pointer variable storing after increment operation.

Useful for post-increment pointer operation.

2007.08.10

A short summary how the problems were repaired.

2007.09.30

2008.04.02

I often want to write meaningless topic.

Accidentally I started to have interests to port a version of Ultima Underworld which works with PC-9821 machine.

DM2 PS/V version uses Real mode of Intel CPU. UW uses Protected mode. It says programming style to absorb hardware dependency is completely different. But I felt I have to consider the Real mode style.

It is interested in the programming technique to absorb hardware's difference.

Usually the function table is prepared to define abstract layer to access hardware.

I researched the table example of DM2 PS/V version. Here is an interpretation of function table. IBMIO supplies these functions to FIRE.