Adding HLE
The 3DO was ahead of its time in a number of ways. One was that it had an operating system. It has preemptive multitasking, threading, high level memory management, dynamic loading of “folios” (libraries), IPC, IO APIs, and general abstractions to much of the hardware. The OS and most/all software for the platform is written in C and compiled using the Norcroft ARM toolchain from that era. The OS is present in the system’s ROM but also included on each CDROM. The ROM OS is pretty thorough but ultimately a bootloader and bootstraps the environment before handing control off to the software on the CDROM. Many of the core OS functions are called via the ARM SWI (software interrupt) instruction. This makes catching and overriding these calls very easy when combined with knowledge of the ARM procedure call standard.
Catching and overriding some or all of the OS functions would theoretically improve performance and/or enable advanced features. Some of these functions are somewhat complicated but one set of SWI calls were a perfect candidate for HLE: the matrix arithmetic functions. The MADAM graphics co-processor has a set of hardware accelerated 16.16 fixed point matrix arithmetic functions. This is used in a number of 3D games and called regularly. While reworking MADAM emulation in the past it was recognized that the OS could and would check the revision of the MADAM chip and had a fallback to software based matrix functions if it recognized it was running on a prototype system without the matrix hardware and that for some games there was a slight speed increase when switching between the hardware emulation and the software routines. So we know that improving the matrix arithmetic situation could help lower end systems struggling with the LLE and it should also be simple to ensure accuracy.
So we’ve added a new, optional, HLE mode that replaces the emulated hardware or software matrix functions with native fixed point calculations performed in straight C. Games that make heavy use of the matrix hardware run a few FPS faster on lower end systems such as the XU4. The option can be toggled in real time making comparison easy. You can see what games use these functions on the 3DO Development Repo site.
In the future a number of other functions would be interesting to replace such as the DSP and filesystem APIs. The DSP never got a development kit so most (all?) games use standard programs provided by The 3DO Company. Meaning that each of these programs could be statically re-implemented in C instead of using the current interpreter. The DSP is probably 1/3 of the CPU cost to emulation so HLE could make a big impact there. With the filesystem API the host storage device could be used directly allowing easier sharing and backup of save games and offering more space than a real system could have.
VDLP Rewrite
The 3DO has a fairly advanced video display line processor. It has programmable functions which provide a number of features including: managing CLUTs (color lookup tables), performing optional horizontal and vertical interpolation (upscaling from the internal 320×240 framebuffer to 640×480), managing background color, integrating secondary video data streams, etc.
The FreeDO/4DO VDLP emulation was not the best. If you read the patents the system seemed reasonably straightforward but the code not so much.
- Has a lot of literal values for masks and shifts making it unclear what is being processed.
- Variable names did not always match the docs.
- Inadvertently reversed logic that rendered the wrong lines.
- A constant bodge value used to offset the rendering line. Didn’t have a functional impact but was confusing and ugly.
- It fails to properly manage the disabling render of a line. This can result in garbage being rendered.
- Lacks the interpolation to full NTSC or PAL frames.
- A general lack of PAL support.
- It failed to properly sync line rendering with libretro’s main callback leading to additional rendering latency.
- Has an unnecessary step in the rendering pipeline. Rather than taking the internal VRAM framebuffer and directly generating the buffer for the host to display it was copying the data into an intermediate structure which later was converted into a host compatible format. This copying offered no additional functionality and could be removed entirely.
- Alternative color formats were ignored completely. Only P555 was supported. Though it is unclear if any software use the other formats.
A new VDLP core has been written from scratch to fix some of the above and enable future enhancements. New features include:
- Lower frame rendering latency
- Properly synced frame generation with libretro run loop
- Improved performance due to removal of extra rendering stage
- Ability to choose any of the libretro pixel formats (XRGB8888, 0RGB1555, or RGB565). Formerly only supported XRGB8888. May improve performance on 16bpp platforms.
- Forced CLUT bypass mode. The VDLP has a special, per line and per pixel ability to disable color lookups and convert the P555 RGB to 888 RGB by a logical shift left of 3. Many games technically use a CLUT but in fact use a default palette that matches the left shift values. While this may cause color distortions on some games it also should improve performance due to a simpler pixel conversion function.
Interpolation is still on the TODO list. It will add non-insignificant cost to the rendering pipeline but was a significant feature of the platform and would be nice to have for the authenticity.
PAL ANVIL Support
Later revisions of the 3DO had a chip called ANVIL which was the combination of the co-processors MADAM and CLIO. The ROM for those systems are very similar to the previous model’s ROMs except that it appeared to be mirrored to a different memory location. It put a single unconditional jump at bootup into that mirrored location but all other references were to the original memory range. This memory range was not properly supported nor was the VDLP and general rendering pipeline designed to work with PAL. All of that has been fixed. You can use EU ROMs which will cause EU games to run in one of two PAL resolutions. For the time being the ROM and resolution/field rate need to be selected manually.
A list of games and their supported region modes can be found here. Few games actually render at full PAL resolutions (PAL2). Most are letterboxed NTSC (PAL1).
Future Work
- New HLE functions as described above
- VDLP interpolation (dev docs, patent)
- Increased emulated system DRAM and VRAM. The OS can support up to 14MB of DRAM & 2MB of VRAM or 15MB DRAM & 1MB VRAM. The emulator needs to be rewritten to enable it. This could help with homebrew.
- Rewrite of the ARM and DSP cores
- Rewrite of the CEL renderer
- Removal of the “hacks” / improve accuracy
- Improve accuracy by adding MADAM memory fence support and proper CPU exceptions such as those related to data alignment
- Automatic detection of NTSC/PAL modes
- Configurable hardware revisions of MADAM and CLIO chips
- Add features to help with homebrew development and debugging
- Other misc things
Renaming of the Project
Development on the original 4DO projected stopped a number of years ago and while the website is still up the project is effectively dead. It’s been noticed that people regularly believe that the old 4DO project and the libretro core are largely identical however the libretro core has gone under significant restructuring, rewrites, and offers new features. To limit confusion we are planning on renaming the project to Opera in the not too distant future. Opera was the code name for the 3DO project.
Related Work
Optimus (author of OptiDoom, the reworked 3DO Doom port) and I have created a new website dedicated to all things 3DO development. We’re writing homebrew tutorials, documenting the hardware, posting development toolkits and homebrew tools, etc. The hope is to spur interest in the platform and lower the barrier to entry.
Right now there is a very small group of individuals working in the 3DO and M2 space. Whether that be documentation, emulation, or homebrew. We’d like to extend an invitation to any and all interested in the 3DO or M2 to help in any capacity or consider the platforms for their current or future homebrew projects. We’ve got a big TODO list that we would love help in tackling. If you’re just interested in following our progress you can create an account on the wiki and subscribe to updates and/or “Watch” the libretro project.