New Core: PX68k (Android/iOS/Windows/Linux/Mac)

Disclaimer: This article was written by Tatsuya79, who has also contributed many improvements to the X-68K core. Developer r-type is the one who made the port

The Sharp X68000 was a home computer released exclusively in Japan in 1987. It was a powerful machine for its time and saw a great number of arcade ports, exclusive titles and doujin (indie) games developed for it, even years after the last model was launched in 1993.

Until now the only way to run Sharp X68000 games in RetroArch was with MAME. Its driver isn’t really the most advanced one and it is quite demanding, excluding many platforms such as smartphones.

Outside Retroarch, PX68k was aimed to be fast enough for that usage. Based on Winx68k, targeting the PSP and ported to iOS and Android by its Japanese developer Hissorii, it was possibly the only X68000 emulator on those platforms. As its development stopped some years ago, compatibility issues due to OS upgrades made its usage rather complicated.

Developer R-Type decided to port it to RetroArch, replacing its old 32 bits based CPU emulation by a 64 bits one from Yabause core. There is also a back end for the cyclone cpu on arm/android but surprisingly it didn’t give any speed enhancement and had more problems than the previously mentioned c68k.

After a common effort to fix various issues resulting from this change (thanks Retro-Wertz), it should now be at the same level of compatibility as the original emulator.

Running some tests on an old Samsung Galaxy S3, where we could barely emulate a 16MHz CPU before with PX68k stand-alone, we now achieve smooth results with a 66MHz setting. This makes it 4 to 5 times faster than before, and the libretro port is probably now the best performing Sharp X68000 emulator you can get for various cheap or old devices.

Testing on an [email protected] with “Akazukin Cha Cha Cha” achieved upwards of 1000fps on the default 10MHz emulated CPU. The same test gives 136fps in RetroArch using the Mame core.

The PX68k-libretro core still keeps the same main limitation of the original: no MIDI emulation. We also need to bring a virtual keyboard back, you can only use real ones at the moment. However, we did make some improvements:

1.) You don’t need to load a particular utility to define the amount of RAM the machine uses any more, there’s now a core option for that.

2.) You can change the CPU speed in real time.

If, like some old DOS games behaved, you encounter one that runs too fast (ex. Arkanoid), you can directly slow down your CPU from a fast 25MHz to the 10MHz clock speed it was programmed for.

We also added some overclock steps as high as 200MHz. High frequencies have the side effect of speeding up the floppy loading time, which is a much welcomed accident on this machine. (100MHz is already a lot faster for that.)

-We made some 8 buttons gamepad profiles which weren’t used that much on the system, but are great for the various Street Fighters II iterations.

You’ll need the bios files, which have been made publicly available by Sharp. Place them in your system/BIOS directory, in a subdirectory named “keropi”. The iplrom.dat and cgrom.dat are necessary, but you do not need the sram.dat. See the core information for a complete list.

L2 button or F12 key brings up the original px68k menu where you can change the inserted disks. They have to be unzipped to be accessible from this menu but can be zipped/archived when launching directly from RetroArch.

After the first boot a “config” file will be generated in the “keropi” folder. You can enter your rom folder into the “StartDir” line to make it accessible from the PX68k-libretro core’s in-game menu.

Shader Changes

Abstract

GLSL shaders now preferred over Cg when possible
Update to latest RetroArch for compatibility with updated GLSL shaders

Cg shaders demoted, GLSL promoted to first-class

Portability and compatibility are major goals for RetroArch and libretro, so we invested heavily in Nvidia’s Cg shader language, which worked natively anywhere their Cg Toolkit framework was available (that is, Windows, Linux and Mac OS X), as well as on PS3 and Vita, and could be machine-compiled to messy-but-usable GLSL (lacking a few features, such as runtime parameters) for platforms that lacked the framework (primarily ARM / mobile platforms). Cg was also so close to Microsoft’s HLSL shader language that many Cg shaders will compile successfully with HLSL compilers, such as those available with Windows’ D3D driver and on Xbox 360.

This was great for us because we could write shaders once and have them work pretty much everywhere.
Sadly, Nvidia deprecated the Cg language in 2012, which left us in a bad spot. Since then, we’ve been limping along with the same strategy as before, but with the uneasy understanding that Nvidia could stop supplying their Cg Toolkit framework at any time. Rather than sit idly by, waiting for that other shoe to drop, we took it upon ourselves to hand-convert the vast majority of our Cg shaders to native GLSL with all of the bells and whistles. TroggleMonkey’s monstrous masterpiece, CRT-Royale, still has a couple of bugs but is mostly working, along with its popular BVM-styled variant from user Kurozumi. Additionally, before this conversion, many of our Cg shaders were flaky or completely unusable on libretro-gl cores, such as Beetle-PSX-HW’s OpenGL renderer, but these native GLSL conversions should work reliably and consistently with any core/context except for those that require Vulkan (namely, ParaLLEl-N64’s and Beetle-PSX-HW’s Vulkan renderers).

With the GLSL shaders brought up to speed, we can finally join Nvidia in deprecating Cg, though it will still remain as an option–that is, we’re not *removing* support for Cg shaders or contexts at this point–and we will continue to use it where there is no other choice; namely, Windows’ D3D driver and the Xbox 360, PS3 and Vita ports. Moving forward, our focus for shaders will be on native GLSL and our slang/Vulkan formats, though we will likely still port some to Cg from time to time.

RetroArch now correctly handles #version directives in GLSL shaders; GLSL shader repo updated to match

There have been a number of updates to the GLSL shader language/spec over its long life, and shader authors can use #version directives (that is, a line at the top of the shader that says #version 130 or whatever) to tell compilers which flavor/version of GLSL is required for that shader. However, RetroArch has long had a strange behavior whereby it injected a couple of lines at the beginning of all GLSL shader files at compile time, and this broke any shader that attempted to use a #version directive, since those directives must be on the first line of the shader. This meant that our shaders couldn’t use #version directives at all, and all of our shaders lacked #version directives until very recently for this reason. These #version-less GLSL shaders are still perfectly compliant GLSL because GLSL v1.10 didn’t support directives, either, but the necessity of leaving off the #version started to cause some problems as we whipped our GLSL shader library into shape.

The error caused by adding a #version directive under the old behavior.

On AMD and Nvidia GPUs, the compilers would just toss up a warning about the missing directive and still expose whatever GLSL features were available to the GPU, which worked out great. On Intel IGPs, however, the compiler tosses the error and then reverts to only exposing the features available in ancient GLSL v1.10 (released way back in 2004). As a stopgap, we gave many shaders fallback codepaths that would still work in these circumstances, but a number of other shaders were either impossible to make compatible or even the compatible result was imperfect.

So, as of this commit (courtesy of aliaspider), RetroArch will no longer reject shaders with explicit #version directives, and we have added those directives to any shaders that require them at the lowest version that still compiles/functions properly. That is, if the shader doesn’t use any features that require greater than #version 110, they will still have no #version specified, and any shader that requires #version 120 but not #version 130 will not have its requirements increased to the higher version for no reason. This should keep our GLSL shaders as compatible as possible with older hardware, and including the #versions explicitly when needed will also make it easier for other programs/developers to utilize our shaders without any unnecessary guesswork due to behind-the-scenes magic.

This change does require a clean break, insofar as older versions of RetroArch will choke on the new #version directives (that is, they’ll fail to compile with the “#version must occur before any other program statement” error pictured above), so users with Nvidia or AMD GPUs must update their RetroArch installation if they want to use the updated shaders. Users with Intel IGPs will be no worse off if they don’t update, since those shaders were already broken for them, but they’ll probably *want* to update to gain access to the many fancy shaders that now work properly on their machines.

Mobile GPUs using GLES had many of the same issues that Intel IGPs had, with many shaders refusing to work without #version directives, but GLES compatibility added in a further complication: GLES requires its own separate #version directives, either #version 100 es or #version 300 es, which are different from and incompatible with desktop GL’s #versions. To get around this, we added a trick in RetroArch to change any #version of 120 or below to #version 100, which is roughly comparable in features to 120, and any #version 130 or above to #version 300 es whenever a GLES context is used. This should get everything working as effectively and consistently as possible on mobile GPUs, but if anything slipped through the cracks, be sure to file an issue report at the GLSL shader repo.

Bounty Updates

We have the bounty system up and rolling with Bountysource and have begun adding awards to a series of issues. The libretro organization has seeded $80 to issues so far, and another $100 has been offered by users. Current issues include adding low-latency input and audio drivers for Windows; extending the mouse axis for games like tyrquake that can be controlled by mouse; support for multiple mice simultaneously, which could enable 2-player light-gun games in several cores; and adding a host virtual filesystem layer, which could enable softpatching on all cores among other benefits. The VFS layer has had the most activity so far and currently sits at $70.

Introducing the Bounty System

One of our goals with getting on Patreon was to experiment with using a bounty system to encourage contributions from outside of the normal libretro/RetroArch/Lakka team, and we’re finally ready to take a stab at it. This is uncharted territory for us, so some of this framework is bound to change as we move forward, but here’s our initial plan:

  1.  The libretro team makes all final decisions on bounty allocations and disbursements. While we intend to listen closely to community input, ultimately we have to be able to make the final decisions.
  2. All contributions must follow coding guidelines and meet approval of the libretro team before disbursements will be awarded. We can’t pay out if the code isn’t usable and/or maintainable by us.
  3. Pursuant to #2, potential contributors should contact the libretro team prior to beginning work to make sure the final product will be acceptable. This is intended to avoid misunderstandings and other conflicts. We don’t want someone to work hard on a fix or feature only to find that it’s not going to be acceptable for whatever reason.
  4. We will try to do as much as we can through Bountysource, where we can link specific issues from our Github repos to bounty values. This is especially applicable to smaller tasks. However, it may not be appropriate for all tasks, and we’ll decide how to deal with those that don’t exactly fit on a task-by-task basis.
  5. Pursuant to #4, potential contributors should contact the libretro team and determine an actual disbursement value based on the magnitude and difficulty of the task. We may need to negotiate up or down to find a fair value, based on the contributors’ skillset, or the amount of tutoring needed to get contributors up to speed with the codebases/APIs involved, etc.
  6. Disbursements can be made in the form of cash payments, the purchase of hardware for development and/or testing, etc. We want to be able to help developers with whatever they need. Sometimes that will be in direct payments, other times it may be in specialized hardware for porting/maintaining or reverse-engineering or whatever.
Again, this framework is a work-in-progress, so if you have any questions or concerns, feel free to contact us, either on Github or in #retroarch on Freenode IRC.

RetroArch 1.4.1 Major Changes Detailed!

It’s been such a long time since we last released a stable (1.3.6), so let’s give some indication of just how much work we’ve poured into the entire ecosystem surrounding RetroArch and libretro since!

Scanning

RobLoach added scanning GoodNES and GoodN64 sets (i.e., instead of just No-Intro sets) and support for playlists using the ScummVM, NXEngine and Lutro cores. He also added a database of SNES translations, an oft-requested feature, and Super Mario World romhacks, so many of those hacks should show up in scans now.

bparker did a significant overhaul of the content-scanning backend to allow for recursive scanning–that is, the ability to scan inside subdirectories without going into each individual subdir manually. This was one of the most frequently requested scanning improvements. bparker also modified the scanning function to peek at the file extensions supported by the scan before comparing against a database, which speeds up the process substantially. In brief and nonscientific testing, scanning a full NES No-Intro set of 2,703 ROMs went from 4 minutes and 22 seconds down to just 53 seconds, representing a 491% speedup, while scanning an SNES No-Intro set of 3,442 ROMs went from 9 minutes and 37 seconds to 1 minute and 35 seconds, for a whopping 607% speedup.

7zip scanning

bparker also upgraded the 7zip support to a full first-class citizen (that is, right up there with standard zips), which means you can now scan/load/whatever content that has been compressed into a 7zip archive.

Configurations

Radius greatly improved and expanded the config override functionality, adding the ability to save overrides directly from the RetroArch menu, rather than having to create them manually with a text editor. He also added the ability to save per-core and per-game shader presets, which is the main thing users wanted to change from core to core (along with retropad mapping, which is already handled via the per-core/per-game remapping function). With all of this added functionality, we decided to completely remove the conflicting and often broken and unpredictable “per-core configs” option, which had already been deprecated but kept around for legacy/transition support.

Netplay

Meanwhile, GregorR has done the impressive and unenviable task of dusting off and overhauling RetroArch’s lag-hiding, peer-to-peer netplay implementation, which had been some of the least-touched, least-understood code in the entire codebase. He has already made great strides on the stability of connections, with a big reduction in (if not outright elimination of) out-of-the-blue desyncs, along with graceful recovery of synchronization following temporary losses of connectivity. Switching from UDP to TCP communication has made it so that only the host needs ports forwarded, which should help with playing games with less-technical friends, and the ability to search for hosts on the same LAN makes it easy to do Japanese arcade-style head-to-head matchups. GregorR also added support for 3+ player netplay, so you can throw down on party classics like Super Bomberman 5 with four of your closest friends. Pursuant to our Patreon goals, we’ll be starting on a netplay matchmaking server solution as one of our top priorities to take advantage of these exciting improvements.

QoL menu improvements for Lakka/RetroArch

Kivutar et al have recently pushed out a pair of major Lakka releases, which include the built-in RetroArch improvements, along with a number of Lakka-specific features/improvements. Users now have the ability to hide advanced settings in the XMB menu, which leads to a greatly simplified default menu in Lakka releases, as well as the ability to hide menu tabs such that only playlists are visible, which is ideal for appliance-style “kiosk” settings, where you don’t want children or other users to monkey around with your settings (these options are also available in non-Lakka RetroArch releases). Kivutar also added the ability to scan for and join wireless networks directly from the Lakka interface, as well as individual history tabs for RetroArch’s built-in multimedia cores, which include a video player, image viewer and audio visualizer (these cores have been built-in for a while but many people didn’t know about them; the history tabs should make them more accessible). You can read more about the Lakka releases here and here.

Fancy new menu graphics features

The XMB menu received some fancy new background shader effects, including an improved ribbon, a nice bokeh effect and a festive and nostalgic snow effect. We also rearranged and consolidated some menus to reduce clutter and hopefully make things easier and more intuitive for new users, and we merged ‘load content’ and ‘load content and detect core’ into a single unified function. We’ve also added secondary, explanatory sub-text to many menu items that should make them less mysterious, and a battery meter that should let mobile users avoid running out of juice unexpectedly.

Ports

Sony – PlayStation3/Vita

The Sony console ports have gotten some much-needed love, with frangarcj doing some really stellar work with the Vita henkaku port, which supports the fancy XMB menu, shaders, dynamic core loading, (mostly) full-speed PSX emulation via dynarecced PCSX-ReARMed and more, while Ezi0 has made great progress in getting the PS3 port back to usability.

Nintendo – Wii/WiiU/3DS

On Nintendo consoles, netux79 has been keeping up with the Wii port, adding support for USB gamepads and fixing some savestate issues with the snes9x-next core, while Twinaphex tracked down and fixed a black-screen bug that’s plagued the nightly builds since just after the 1.3.6 release. Aliaspider and Twinaphex have made a lot of improvements to the 3DS port, including fixing a persistent screen-tearing issue, and aliaspider has also made great progress with the nascent Wii U port.

Cores

On the core front, Twinaphex has greatly improved the error handling on many cores so that they are less prone to bringing the entire program down (i.e., segfault) when they choke. While this is certainly not as sexy or user-facing as many of the other improvements, it will lead to a more stable, user-friendly experience with fewer mysterious crashes that give no information as to their cause.

Twinaphex also ported Ryphecha’s unbelievably-awesome new Sega Saturn emulator to libretro (listed as “mednafen-saturn” in the online updater). Saturn has long been considered one of the most difficult and esoteric consoles to emulate, and Ryphecha deserves a virtual high-five for doing such a great job with it and generously releasing it under the GPL license. While it is currently only available on x86_64 platforms (i.e., no ARM, no 32-bit x86), its emulation quality and accuracy is already top-notch.

N64 emulation has had some exciting development, as well, with loganmc10‘s Glupen64-libretro port, which combines a shallow fork of mainline Mupen64plus (i.e., not the heavily modified fork that mupen64plus-libretro is based on) with gonetz’s crowdfunded GLideN64 RDP plugin. This core handles many hard-to-emulate effects as compared with the other HLE plugins and gets good performance on even modest hardware. As such, it has become the default core for N64 in the RPi3 spin of Lakka and also performs quite well in Android (and more stable there than regular mupen64plus-libretro, according to user reports). Meanwhile, Seru-kun added support for 64DD disks to Mupen64Plus-libretro, making it the second N64 emulator (after PJ64) to support the Japan-exclusive add-on.

Leiradel and meleu did a massive cleanup on the Retro Achievements/cheevos system, whereby all systems that RetroArch supports have at least one core that includes achievement support, and all systems supported by Retro Achievements is represented. Moreover, spurious achievements should no longer be awarded right when the game starts (a common problem, previously) and achievements will no longer be awarded in the event of a failure to actually meet the requirement(s).

RetroArch v0.9.9 Released – where to get it on each platform

RetroArch v0.9.9 has officially been rolled out on all platform targets.

The new platforms that are supported with this release of RetroArch are as follows:

  • iOS (both jailbroken and non-jailbroken – non-jailbroken requires that you are a registered developer and can compile your own copy of RetroArch + cores)
  • Blackberry 10
  • Blackberry Playbook Tablet OS

The other platforms which are already supported by the RetroArch/libretro projects have all received updates (with some pretty extensive changes – more on that in an upcoming blog post).

WHERE TO GET IT

Windows: New users can download 32- and 64-bit flavors of RetroArch and RetroArch-Phoenix from Themaister’s site:

http://themaister.net/retroarch.html

Existing users can/should download the new version through RetroArch-Phoenix’s built-in ‘RetroArch Updater’ utility. (this is the preferred update method for existing users to save massive bandwidth!)

Mac OS X users can download hunterk’s builds from this post on the libretro forum:

http://forum.themaister.net/viewtopic.php?pid=459#p459

Debian/Ubuntu/Mint users can add hunterk’s Launchpad PPA repository to their Synaptic/apt sources:

https://launchpad.net/~hunter-kaller/+archive/ppa

iOS users can find RetroArch iOS in one of Cydia’s default repositories – ZodTTD & MacCiti.

You can also add our own Cydia repository in order to get it, located at:

http://themaister.net/cydia

Most cores will work with both tethered and untethered jailbreaks, but cores that require the use of a dynamic recompiler (dynarec; DeSmuME and PCSX-ReARMed) will require a full, untethered jailbreak to function.

Android users can get the latest version from the Google Play Store. Xperia play controls seem to be wonky, but we hope to have that fixed very soon.

Wii users should use this package:

https://anonfiles.com/file/4536ac12f0071a397b2f1d70672814cf

Blackberry Playbook users should use this package:

http://themaister.net/retroarch-dl/blackberry/playbook/RetroArch-1_0_0_1.bar

Blackberry 10 users should use this package:

http://themaister.net/retroarch-dl/blackberry/bb10/RetroArch-Cascades-1_0_0_1.bar

PS3 users can get the DEX and CEX versions from the usual sources.

Xbox1 and Xbox360 users can get their respective versions from the usual sources.

OpenPandora users can get builds from lifning’s repo:

http://repo.openpandora.org/?page=detail&app=retroarch.lifning.001

RetroArch v0.9.9 Coming Soon

It’s almost time for a new release of RetroArch, and there a number of big changes coming up. First of all, RetroArch 0.9.9 will mark the release of RetroArch on iOS and Blackberry 10/Playbook Tablet OS. These ports were made possible with the help of CatalystG and meancoot, respectively – for which many thanks. The iOS port will be released on Cydia and on our forum. It is possible to run it on a non-jailbroken device – but it will require that you are able to code-sign yourself (ie. if you are a registered Apple developer with the ability to code sign).

PCSX ReARMed on iOS

For iOS, perhaps the single biggest hurdle was getting PCSX-ReARMed working, which required notaz to rewrite much of the assembly code to work with Apple’s ancient GAS assembler version (big thanks to him for that!). With that completed, this should be the first time PCSX ReARMed will appear on iOS – through RetroArch.

RGUI

Elsewhere, Themaister and Squarepusher have been toiling away at a million other features, including the promotion of RGUI to a robust and feature-filled in-game menu system for the platforms that otherwise lacked such a thing, particularly the PC platform (i.e., Windows, Mac OS X and Linux). From its humble beginnings with the Gamecube/Wii port, RGUI now provides a way to change emulation cores, swap out ROMs, configure shaders and more, all without leaving the fullscreen gaming interface:

rgui

Cave Story (NXEngine)

ToadKing and Squarepusher have also done some work on ‘uncrippling’ Cave Story (ie. NXEngine). Previously, the file I/O would make it unbearably slow on consoles. This has mostly been fixed now that everything is pre-cached into RAM at initial startup. There are still some incidental dips to 59.50fps and 59.2fps, though, which causes some sound stuttering. The cause of these dips is still unknown but we feel that–compared to before–NXEngine can be safely released on consoles now without being an utter embarrassment. “Xbox 1/360 will require some further patching up of the codebase because NX Engine did some global symbol table trickery and the MSVC linkers have the (oh so ‘smart’) tendency to ‘strip away’ unreferenced symbols as an ‘optimization feature’ with no way to stop it from doing that (even /ref:noopt doesn’t help there),” Squarepusher noted.

TyrQuake

The port of TyrQuake will also be bundled with RetroArch 0.9.9. A lot of work went into making it work on Xbox 1 and Xbox 360 – including making the C99 codebase cross-compilable as C++98 and (for Xbox 1) resorting to a hacked-up template ‘typeof’ implementation (ye, don’t ask) for MSVC 2003. “I also threw in some additional ‘hackish’ features like ‘dither filtering’ (borrowed it from some guy that implemented it earlier) – this more or less looks like the Unreal 1 software renderer’s ‘bilinear filtering’ implementation,” Squarepusher said. “There is also a third-person chase cam view and a way to ‘lerp’ the animations (ie. add key-frame interpolation in order to make the animation of enemy models look smoother and have more frames of animation than they originally did).”

“I plan to eventually rebase the TyrQuake port and push it upstream to the original authors (ie. the maintainers of TyrQuake) – I did a lot of careless code rewriting that I’ll be sure to avoid for the rebase,” he added.

Shaders

There has been a major overhaul of the way shaders are handled, which has paved the way for advanced, multipass shaders that can be easily setup by end users without needing to tinker with any code. As part of these changes, the old XML/GLSL shaders with fixed-pipeline functions have been deprecated, but will still work just fine. In the future, we ask that interested shader authors try to stick to the multiplatform Cg format when possible. The GLSL/GLES formats will still be supported for compatibility with platforms that don’t support Nvidia’s Cg Toolkit, such as Android and iOS, and Cg shaders can be converted to these legacy formats programmatically using Themaister’s cg2glsl python script.

A couple of examples of newly written shaders that utilize some of the recently added features are Themaister’s NTSC Composite shader, which should work well on any libretro core, and Harlequin’s Gameboy shader:

There is no firm release date for v0.9.9, but if you would like to try any of these features out or get involved in the development, you can grab the code from git and compile it yourself for your platform of choice. If you have any questions about these features or RetroArch/libretro in general, stop by the libretro forums or drop by #retroarch on Freenode IRC.

Discuss this post