Upcoming – RetroArch 1.7.8 – AI Service (Machine Translation / OCR / Text-To-Speech)

Libretro Team

Written by Barry Rowe

What is the AI Service?

The AI Service lets you translate games, or add automated voice-overs capability in real time.  Instead of using a ROM patch, this is done by RetroArch taking a screenshot and then sending it to the AI Service listed in your config, which will do OCR (optical character recognition), machine translation, and/or text-to-speech.  The service will then return the result and will either be displayed on the screen, or played through the audio output of RetroArch.  As is the nature with AI though, accuracy can be an issue.  However, machine translation can give a good gist of what’s being said, especially for some language pairs, and text-to-speech can be of great benefit for accessibility.

How to set it up

See https://docs.libretro.com/guides/ai-service/ for a more detailed guide about how to setup the AI Service for your use case (translation or voice-over).  Any core using the RGB565 or RGB8888 pixel formats should work.  Cores using a hardware buffer however are not yet supported.  If you’re unsure if your core is supported, see the above documentation on how to check.  The main supported service is the vgtranslate project

Supported Services: VGTranslate, ZTranslate Service

VGTranslate ( https://gitlab.com/spherebeaker/vgtranslate ) is a lightweight server that you can run on windows or linux.  With it running, your RetroArch device can send its screen captures over your home network and the service will translate it and send it back to RetroArch.  This makes it possible to run the computationally expensive AI code off-device, so you can run translation on a Raspberry Pi or other device.  Local only translation is still in an experimental state, so using a Google Cloud key with VGTranslate is currently suggested.
It’s also possible to use an external service instead of running VGTranslate yourself.  ZTranslate.net has such a service listed in its documentation.

Caveats

Certain text will be easier to OCR than others.  Logos/title screens or very stylized text can be difficult for OCR engines to deal with, and some language pairs are easier to translate between than others.  Japanese-to-English in particular seems to be the most difficult use case, since OCR engines can have difficulty with Japanese pixel fonts on textboxes with transparent backgrounds.  Many games will work fine, but some may not work that well, depending on how the game draws the text.  Future developments will help improve this and expand the number of playable games with this feature.

RetroArch on Steam now coming sometime soon Q3 2019!

Libretro Team
In order to do the Steam release justice, we have some infrastructure work remaining to do that will take more time. Our new release estimate therefore is sometime later this quarter. We will keep you updated on our progress.

The net positive of this is that we can ensure the release is more polished and that it will meet the user expectations of your average Steam user.

RetroArch 1.7.8 however is going to be imminent within the next two weeks.

RetroArch Disc Project – Alpha testing has begun! Details inside

Libretro Team

Ever since the announcement earlier on June 30th, we have been working hard on the RetroArch Disc Project, and we are now getting to the stage where we want users to start alpha testing the latest code.

We invite you to come over to our Discord channel (also linked to on the RetroArch website – click on the tab ‘Discord’)- and join the channel #discordproject.

How to apply for alpha testing

For the purpose of this alpha test, we need you to meet the following requirements:

  • You are using the latest nightly versions of RetroArch (available here).
  • You are using RetroArch on PC, and are running either Linux or Windows with the CDROM support included.
  • Make sure the cores you are going to be testing are the latest up-to-date versions. Update them daily from the Core Updater if unsure – the cores should have physical CDROM support, otherwise the testing won’t work.
  • You have an optical disc drive (either USB or internal, any CD-ROM/DVD-ROM/BD-ROM drive should do).

Report issues you’re having with physical CDROM playback. Be as detailed as possible in your feedback – we are especially interested in which optical disc drive you’re using.

Be prepared that you might be asked by a moderator to run ‘retroarch_debug.exe’ and return a log message back to us in order to learn more about the issues you’re having. ‘retroarch_debug.exe’ is a debug executable that should be shipped with every RetroArch nightly version for Windows. For Linux users, we assume that they can build from source and compile with CDROM_DEBUG=1 manually in order to run these more detailed debug tests.

Compatible cores

Listed below are some of the cores that we added physical CD-ROM support to, and the ones that we would like you to test –

  • Beetle PSX (HW and non-HW)
  • Beetle Saturn
  • Genesis Plus GX
  • 4DO
  • Redbook Audio Player (Audio CD core)

We have also added support to more cores, but for now we want to go with these cores as they have been most extensively tested.

A couple of known edge cases:

  • Beetle PSX HW: The ‘vulkan’ renderer can exhibit issues where either the CD will not work, or some other corruption issue will manifest itself. We recommend you use ‘opengl’ or ‘software’ for now while you test CDROM.
  • Beetle PSX HW: Make sure the core option ‘CD Access Method’ is not set to ‘precache’. If you do, real discs won’t work.

How to load discs

1.You need to first start up the core itself (there is no automatic system detection yet when you insert a disc).
Go to Load Core, and select the core you want to use.

2. Once the core is loaded, select ‘Load Disc’.
3. It will now list all optical drives. Make sure the disc you want to use has been inserted into the optical drive tray and make sure the tray is closed. Then select the drive. The core will now start with your CDROM as input.

How to dump discs

RetroArch can dump discs to internal storage in bin/cue format. The video above will show you how.

  1. Select ‘Dump Disc’.
  2. It will now list all optical drives. Make sure the disc you want to use has been inserted into the optical drive tray and make sure the tray is closed. Then select the drive.

The disc will now be dumped to the ‘Downloads’ directory. It will continue showing the progress bar until the operation has finished. In the meantime, you can play any core/game (as long as it doesn’t involve real CD playback) until the dumping has finished.

Redbook Audio Player – Libretro core for music CD playback!

As a testbed for the Disc Project, we made a redbook audio CD player Libretro core!

You can grab this core for Windows and/or Linux by going to Online Updater, and downloading it from the list (Redbook Audio Player).

Known potential issues

  • Right now, CDROM works better on Linux vs. Windows.
  • FMV playback with Beetle PSX HW has the potential to be slow on some optical disc drives on Windows. Cause yet unconfirmed.
  • There is a known issue with redbook audio – the redbook audio track begins a few seconds later from its starting period.
  • Libcrypt copy-protected PlayStation1 games will not work right now, but any other copy-protected PlayStation1 disc should.
    Here is a listing of some Libcrypt-protected games from Europe: https://hastebin.com/ruyacahuba.http . You can generally assume these won’t work.

RetroArch coming to Steam this month – July 30

Libretro Team

RetroArch will be releasing on Steam on July 30, 2019!

https://store.steampowered.com/app/1118310/RetroArch/

Read our release announcement here –

So RetroArch is finally coming to Steam two weeks from now!

RetroArch is the official reference frontend of the popular Libretro API. While there is nothing particularly about RetroArch or the Libretro API that has anything to do with emulators, most do it use it for this purpose.

We want to grow our base and as of this month we have been putting a heavy focus into making sure that RetroArch can run originally bought content on game discs. There are further announcements to follow in the coming weeks, and we are open to dialogue with game developers/publishers that have the rights to original IP who want to bring their games over to Steam through the use of RetroArch.

Do note that RetroArch is an agnostic frontend/backend application that can be used to run modular programs. As such, it can run real games, media engines, and emulators, as long as these programs are converted/ported into modular pieces of software known as ‘Libretro cores’.

So, without further ado, here’s the details on our initial release:

  • Free.
  • Windows version first, Linux and macOS versions will release later. We’re a bit wary of the support burden that will come with a much wider audience, so we want to do the Windows version first to make sure we can handle the demand.
  • At the beginning, there will not be any difference between the version you can get on Steam and the one available on our own website. As such, no Steamworks SDK functionality will be present at launch or any additional Steam features. However, after the initial launch, we will begin exploring options on how we can start leveraging Steam’s functionality as a platform.
  • Release date will be approximately 30th of July.

RetroArch as a program serves as the fundamental backbone of many retro gaming machines out there in the wild. Even when it is not deliberately namechecked, chances are it’s running in the backend of said retro program. Same goes for its sister project Libretro. For instance, several programs available on Steam already, such as Grid Cartographer, and New Retro Arcade, integrate Libretro support to allow you to run the software that has been ported to the Libretro API. As for Libretro, this API is being leveraged by several retro gaming companies right now to serve as the backbone of their retro emulation services. Given all this interest and burgeoning activity, we feel now is the time to finally bring the official Libretro frontend to Steam. RetroArch is truly in a class of its own on a technical level when it comes to latency, shader features, and now soon-to-be disc loading and disc dumping. We can’t wait to bring you complete control over your retro gaming library coming this summer on Steam!