RetroArch 1.7.8 – AI Service – How To Set It Up

Welcome to the future! Sometime ago, a #RetroArch bounty got posted proposing OCR (Optical Character Recognition) and Text To Speech services being added to RetroArch.

Some months later, and here we are – a bounty hunter valiantly took on the challenge and there is now a fully fledged AI Service up and running that works seamlessly with #RetroArch!

You use the AI Service like this – you enable the AI Service (should be enabled by default), you then setup the server URL (could be a local network address if you have the server up and running in your own network, or a public IP/URL in case you’re going through a service). After that, you only need to bind a button or key to the so-called “AI Service” action. You can bind this key by going to Settings – Input – Hotkeys.

In this video, you can see each of the two modes that the AI Service currently is capable of doing –

Speech Mode – Upon pressing the AI Service button, a quick scan is done of the text, and the recognized text is then translated to speech. You can press the AI Service button at any time and it will try to process the current snapshot of the screen it made. This mode is non-interruptable, meaning the game will continue running when you hit this button, and the output speech will take as long as it takes for the server to respond to your query and pipe the sound to RetroArch.

Image Mode – In image mode, it tries to replace the text onscreen with the output text. For instance, in the video you see above, the game is played in Japanese, so when we hit the AI Service button, it tries to replace the Japanese text with English translated text. This mode is interruptable – this means that when you hit the AI Service button, it pauses the game and shows you an image with the replacement text UNTIL you hit either the AI Service hotkey or the Pause hotkey again, then it will continue playing.

We encourage everybody that wants to submit feedback to us on this amazing revolutionary feature to go to our Discord channel and in specific the #retroarch-ai channel. We’d love to hear your feedback and we’d like to develop this feature further, so your input and feedback is not only appreciated but necessary!

VGTranslate using Google API keys

1.0 Get a Google API key

  • Create a billing account (taken from https://cloud.google.com/billing/docs/how-to/manage-billing-account )
    • Go to the Google Cloud Platform Console Manage billing accounts page and sign in or, if you don’t already have an account, sign up.
    • Click the Create account button.
    • Enter the name of the billing account, then click Continue. Note that if you see an Organization  drop-down, then you must also select an Organization before you can continue.
    • You might be prompted to select your country. The choice will affect the payment options you have in  the next step. The currency for your billing account is based on the country you select.
    • Choose the payments profile that will be associated with this billing account. You can choose an  existing payments profile, or create a new payments profile. If you choose to create, follow the  instructions on the screen to set up your payments profile.
    • Set your Account type as individual
  • Create an API Key: (taken from https://cloud.google.com/docs/authentication/api-keys )
    1. Navigate to the APIs & Services Credentials panel in GCP Console.
    2. Select Create credentials, then select API key from the dropdown menu.
    3. Click the Create button. The API key created dialog box displays your newly created key.
    4. This is your GOOGLE_API_KEY that we will use later.

2.0 Setup VGTranslate

  • Windows:
    • Download a release here: https://ztranslate.net/download/vgtranslate_serve_v1.01.zip?owner=
    • Change the vgtranslate/vgtranslate/default_config.json file as follows:
    • {     “local_server_ocr_key”: :<GOOGLE_API_KEY HERE>”,     “server_host”: “ztranslate.net”,     “default_target”: “En”,        “server_port”: 443,     “local_server_host”: “localhost”,     “local_server_port”: 4404,     “local_server_api_key_type”: “google”,     “local_server_ocr_processor”: {},     “user_api_key”: “”,     “local_server_translation_key”: “<GOOGLE_API_KEY HERE>”,     “local_server_enabled”: true
      }

    • rename default_config.json to config.json
    • Run serve.exe.
  • Linux:
    • Download the vgtranslate project from https://gitlab.com/spherebeaker/vgtranslate.git or use git clone.
    • Go to the extracted vgtranslate directory where setup.py is located.
    • Using python2.7, run: python setup.py install -If this step fails, you can try removing the problematic dependency.  Dependencies like gender-guess and kivy are not needed for most functionality. -Change the default_config.json file as in the windows case.
    • Go into the vgtranslate subdirectory and run: python serve.py

3.0 Setup RetroArch

AI Service Settings
  • Start RetroArch -Settings->Input->HotKey Binds->AI Service
  • Change to key/button of your choice
  • Settings->AI Service -Set AI Service Output to “Image Mode” (text translation) –
    OR set AI Service Output to “Speech mode” (text-to-speech)
  • Set AI Service URL to “http://localhost:4404”
  • Set AI Service Enabled to on.
  • Set Target Language to your language of choice
  • Source language will be used as a hint for the OCR, but can be left to “Don’t Care”
  • Run a game of your choice
    1. Image Mode (Text translation)
      1. Press your hotkey to pause the game, and wait for a translation to come back.
      2. Press your hotkey again to unpause and continue
    2. Speech Mode (Text To Speech)
      1. Press your hotkey to read out the text currently on the screen.
Bind the AI Service hotkey to a key or button
Bind the AI Service hotkey to a key or button

4.0 ZTranslate Service API Option

ZTranslate API Key method

  • Go to https://ztranslate.net/signup
  • Enter a display name, email, and password, and click register
  • Check your email for the verification link and click on it.  If you don’t see it, check your junk/spam folder.
  • In the top menu, click on “Settings”
  • Copy the API KEY near the bottom of the page.  This is your ZTRANSLATE_API_KEY.
  • Follow instructions in 3.0, but
    • For AI Service URL, put in: “http://ztranslate.net/service?api_key=<ZTRANSLATE_API_KEY HERE>”

5.0 – ZTranslate API via VGTranslate

Follow instructions 2.0, 3.0, and 4.0, except:

  • when modifying default_config.json, use the following:

    {     “local_server_ocr_key”: “”,     “server_host”: “ztranslate.net”,     “default_target”: “En”,        “server_port”: 443,     “local_server_host”: “localhost”,     “local_server_port”: 4404,     “local_server_api_key_type”: “ztranslate”,     “local_server_ocr_processor”: {},     “user_api_key”: “<ZTRANSLATE_API_KEY HERE>”,     “local_server_translation_key”: “”,     “local_server_enabled”: true }

  • Change AI Service URL to “http://localhost:4404”.
  • This will use the ztranslate API method, but will reduce latency compared to hitting the service API directly.

Core News – PUAE (Amiga emulator) and VICE (Commodore 64)

sonninnos and rsn8887 made a lot of improvements recently to two home computer Libretro cores.

PUAE Amiga core improvements

PUAE Amiga Core IK+
PUAE Amiga Core IK+
  • fix two player controls (sonninnos)
  • Implement new core options with some small print explanations (sonninnos)
  • Implement hotkeys for status bar and virtual keyboard that can be remapped instead of using hardcoded buttons (sonninnos)
  • Combining mouse speed indicator and other status bar indicators (sonninnos)
  • Allow starting the emulated Amiga without content (sonninnos)
  • Add many core options, such as a cycle exact option (sonninnos and rsn8887)
  • Allow to load any single, bootable .hdf hard drive file directly via load content (rsn8887)
  • Allow disabling the use of system/WHDLoad.hdf via a new core option (rsn8887)
  • Fix quickmenu->disk control options for swapping disks on the fly (rsn8887)
  • Change defaults to more sensible values (sonninnos)
  • Add Switch version of the core (rsn8887)
PUAE Amiga Core on Switch
PUAE Amiga Core on Switch

PUAE can play IK+ with sound FX on Nintendo Switch. Prince of Persia plays well too. These games don’t work on standalone UAE4all2. To enable sound fx in IK+, use model type A500 and turn off compatible mode in Quick Menu-> Options.

To swap disks, select Quick Menu -> Disk Control -> Append to insert a new disk in drive df0. When this option is selected, the virtual tray is opened, the disk is inserted, and the virtual tray is closed again. This all happens automatically. A new index number is also created. The disk becomes associated with that number. The cycle tray option ejects the current disk. When the virtual tray is open, one can change the index number in Quick Menu -> Disk controls to choose a different disk index. The menu remembers a history of all appended disks within one session, and assigns index numbers to each disk as you append them one by one. This is useful for games that keep asking for disk swaps back and forth. After manually opening the tray and selecting the next index to insert, choose cycle tray again to close the tray. M3u files loaded as content also populate the index values in disk control options.

New PUAE Options
New PUAE Options

The core automatically switches between Amiga 500 and Amiga 1200 hardware configurations in case the content filename includes (Amiga 500) or (Amiga 1200).

The touch mouse with single-finger tap-to-click and two-finger tap-to-right-click also works on Switch with this core.

Vice C64/C128/C16/Plus4 core improvements

Vice C64 Core on Switch
Vice C64 Core on Switch
  • update to latest Vice version 3.3 (Sonninos)
  • savestates (rsn8887 & sonninnos)
  • fixed Autosave states, it now works as long as some content is loaded (rsn8887)
  • allow holding directions to move more quickly across the virtual keyboard (rsn8887)
  • switched to new core options API, now there are small print descriptions below the core options (rsn8887)
  • more virtual keyboard themes, including some with more transparency (sonninnos)
  • reSID resampling setting, can be set to fast for extra performance, useful on Vita (sonninnos)
  • turbo fire with adjustable speed (sonninnos)
  • datasette control hotkeys (sonninnos)
  • allow using physical cursor keys even when cursor keys act as joy input (sonninnos)
  • allow physical return key to push a button on virtual keyboard (sonninnos)
  • virtual keyboard is much more responsive now, keys can be held down (rsn8887)
  • x64sc core (more accurate) on all platforms (sonninnos and rsn8887)
  • xpet core on all platforms (sonninnos and rsn8887)
  • xplus4 core on all platforms (sonninnos and rsn8887)
  • xvic core on all platforms (sonninnos and rsn8887)
  • x128 core on all platforms (sonninnos and rsn8887)
  • Automatic joyport switching when rom filename ends in _j1 or _j2 like on C64 mini. Only works with extracted roms (sonninnos)
  • Virtual keyboard themes (in tomorrow’s nightly)(sonninnos)
  • Add Display Borders option for larger game screens (rsn8887)
  • Add Autostart Warp option for warping between load command and run command (rsn8887)
  • Fix colodore palette, it didn’t work at all before (rsn8887)
  • Fix issues with on-screen keyboard keys sometimes not working or producing wrong input (sonninnos)
  • Properly implement Quick Menu-> Controls, now users can enable/disable joysticks there (rsn8887)
  • Remove confusing Retrojoy option, use quickmenu->controls to set input to joystick/keyboard/disabled now (sonninnos and rsn8887)
  • Properly use RetroArch system directory, instead of home dir or root folder (rsn8887)
  • Display four joystick inputs instead of two in statusbar (sonninnos)
  • Implement holding down Warp Mode hotkey, however it maxes out at screen refresh rate unless Vsync is turned off in Retroarch->Settings->Video (sonninnos & rsn8887)
  • Allow mapping buttons to blank (unmapped) in quickmenu->options (rsn8887)
  • Implement more logical hotkey mapping in quickmenu->options (sonninnos)
  • Fix problem with up arrow key producing Run/Stop input (sonninnos)
  • Revamp on-screen keyboard layout to fit on one page (sonninnos)
  • Make on-screen keyboard much easier to use: no need to press select anymore to enable dpad control, no need to move a mouse pointer (sonninnos)
  • Add Drive Sound Emulation and Drive Sound Volume options (sonninnos)
  • Aadd Reset option (sonninnos)
  • Remove obsolete Vice Menu GUI, because that can all be configured using core options now (sonninnos)
  • Automatically force drivetype on loading d71 or d81 content (sonninnos)
Vice Core Switch, Bruce Lee
Vice Core Switch, Bruce Lee
Vice C64 Save New York on Switch
Vice C64 Save New York on Switch

PSA: RetroArch 1.7.8 – Important changes for people using the Google Play build

Starting as of September, Google has started requiring app developers to update their apps with 64bit AArch64 support. Any app that fails to comply with this will no longer be able to upload any new APK to the Google Play Store.

Until now, the main RetroArch build has been built for 32bit ARM processors. This means that even on a 64bit device, if you were using this 32bit build, you would still have access to all the 32bit compiled cores.

This all changes now as a result. Under the new model, we are forced to build in both 32bit and 64bit support for the main RetroArch app. What this means is that 64bit devices will default now to the 64bit version, whereas 32bit devices will default to the 32bit version. It’s not possible to boot into 32bit mode for the Google Play Store version.

We unfortunately cannot do anything about this. What this means is the following:

  • Some of the cores available for 64bit ARM might lack a dynamic recompiler. As a result, cores like PCSX ReARMed and Picodrive might be significantly slower now as a result on a 64bit device. Fortunately, work is underway to backport a 64bit dynarec to PCSX ReARMed, and similar for Picodrive.

On the plus side, Aarch64 tends to have about a 10% performance boost in general for RetroArch vs. regular 32bit builds. So it’s not all bad news.

If you’d still like to use a 32bit only build, you can go to our buildbot and download the 32bit only version there. We just are not allowed to upload this version to the Play Store, since every app uploaded from now on has to work for 64bit as well.

Important news for people upgrading

If you are upgrading the app from 1.7.7 to 1.7.8, you might find that the core updater URL is still pointing to the 32bit cores, and you might find that a lot of the cores you downloaded will no longer work (because it boots you into 64bit mode now). 32bit cores cannot be ran on 64bit apps and vice versa. So, what you have to do is

  • First, make sure that you reset the Core Updater URL properly. Go to Settings -> Network, and scroll all the way down to ‘Updater’. Go to the entry ‘Buildbot Cores URL’, and long press. If done correctly ,it should reset the URL. Now exit the program and restart again.
  • From there, you will have to redownload the cores from the Online Updater.

Alternatively, if you don’t want to bother with all this, just reinstall the entire program from scratch, that way it will start out with sane defaults.

Other considerations

What this also means is:

  • The 64bit specific version on the Play Store is pretty much redundant now, and you might as well use the main Play Store version. We are unsure yet if we are going to delist the 64bit only version or have it remain up. Just know that it no longer serves a purpose.
  • RetroArch can now be built using Gradle/Android Studio. This should significantly reduce the barrier to entry for developers wanting to get to grips with RetroArch, but were unable to do so in the past as it required an old Android toolchain as well as tools like ant.
  • It might appear on Android TV soon for 64bit apps.

Hopefully this upgrade process will have as little bumps in the road as possible. We’re sorry for this inconvenience but this is really not our fault, and our hands are tied. As ever, if you want a 32bit version on your 64bit device, go to our buildbot, and download the latest dated version ending with ‘RetroArch_ra32.apk’.