{"id":47360,"date":"2019-08-08T00:57:28","date_gmt":"2019-08-08T00:57:28","guid":{"rendered":"https:\/\/www.libretro.com\/?p=47360"},"modified":"2020-05-27T02:59:23","modified_gmt":"2020-05-27T02:59:23","slug":"upcoming-retroarch-1-7-8-ai-service-machine-translation-ocr-text-to-speech","status":"publish","type":"post","link":"https:\/\/www.libretro.com\/index.php\/upcoming-retroarch-1-7-8-ai-service-machine-translation-ocr-text-to-speech\/","title":{"rendered":"Upcoming &#8211; RetroArch 1.7.8 &#8211; AI Service (Machine Translation \/ OCR \/ Text-To-Speech)"},"content":{"rendered":"<p><iframe loading=\"lazy\" width=\"750\" height=\"422\" src=\"https:\/\/www.youtube.com\/embed\/_EbfRIKA15Q?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen><\/iframe><\/p>\n<p><strong>Written by Barry Rowe<\/strong><\/p>\n<h2>What is the AI Service?<\/h2>\n<div>The AI Service lets you translate games, or add automated voice-overs capability in real time. \u00a0Instead of using a ROM patch, this is done by RetroArch taking a screenshot and then sending it to the AI Service listed in your config, which will do OCR (optical character recognition), machine translation, and\/or text-to-speech. \u00a0The service will then return the result and will either be displayed on the screen, or played through the audio output of RetroArch. \u00a0As is the nature with AI though, accuracy can be an issue. \u00a0However, machine translation can give a good gist of what&#8217;s being said, especially for some language pairs, and text-to-speech can be of great benefit for accessibility.<\/div>\n<h2>How to set it up<\/h2>\n<div>See\u00a0<a href=\"https:\/\/docs.libretro.com\/guides\/ai-service\/\" target=\"_blank\" rel=\"noopener\" data-saferedirecturl=\"https:\/\/www.google.com\/url?q=https:\/\/docs.libretro.com\/guides\/ai-service\/&amp;source=gmail&amp;ust=1565312098529000&amp;usg=AFQjCNEnpiCHRJh1PZ4Lw-8rz6nhzNdOIg\">https:\/\/docs.libretro.com\/<wbr \/>guides\/ai-service\/<\/a>\u00a0for a more detailed guide about how to setup the AI Service for your use case (translation or voice-over). \u00a0Any core using the RGB565 or RGB8888 pixel formats should work. \u00a0Cores using a hardware buffer however are not yet supported. \u00a0If you&#8217;re unsure if your core is supported, see the above documentation on how to check. \u00a0The main supported service is the vgtranslate project<\/div>\n<h2>Supported Services: VGTranslate, ZTranslate Service<\/h2>\n<div><\/div>\n<div>VGTranslate (\u00a0<a href=\"https:\/\/gitlab.com\/spherebeaker\/vgtranslate\" target=\"_blank\" rel=\"noopener\" data-saferedirecturl=\"https:\/\/www.google.com\/url?q=https:\/\/gitlab.com\/spherebeaker\/vgtranslate&amp;source=gmail&amp;ust=1565312098529000&amp;usg=AFQjCNE3LeX3kzLBgcvp_ZznzyIxgBnThQ\">https:\/\/gitlab.com\/<wbr \/>spherebeaker\/vgtranslate<\/a>\u00a0) is a lightweight server that you can run on windows or linux. \u00a0With it running, your RetroArch device can send its screen captures over your home network and the service will translate it and send it back to RetroArch. \u00a0This makes it possible to run the computationally expensive AI code off-device, so you can run translation on a Raspberry Pi or other device. \u00a0Local only translation is still in an experimental state, so using a Google Cloud key with VGTranslate is currently suggested.<\/div>\n<div><\/div>\n<div>It&#8217;s also possible to use an external service instead of running VGTranslate yourself. \u00a0ZTranslate.net has such a service listed in its documentation.<\/div>\n<h2>Caveats<\/h2>\n<div>Certain text will be easier to OCR than others. \u00a0Logos\/title screens or very stylized text can be difficult for OCR engines to deal with, and some language pairs are easier to translate between than others. \u00a0Japanese-to-English in particular seems to be the most difficult use case, since OCR engines can have difficulty with Japanese pixel fonts on textboxes with transparent backgrounds. \u00a0Many games will work fine, but some may not work that well, depending on how the game draws the text. \u00a0Future developments will help improve this and expand the number of playable games with this feature.<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Written by Barry Rowe What is the AI Service? The AI Service lets you translate games, or add automated voice-overs capability in real time. \u00a0Instead of using a ROM patch, this is done by RetroArch taking a screenshot and then sending it to the AI Service listed in your config, which will do OCR (optical [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[265,248,5,249],"tags":[],"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/www.libretro.com\/index.php\/wp-json\/wp\/v2\/posts\/47360"}],"collection":[{"href":"https:\/\/www.libretro.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.libretro.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.libretro.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.libretro.com\/index.php\/wp-json\/wp\/v2\/comments?post=47360"}],"version-history":[{"count":4,"href":"https:\/\/www.libretro.com\/index.php\/wp-json\/wp\/v2\/posts\/47360\/revisions"}],"predecessor-version":[{"id":47364,"href":"https:\/\/www.libretro.com\/index.php\/wp-json\/wp\/v2\/posts\/47360\/revisions\/47364"}],"wp:attachment":[{"href":"https:\/\/www.libretro.com\/index.php\/wp-json\/wp\/v2\/media?parent=47360"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.libretro.com\/index.php\/wp-json\/wp\/v2\/categories?post=47360"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.libretro.com\/index.php\/wp-json\/wp\/v2\/tags?post=47360"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}