Obituary – Near

Hi there community,

last Sunday a tragedy befell the emulation community, when Near tragically took their own life. While we feel Near needs no introduction, we feel it is only right to let people know of the extent to which Near’s work laid the foundational framework of Libretro/RetroArch, and what other great projects they worked on, including of course bsnes (which needs no introduction at this point).

Among Near’s other great accomplishments: libsnes (which later turned into our fork, libretro), libco (a cooperative multi-threading library), Higan (a multi-system emulator), Ares, and various other auxiliary projects.

Out of respect for Near’s untimely passing, we have delayed the release of the next RetroArch by a full week. You won’t be seeing us doing any promotional material or coverage for it until then, and who knows, we might even skip going into it at all. We feel this right now is more important and should get front and center coverage.

We asked three people familiar with Near to provide their own eulogy. Recognize that we speak from the heart and that our purpose in doing this is to pay proper respect and tribute to a great programmer in the emulation scene, the likes of whom we might never get again.

Hunter Kaller

About 15 years ago, I was just getting into open-source software and the Super Nintendo was always my favorite console, so I was excited to find a relatively new open-source emulator that ran even the weird, unpopular games. The author of the emu had a forum where they posted their releases and all the cool stuff they were working on, and that forum was home to a tight community of other smart, creative folks.

That forum was the old bboard, and that emu author was, of course, Near. As we all know, Near was extremely prolific, driven in their pursuit of perfection, and generous enough to share their many accomplishments with the world at large. In addition to their work on bsnes, Near also documented their relentless reverse engineering escapades on the bboard, along with their efforts to understand the entirety of as many facets of software development as they could, top to bottom. The bboard was home to some pretty epic (in the classical sense) threads in which Near would dive head-first into topics–like the fundamentals of signal resampling–that most of us outside of graduate-level computer engineering programs consider black magic.

It is no exaggeration when I say: without Near and the community they cultivated on the bboard, there would be no libretro and no RetroArch.

Libsnes (which would serve as the basis for libretro) was Near’s design to decouple their backend code from the endless frustrations of frontend coding. The bboard is where Themaister first developed and released SSNES, which would become RetroArch. They met Twinaphex while attempting to port bsnes to the PS3 via libsnes+SSNES on Near’s behalf. To this day, many of our cores depend on Near’s libco cooperative threading library. I could go on.

With that said, we did not always agree or even get along. Near’s perfectionism frequently put them at loggerheads with individuals who felt some thing or other was already “good enough” (that is basically the tl;dr of how libsnes became libretro), and, on a personal level, I do not presume Near considered me a “friend” (or even thought of me much at all, for better or for worse). Like most of the people reading this, I suspect Near had a much larger effect on my life than I had on theirs.

Nevertheless, I hope Near understood the immense positive effect they had on my life and the lives of countless other individuals, not just through their numerous accomplishments but also as a compassionate and insightful human being. The world is a less-interesting place now, without Near.

Alcaro

We were never the closest friends; I was around for a while, but I was always more interested in bsnes than you, Near.

I now realize I was wrong. Too many open source maintainers are valued only for their contributions and otherwise taken for granted, leading to tragedies like this.
May you finally have the peace you were denied in life.

Daniel De Matteis

I have been a bsnes fan since the very beginning when it was first announced. I remember running Mega Man X2 on an Pentium 4 PC at the time in 2005 with a premature version of bsnes and feeling that finally there was an emulator that could get the sound exactly right, and it played exactly right. Those are memories and experiences that I will always cherish.

I was not even really involved in programming until around half a decade later. I had a chance meeting with Themaister around 2010 when Near was trying to see if bsnes could run acceptably well on a PlayStation3. This was my first introduction to Near back then as a non-user and to me this meant a lot. We pretty much got stuck at the 50fps mark, but what that chance occurrence did show me was the massive potential of libsnes as an emulation abstraction layer and how easily software could be ported across platforms so effortlessly without having to maintain multiple copies of a codebase per platform. I was sold there and then on the entire potential of libsnes and SSNES. And one thing lead to another.

Fast-forward to 2021 and it’s been well over a decade since the project started and I’ve been running it now for all this time. You now know of libsnes as libretro and SSNES as RetroArch but the core concept has remained fundamentally the same, right down to the same API. Where there was in 2010 only one emulator implemented as a core (bsnes), now there is nearly 200 implementations, and not all of them even emulators. The only credit I can take in this is that I have put an inhuman amount of time in building the road so that people will come with daily maintenance and coordinating of projects so that everything works well within Libretro/RetroArch as an ecosystem. But Near absolutely deserves the credit for coming up with the foundational pillar on which this all stands.

For what it’s worth, I regret that for a long time, we did not see eye to eye, and at one point there was a fair bit of animosity between us. Thankfully in 2017 we were able to shake hands and leave the past for what it was. There were hurt feelings, disgruntlement and disillusion on both sides about the emulation scene, the way it was becoming more commercialized and co-opted by commercial interests [from private correspondences I can confirm this bothered Near deeply too], the sides people took in that while it was happening, and people getting in the middle trying to amplify those grievances and wounds. I apologized then, but I will also apologize in the present for how things went before.

Like indicated earlier, I was a fan of Near’s work before I got involved myself in the scene, and going into this had nothing but the highest admiration and apprecation for Near’s work. Ultimately the last thing I wanted was for there to have been a falling out. I tried reaching out many times before trying to squash this situation since ultimately I regretted that it had come to this and felt it should not be like this. Ultimately without a mediator I was getting nowhere with this.

In 2017, thankfully all of this we were able to put aside forever thanks to the help of a mediator, and both sides assumed good faith from that point forward. I was hugely relieved by this and took this very seriously on my side. I was committed from this point on to show that not only was there always appreciation and respect, but even more importantly, I wanted to show this not in words, but by action. Since then, I worked together with Near on many occasions. This includes involving them in a Eurogamer article about emulation licensing and helping them with license abuses from third parties. Various email correspondences centered around emulation and the libretro project have been exchanged and we had been active supporters of Near’s Patreon. Additionally contributors upstreamed libretro support into bsnes in 2018 sparking various collaborations and friendly terms. Another example is the libretro-backed Parallel RDP project where we encouraged themaister to help integrate it into the N64 emulation in ares for accurate graphics emulation. I am grateful and thankful we were able to do all these things together. To date, my biggest regret is that we had that unfortunate initial falling out in late 2010. But, I am grateful to Near that there was the willingness to leave the past behind in 2017 and start over again. Ultimately, it could have all been so easily avoided and the things that divided us were ultimately so minuscule as to not even matter.

While I won’t profess to be a brilliant coder myself, I do recognize brilliance and potential where I see it. And projects like hiro, libco, and libsnes definitely show that raw genius that was in Near. Hiro for instance was a crossplatform GUI wrapper that would wrap complicated OS-dependent windowing APIs around one common abstract interface. There exist other examples like this but it was well designed and versatile, and used to full effect in Higan/bsnes. I would be remiss not to point out another great invention, libco. It’s a cooperative threading library, very small in size, that allows one to jump from one place to the next without resorting to actual real ‘threading’. Libco’s legacy (and thereby Near’s legacy) lives on in many libretro cores today. And libsnes needs no introduction, it morphed into libretro and has largely stayed fundamentally the same.

Near’s last few projects were equally as important. Ares was a multi-system emulator that covered all sorts of classic game computer and handheld computers from the ’80s to the early ’00s. One contribution that came from us was the aforementioned Parallel RDP Vulkan-powered plugin which would allow fullspeed accurate video emulation of the N64, as well as Parallel RSP. But Near also had spent nearly a decade or more on a translation of the videogame Bahamut Lagoon, which they considered their magnum opus. This was a painstaking work of passion with the kind of attention to detail few other localizations can boast. Near actually started this translation before even creating bsnes itself, as the Bahamut Lagoon translation was the catalyst for bsnes to emerge as a project. Current SNES emulators at the time had a lot of issues that were difficult to work around, with Near preferring a more clean approach without resorting to all the hacks.

I’m afraid it will be highly unlikely we will ever see a programmer as gifted involved in open-source emulation again, certainly not one with as much commitment and passion. Everyone with an appreciation for the Super Nintendo (yes, even Nintendo themselves) owes a huge amount of debt to Near for doing the near impossible in documenting and preserving this system and its back catalogue for all of posterity. It is very rare that you see someone with such selfless commitment dedicating themselves towards fully preserving a system.

Person who wishes to remain anonymous

Near was a very kind and sweet person, who cared for others. They were very talented and was amazing at what they did. I hope that they have found peace.

Lakka 3.2 released!

Lakka 3.2 has just been released! To learn more, check out this article on our sister site Lakka.tv here.

This version is based on the latest RetroArch version, 1.9.5.

We have the next version of RetroArch (v1.9.6) scheduled for next Sunday (26/6/2021). You can always view our roadmap for Lakka and RetroArch here.

RetroArch 1.9.5 released!


RetroArch 1.9.5 has just been released.

Grab it here.

If you’d like to learn more about upcoming releases, please consult our roadmap here. The next version of Lakka (with an updated RetroArch 1.9.5 version) is scheduled to be released a week from today.

Remember that this project exists for the benefit of our users, and that we wouldn’t keep doing this were it not for spreading the love with our users. This project exists because of your support and belief in us to keep going doing great things. If you’d like to show your support, consider donating to us. Check here in order to learn more. In addition to being able to support us on Patreon, there is now also the option to sponsor us on Github Sponsors! You can also help us out by buying some of our merch on our Teespring store!

Release notes

Be sure to also read our Libretro Cores Progress Report (a link will appear here later).

Direct3D 10/11/12 should now allow for fastforwarding while in fullscreen mode. The window title should now fully update periodically as well when using Direct3D 10/11/12 drivers. For instance, frame count and fps was not shown in the window title before. This should now be corrected.

CRT SwitchRes support has been completely overhauled. Read below for the details.

Highlights

Enhanced search functionality to the ‘Manage Cores’ menu

1.9.5 adds enhanced search functionality to the Settings > Cores > Manage Cores menu (with behaviour identical to that of the core downloader

1.9.5 also wires up proper left/right input actions for core manager entries, so the Manage Cores list can be scrolled rapidly by pressing/holding left/right.

API extension for setting ‘need_fullpath’ based on content file extension and to request persistent frontend content data buffers

Before, the libretro API had two major shortcomings when it comes to handling content:

* The need_fullpath parameter, used to define whether content is loaded by the frontend or handled by a core, is set ‘globally’ at the core level. This means any core that supports both CD-type and ‘normal’ ROM content has to set need_fullpath = true for all content, since CD-type files are generally too large to fit in memory – and in doing this, they are then obliged to load small ROM files internally, disabling any frontend softpatching support, and requiring the frontend to extract compressed content to temporary files on disk.
* The content data buffer passed to retro_load_game() is transient – a core must assume that it will become invalid once retro_load_game() returns. This means any core that sets need_fullpath = false must duplicate the content data buffer – so for example, loading a 32 MB GBA ROM with mGBA requires 64 MB of free RAM. This is a substantial issue on platforms with limited memory, and means that some cores are forced to set need_fullpath = true just to minimise RAM consumption.

1.9.5 seeks to address these problems with the addition of a new RETRO_ENVIRONMENT_SET_CONTENT_INFO_OVERRIDE environment callback:

* This allows need_fullpath to be set based on file extension
* It also allows cores to request a persistent content data buffer, which is guaranteed to be valid for the lifetime of the core (such that data duplication is not required)

Accompanying this is a new RETRO_ENVIRONMENT_GET_GAME_INFO_EXT callback, which allows extended content information to be retrieved in retro_load_game() (specifically, it provides content file path/metadata information – often essential when loading content from compressed archives – and buffer ‘persistence’ status)

The manner in which this functionality is implemented has a further side benefit. On all platforms that support runahead, for any core that sets need_fullpath = false another copy of the content data buffer must be created and stored in case the user toggles second instance runahead on while the core is running. Now that we have a mechanism for maintaining persistent content data buffers, we just enforce persistence whenever runahead is supported – so the initial frontend-loaded content buffer can be used directly by the second instance core, with no duplication.

NOTE: A couple of cores are already implementing this new feature. Unfortunately, not every emulator will be able to directly use the persistent buffer without copying it first into a temporary buffer. Some emulators like writing back to the ROM buffer in-memory (such as adding speedhacks or other alterations). In instances where this happens, we unfortunately cannot just directly set a pointer to the persistent buffer and need to still have a memory buffer copy.

New CRT SwitchRes implementation

Check out this article for all the details here.

What is new?
* Better modeline generation.
* Faster and more stable switching.
* Windows dynamic resolutions. No longer limited to locked resolutions/Hz. Timings are modified on-the-fly (ATI only)
( Stable resolution restoration. (If RetroArch should crash you will be stuck in the previous resolution that RetroArch was in)
* Improved super resolutions with Integer scaling.
* Fixed primary monitor issue. You no longer have you use your primary monitor only.
* Improved monitor indexing. Allowing for multi-monitor support.
* Low and High resolution options for the RetroArch menu.
* (RawInput support for absolute mice devices (Lightguns, to name a few) had issues with resolution changes, this has been fixed co-ordinates are reassigned after a resolution change.
* New CRT SwitchRes Menu options to remove the need to edit the switchres.ini.
* Switchres.ini can be used to set some advanced options.
* Monitor profiles – these allow you to change the modeline generation profile to fit many TV/monitors. (Some available from the CRT SwitchRes menu)
* Custom CRT ranges – this allows you to set custom monitor profiles. This can help with uncommon TV/Monitors and geometry issues.

Changelog

1.9.5

  • ALSATHREAD: Make alsathread default for all ALSA devices with threads
  • ARCHIVE: Fix loading of archived content with file names containing ‘#’ characters
  • CHEEVOS: Upgrade to rcheevos 10.1
  • CHEEVOS: Challenge indicators
  • CHEEVOS: Group achievements by category in quick menu
  • CHEEVOS: Relabel ‘Start Active’ with ‘Encore Mode’
  • D3D10: Window title should now update
  • D3D11: Window title should now update
  • D3D11: Allow fastforward in fullscreen
  • D3D12: Window title should now update
  • D3D12: Allow fastforward in fullscreen
  • CRT/SWITCHRES: New implementation
  • FONTS: Improve message wrapping with CJK languages
  • FONTS: Fix garbled characters when converting encodings
  • INPUT: Allow the 8 analog stick directions to be used as keys for core keyboard mappings
  • LIBRETRO: Add API extension for setting ‘need_fullpath’ based on content file extension and to request persistent frontend content data buffers
  • MENU/SEARCH: Add enhanced search functionality to the ‘Manage Cores’ menu
  • OPENDINGUX: Fix black screens when triggering gfx driver initialisation via menu actions
  • UNIX: Get better battery stats on sysfs nodes
  • VIDEO: Extend Frame Delay range to 19 to accommodate PAL land too
  • WIFI/LAKKA: Add nmcli to wifi drivers
  • WIFI/LAKKA: Add wifi configuration menu
  • X11: fix fullscreen when swapping monitors/resolution

Lakka 3.1 Released!

Lakka 3.1 has just been released! To learn more, check out this article on our sister site Lakka.tv here.

This version is based on the latest RetroArch version, 1.9.4.

We have the next version of RetroArch (v1.9.5) scheduled for next Saturday (12/6/2021). You can always view our roadmap for Lakka and RetroArch here.

RetroArch 1.9.4 released!

EDIT/UPDATE 31/5/2021: We have pushed a new 1.9.4 version. Specifically this fixes a bunch of regressions in the Vita port which unfortunately snuck into the initial 1.9.4 version. If you are using RetroArch on a PS Vita, we highly recommend you redownload the stable again. There are no real changes for other platforms.


RetroArch 1.9.4 has just been released.

Grab it here.

If you’d like to learn more about upcoming releases, please consult our roadmap here. The next version of Lakka (with an updated RetroArch 1.9.4 version) is scheduled to be released a week from today.

Remember that this project exists for the benefit of our users, and that we wouldn’t keep doing this were it not for spreading the love with our users. This project exists because of your support and belief in us to keep going doing great things. If you’d like to show your support, consider donating to us. Check here in order to learn more. In addition to being able to support us on Patreon, there is now also the option to sponsor us on Github Sponsors! You can also help us out by buying some of our merch on our Teespring store!

Release notes

Be sure to also read our Libretro Cores Progress Report – lots of work has gone into all of the various cores that are maintained (either by us or elsewhere), and it’d be a shame if the work goes unnoticed. Read it here.

There were some issues with RetroArch on PlayStation TV devices which should now be resolved. Additionally, it’s possible to run RetroArch at 720p now on a PSTV if you use the Sharpscale plugin.

Just like in version 1.9.3, we have been going back and improving code in RetroArch to improve file I/O performance, something that is very important for systems suffering from slow disk storage. Most game consoles would fall in this boat because all file I/O tends to be typically unbuffered on homebrew SDKs. In the process, we have discovered some parts where RetroArch was being inefficient when loading files from compressed files (such as .zip or .7z files). In the past, it would extract this file first to a temporary directory on the disk, and then it would read from this file and load it into the RAM buffer. Now we load it into the RAM buffer directly from the compressed file without first extracting it to disk. As if that wasn’t bad enough, on any platform that supports runahead, we would have to create another copy – even when runahead is disabled. And if cheevos are enabled, that’s another copy. All things combined, it would take 128MB of RAM to load one 32MB GBA ROM. As of 1.9.4, this RAM usage is severely cut down for cores that set ‘need_fullpath’ to false.

PlayStation2 users get a new core, prboom (a Doom 1/2 game engine). Thanks to a new and improved toolchain for PS2, this runs at a very impressive framerate, targeting 60 frames per second with stock settings. There might be some minor dips to the 50s in the busier scenes but nothing too serious, and disabling settings like ‘Wiggle Geometry Fix’ might help alleviate that.

Highlights

Prevent unnecessary extraction (to disk) of compressed content files

In previous versions, when loading content from compressed files, RetroArch always extracts the archive to a temporary file – even when cores specify need_fullpath = false. This is incorrect behaviour. If a core does not explicitly need to load a file from disk via some internal mechanism, the frontend should merely provide it with a data buffer. RetroArch was doing this, but in absurd fashion, i.e.:

  • Content is extracted to a temporary file on disk
  • Temporary file is loaded into a memory buffer and passed to the core
  • Temporary file is deleted when core is unloaded
  • This is a huge unnecessary performance overhead, and it causes significant unnecessary wear and tear on flash storage devices…

1.9.4 fixes the issue. Now if compressed content is loaded into a core that sets need_fullpath = false, the file will be loaded directly into memory – no disk writes will occur.

Additional notes:

Previously, it was in fact impossible to load content inside zip files directly into RAM. This has now also been resolved.

The end result? Less read/write on disk storage, which will make a big difference in terms of game content loading time on systems with slow file I/O (typically game consoles).

Option to select between ‘touched’ elements and physical controller inputs when showing inputs on overlays

The ‘Show Inputs on Overlay’ option was previously ‘broken’ when using remaps: the mapped button is highlighted rather than the pressed button, which is confusing for users and at odds with every other application (in existence) that has on-screen touch controls.

1.9.4 remedies the situation by changing the ‘Show Inputs on Overla’y option from a bool to an enum, with the following settings:

  • OFF: No inputs will be highlighted
  • Touched: The overlay element that is touched/clicked will be highlighted, regardless of which RetroPad button it corresponds to (default setting on mobile platforms)
  • Physical (Controller): Actual inputs passed through to the core will be highlighted (including remaps). This is the default setting on non-mobile platforms, and can be used by streamers, speedrunners and suchlike to show the actual controls they are using

When Show Inputs on Overlay is set to Physical (Controller), the index of the ‘physical’ device to be monitored can be set via a new Show Inputs From Port option (this option did in fact exist already, but was hidden/disabled by a long standing bug)

Changelog

1.9.4

  • CHEEVOS: update rcheevos to v10.0.0
  • CONTENT LOADING/FILE IO: Prevent unnecessary extraction (to disk) of compressed content files when need_fullpath is false
  • CORE INFO/FILE IO: Enable core info cache by default now for all platforms
  • CORE INFO/REGRESSION FIX: Fix regression caused by core info file caching – Downloads was no longer showing up in Load Content
  • FILE IO/COMPRESSED: Ability to load content inside ZIP files directly into RAM
  • INPUT/OVERLAYS: Add option to select between ‘touched’ elements and physical controller inputs when showing inputs on overlays
  • INPUT REMAPPING/OVERLAYS: Prevent duplicate inputs when using remaps with input overlays
  • LAKKA: Add brightness restore hook
  • LOCALIZATION: Fetch translations from Crowdin
  • MENU/OZONE: Added simple playlist entry enumeration
  • MENU/XMB: Fix display of ‘Maximum Users’ menu entry dropdown list
  • PS3/PSL1GHT: Joypad driver works again
  • PSTV: Fix Vita input driver for PSTV
  • PSTV: Support for 720p on PSTV when using ‘Unlock framebuffer’ in Sharpscale plugin
  • RPNG: Fix some memory corruption if processing broken input PNG file
  • SECURITY: Fix CVE-2021-28927