Unable to restore restore Roon backup due to System.IO.IOException: No space left on device

I also posted this question on the Roon community forum. I’m posting here because it looks like the root cause might be available storage on the CX.

Roon Core Machine

Antipodes CX running 1.8 (build 850) with 1.8TB on board SSD, of which 860GB is used.

Networking Gear & Setup Details

Unifi USG router and Unifi switches

Everything wired.

Connected Audio Devices

N/A at this point, but Meridian 818v3 is the main endpoint.

Number of Tracks in Library

roughly 25,000 tracks

Description of Issue

I did a fresh backup earlier today from a Mac Mini M1 running RoonServer 1.8 (build 831) to a never-before-used backup location on my QNAP NAS with plenty of storage capacity left (10TB at least). I confirmed that the RoonServer log did not show any errors or warnings during this backup. Backup status on the Mac Mini says: Last successful backup at 20:56:57.

When I try to restore this to my Antipodes CX running AMS 3.1 and Roon 1.8 (build 851), the restore gets to about 40% or so until it shows a “Restore Failed” error. The log suggests that the recover fails here:

11/30 15:09:53 Warn: [backup] filehash: e74e108d1a290c6f8e91b8fbfbe66cc21508f58618eebfae5a26b818b57f35cc, filepath: Database/Core/0448225cea1a4a62b253924cf873c75e/images_1/oj/ea/ojeaaaaa.1024.jpg 
11/30 15:09:53 Warn: [backup] restore, manifest file is corrupt: b_20211130192934_2a8de06c-27de-4dd0-bc04-7a8ead6dedb5 - System.IO.IOException: No space left on device 
at System.IO.FileStream.WriteNative(ReadOnlySpan`1 source) 
at System.IO.FileStream.FlushWriteBuffer() 
at System.IO.FileStream.Dispose(Boolean disposing) 
at System.IO.Stream.Close() 
at RoonBackup.Backups.Restore(Canceler canceler, IFileSystemService fs, IFileSystemDir rootdir, String machineid, String backupname, String outdir, Action`2 progress_cb) 
11/30 15:09:53 Warn: [broker/backups] restore first half failed on FileBrowser.Entry: \\10.0.1.18, homes : \xxxxxxxx\tmproonbkup\RoonBackups: NotAvailable

The Mac Mini was set up a few weeks ago as a temporary Roon core while the CX went in for service, using a backup from that same CX, stored on same QNAP NAS. The Mac Mini Roon core was working fine.

I haven’t 100% ruled out that this error is referring to a device on the CX that holds the Roon database. That said, on Antipodes servers this level of information is not exposed. I only get to see the storage details for the music library, which is on a removable SSD, for which the CX has two slots.

Further, if I don’t restore a backup, and instead let Roon discover the music on the SSD and build a new library, there are no issues. All 2,000 or so albums and 25,000 tracks import. Audio zones are discovered and music plays. This makes it seems less likely that there is a storage issue on the CX that would be triggered by a restore.

I also tried a restore from the backup manager, after having Roon scan my library as if it were a brand new library, in the hope that somehow the correct storage location would be used this time. Unfortunately, this produced the same error.

12/01 07:23:48 Warn: [backup] restore, manifest file is corrupt: b_20211130192934_2a8de06c-27de-4dd0-bc04-7a8ead6dedb5 - System.IO.IOException: No space left on device 
at System.IO.FileStream.WriteNative(ReadOnlySpan`1 source) 

In the end I was able to restore my backup.

The root cause appears to have been a setting in the Antipodes CX Server dashboard labeled “Expose Roon Server database for backup and edits” that I didn’t have enabled.