DigiWorldz, one of OpenSim’s most successful commercial grids, went down Tuesday due to a database issue and will be down for a few days while the grid uses the opportunity to do a major update. All in-world events have been canceled, and residents are used to follow the grid on social media on Facebook, Twitter, Discord, and MeWe to track the status of the grid. The Facebook group seems to be the most active. Users can also visit the DigiWorldz beta grid during the downtime.
Before this outage, the grid has had three days of total unplanned downtime over the course of the previous year, grid owner Terry Ford told Hypergrid Business.
“That means DigiWorldz has been available to users for 99.178 percent of the year, which is pretty good and we’re quite proud of that,” he said. “We’ve been blessed by very little downtime up to this point and for that we are thankful.”
And while the current outage wasn’t planned, the grid has been preparing users since March that the grid will at some point need a week to upgrade code and do server maintenance.
“With the sudden onset of the database issue, we decided we may as well just do the code upgrade etcetera while we were down so we didn’t have to take the grid offline again in a week or so,” he said.
The grid has a backup system with an alternate core server, which has been only needed once so far, said Ford, last September. The backup isn’t as fast as the main grid but it is available to use in emergencies.
But DigiWorldz currently has more than 900 regions — a total area of 8,369 standard region equivalents, since some regions are large, variable-size regions — on a total of 38 servers.
So when a drive failed on the grid’s RAID storage array at 9 p.m. eastern time Tuesday evening, getting it back up and running wasn’t as easy as flipping a switch.
It requires taking the grid offline, changing configurations, and restarting the grid, which takes time because of the grid’s size.
“Restarting DigiWorldz is not a quick task to perform,” said Ford. “When we considered how long it would take to reconfigure, then restart all of the regions while we repair the main core machine, then once again take the grid offline to switch back over to the repaired main core, only to announce to our users we would once again be taking the grid offline in a week to perform our code upgrade and maintenance, it made better sense to just do it all while we were down.”
But, in the future, the grid will be a lot more resilient, said Ford.
“The good news is that we will be on new code, our servers will be fresh, and we will be implementing a new custom asset server which will serve our assets to our users much faster,” he said.
The new system will also be “self-healing,” he added. Assets will be stored in four different locations, all live.
“If, for some reason, one of our assets nodes becomes unavailable due to network or hardware issues, our new asset server will simply look on one of the other nodes for the asset,” he said. “If or when the failed node should return, any asset uploads or saves it may have missed will be sent to it.”
Now with content protection
The new asset server will also help us in our fight against “Copybot” intellectual property violations, he said.
“We will now be able to designate an asset as being banned, and any attempt at bringing in an asset designated as banned via viewer upload, OAR load, IAR load, or hypergrid will fail as our asset server will catch it, even if it’s been renamed, even if the owners or creators have changed,” he said.
OARs are full region export files. Users on many grids, including DigiWorldz, can save their entire regions as OAR files and then upload them to other grids. Or they can upload third-party OAR files. For example, Outworldz offers a large collection of pre-made, legally licensed, free OAR files for anyone to use.
IARs are export files of inventory folders. An IAR file can contain one virtual object, or a large collection of them. Many free IAR files are also available at Outworldz.
However, some hackers distribute content, mostly illegally collected in Second Life, in the form of pirate OAR or IAR files, or through in-world stores. When this content shows up on a legitimate grid, it hurts the grid’s reputation, and also damages OpenSim in general in the eyes of content creators. With so much legal content available for free, or for sale at very low prices at in-world stores or on the Kitely Market, the piracy hurts OpenSim grids and users for no good reason.
Ford said that the new self-healing, content-protecting asset server will be available to other grids who use DigiWorldz’ hosting services.
“While our database issue was unforeseen — and we’re sorry we didn’t have more time to announce the downtime — it will result in less downtime later and in the long run it will make DigiWorldz a better performing grid,” he said. “We’ll be better positioned to fight illegal content proliferation, and our users’ assets will be better protected from loss.”
Source: Hypergrid Business
