Meeroos Addresses It's Pet Issues

 

Longtime residents will likely remember the Meeroos, even if they never got one. Introduced in 2011, these cute virtual pets known for their wanting to snuggle were very popular then. Almost fifteen years later, they're still around, with a smaller but still active fanbase.  Recently a friend of mine who owned some started telling me about trouble. But after several days, there was finally a response from someone in the company behind them.

Read the story in Design. 

EOTB: Linden Lab Acknowledges Some Residents Are Having PBR Related Issues

 

If you've been having issues with the viewer in Second Life since the PBR (Physical Based Rendering) updates, you're not alone. Numbers of people have complained since it's introduction of increased lag, longer rezzing times, and crashes.

 

In their latest post on the official blog, Linden Lab was basically acknowledging there was a problem,  

We are always striving to add more improvements to Second Life and we’ve been particularly excited to roll out integration of PBR Materials, which has the potential to improve and modernize much of the look and feel of our virtual world. However, the introduction of PBR has not gone as smoothly as we would like and we have heard community frustrations and feedback with recent PBR-enabled Viewer releases of both the Official Second Life Viewer and Firestorm Viewer.

We share your frustration and are taking steps to make things better for everyone on both lower-end and high-end systems. This means that we’re actively focused on improving the performance in a variety of areas including improved FPS, VRAM management, texture loading, alpha blending, and even how lighting is rendered within the world. Additional fixes are already in our development pipeline and will head to QA soon.

Of what to do, the Lab suggested that those using the Linden Viewer download this update (link).

If you are having issues with the Official Second Life Viewer, you can immediately download an update here that addresses many of the issues affecting performance, particularly among those who may be using older or less powerful computers. We are continuing to work on additional fixes that will further address texture loading issues and low framerates. If you don’t update proactively now, you’ll soon see a prompt when you go to login to Second Life in the coming days encouraging you to update. 

 

For those using Firestorm, the Lab stated, "We are working closely with the Firestorm team to coordinate the rollout of updates that will improve your performance." They would mention that Team Firestorm made the decision to keep their last pre-PBR viewer around indefinitely .

 

The next release of the Firestorm viewer will include all the performance fixes we are working on. You’ll be prompted to update your Firestorm Viewer once it is ready in the next few days, but you can also be one of the first to try it by joining the Firestorm Preview Group inworld and continue to stay connected with their development progress on the official Firestorm Viewer website and its dedicated download page.

For the blogpost in full, Click Here. 

Bixyl Shuftan

Breaking News: PBR Mirror Bug Results in Mirrors Breaking, Seven Days of Bad Luck

 Update: April Fool's!

 

Linden Lab just announced there's a glitch in the PBR (Physical Based Rendering) code that affects mirrors in Second Life. Part of the glitch is that bumping into mirrors, touching them with an object, or dropping a mirror onto the ground can result in them breaking. 

 

And when the mirrors break, the person who caused it will go through a number of glitches, such as trouble rezzing, teleporting failures, crashing at sim crossings, difficulties in shopping at Marketplace, and more, for a period of several days. 

 

Supposedly for most involved, the period of time is seven days. Then the issues all clear up. 

To see Linden Lab's statement, Click Here.

Linden Lab Explains "February 1st Outage"

 

With the scale of the disruptions on Wednesday February 1st, it was likely Linden Lab would find themselves obligated to their customer base, the residents, to try and explain what happened. It turns out they made a statement about the "Recent February 1st Outage" the following day in the "Tools and Technology" section of the official blog. 

This Wednesday, February 1st, was a rocky one for Second Life, especially for regions on our Release Candidate (RC) channels. As with many large-scale services, we experience service interruptions from time to time, but yesterday's problems were unique and noteworthy enough that they deserve a sincere apology and more thorough explanation. 

 

The Lab stated the main cause was "a bug in object inventory behavior that was introduced by new code" saying the bug, "caused some objects' scripts to enter a non-running state."

RC rolls are regularly scheduled for 7AM Pacific (SLT) every Wednesday. While conducting one of these deployments, we received initial reports of issues relating to script behavior at 8:30AM. We immediately started investigating, and by 9AM it was clear that we needed to stop the roll, because the issue was pervasive and clearly related to the new changes. As we do in these situations, we declared an “Incident” and an Incident Commander took charge. 

By 9:30 we started a rollback, reverting the affected simulators to their previous version. We also evaluated what additional actions we would need to take, as it was unclear that a rollback alone would start the scripts which had stopped. This quickly proved to be the case, and we came up with a new plan - one that would ensure that scripts would perform as expected going forward, but possibly undo the changes our residents had been making since the time we had introduced the bug. Although this meant more downtime, it prevented further content loss, and proved to be the best way to put the grid back in order. The team came up with a quick, clean, and efficient way to achieve this and get everyone back on track. 

By 12PM the decision was made to take this direction. By 1:15PM the code was complete, by 1:40PM we confirmed that it worked. By 2:25PM all regions were brought back up.  

The report in full, signed by Grumpity Linden "and the Second Life team," can be found (here).

"I’m grateful to know that while we may make mistakes, we will not sweep them under the rug, nor look for someone to blame - we will come together and make it right."

Grid and Viewer Glitches?

 

 Lately it seems my alt Rezzdarnnit is living up to his name.

It's been a little more than a week since the February 1 outages that took out a number of sims for a while, caused people to have problems logging in, and other problems. Since then, I've heard of, and to a lesser degree experienced, a few issues that aren't just the usual lag. Some people are taking longer to rezz, longer to see others rezz, longer to see sims rezz, and some trouble logging in.

  

The news isn't *entirely* bad. Since upgrading my Firestorm on the day before the required update, my frames per second is sometimes quite a bit more than it was before, at least when there's few to no other avatars around. 

Checking the Grid Status Report, there was a mention of a long-running issue "Online Friends Display Issues," for close to a month, " We are aware that some of our Residents are currently experiencing issues with online friends not showing correctly. Our engineers are hard at work to correct this behavior as soon as possible; however, there may be an extended time frame before this occurs." 

 

This hasn't been the first time the Grid has had more than the usual amount of bugs. During Second Life's move to Amazon Cloud servers, there were many to the point Linden Lab was asking for patience for the many weeks it would take. What could be causing these issues, we don't know, yet.

Hopefully things will be fixed soon.

Bixyl Shuftan

Glitches Plague Second Life

 

 

 Second Life invariably has occasional glitches. But February 1 2023 was worse than usual. Numerous sims were offline. Some avatars wouldn't rezz properly or at all. People had trouble logging on. The Lab had been doing some rolling restarts. But it soon realized it had a problem, admitting to outages and script errors on it's Grid Status Report.

 

We are investigating reports of occasional errors upon logging into the viewer as well as reports of slowness or time outs on web pages such as the Marketplace and my.secondlife.com. We are also aware some residents are appearing as a cloud for longer than expected upon logging in. Our engineers are actively trying to alleviate the issue. We apologize for the inconvenience.

 

Some residents would describe their problems on a couple forum threads here and here, "I've been an orange cloud for three hours." Draxtor Despress' Second Life Book Club would have to postpone it's weekly show to Saturday due to the glitches, which showed in his recording. Checking the maps, one friend commented, "I don't think I have ever seen so many sims down at once on the map."

 

On 3:45 PM , the Lab posted that the issues were resolved. But for about an hour after that, I continued to see people having problems with their avatars rezzing. By 6PM, it appeared things had finally cleared up.

 

It;s safe to say this was Second Life's most glitched day in many months. Fortunately, it wasn't as bad as the early days in which it seemed Linden Lab had to shut down the Grid for a few hours every week to "bang on things" until the bugs were fixed.

 

Image Credit: Vicious Hollow, Matoazma Kazyanenko

 

Bixyl Shuftan

 

EOTB: The Lab Adresses Recent Outages

 

Second Life has seen it's share of glitches over time. But last week there were a number, notably the rolling restarts on Tuesday January 27 that went snafu and resulted in some sims offline for a while. A few days ago in the "Tools and Technology" section of the official blog, the Lab came out and admitted there were multiple problems.

*  *  *  *  *

Well, this is not the way we wanted the past week to go -- multiple outages in the same week! Not fun for anyone. Here’s what happened.

Login and Inventory Issues: Monday, 1/24 - Friday, 1/28
On Monday, January 24, residents’ inventories stopped responding to requests. Some residents were unable to log in as a result. We identified the affected accounts, addressed the problem and restored services to full operation in a little under 3 hours. 

Beginning on Wednesday of this week, we received reports of intermittent inventory issues – failures to rez, unpack and delete inventory items; and intermittent errors upon login.

When the number of reports increased, it became clear that this was an outage. Several Lindens jumped in to diagnose the reports. After some digging, we discovered our back end infrastructure was being overloaded. Once this was resolved, the positive impact was almost immediate. The data now looks good -- on the back end and we are no longer receiving inventory reports.

We’re taking steps, including a deploy late last week, to prevent these issues in the future - and we have already seen progress in making the service more robust.

Weekly Rolling Restarts: Tuesday, 1/25
Every week we restart the simulator servers to keep them running smoothly. We only restart a certain number at the same time, allow them to finish, then start another batch. That’s why we call them “rolling restarts.” But on Tuesday a recent upgrade to the simulators meant that the usual number of simultaneous restarts was too much for the system to handle. The result was load spikes and numerous regions going down. Ultimately more than 12,000 regions were stuck in restart mode, which is 40% of the grid. Not good!

The team came together quickly to bring simulators back up in smaller batches, and then manually fixed blocks of regions. After some trials and monitoring, we found a smaller number of concurrent restarts that worked better. By 1:00 PM all regions were restored and operating normally. We apologize for this lengthy outage!

Advance… Retreat!: Wednesday, 1/26
We had planned a deploy to the production grid that would help us gather information on group chat performance. Unfortunately, the procedure used during testing did not work in production. We figured -- OK, we’ll just roll back. But then the rollback itself had problems!  Residents experienced issues with login, group chat, and presence information. The team was able to isolate the problem and complete the rollback, getting a few more gray hairs in the process. We’re going to take what we’ve learned and do a better deploy that will give us the information to improve group chat in the long run.

Last week was no fun for a lot of people at the Lab. Thank you for your patience. We really don’t like interrupting your enjoyment of Second Life. Here’s hoping we don’t have to come back with another one of these blog posts for quite some time! 

*  *  *  *  *
 
The Lab expressed hope that "we don’t have to come back with another one of these blog posts for quite some time! "
 

Technical Difficulties For The Grid on Tuesday

 *

Tuesday January was the day rolling restarts were done this week around the Grid. Usually they go routine. But this time something went wrong. Some residents would have trouble logging in, and others noted some sims looked offline. When I tried logging in on Sunweaver Bay, the login screen got stuck in the middle of the progress. In various Second Life Discord channels, people noted their difficulties.

Eventually, the Lab noticed there was a problem, and posted on the Grid Status Report.

We are aware rolling restarts are taking longer than normal, and we are working to bring these regions back online as quickly as possible. We thank you for your patience! Please follow this blog for further updates. - 08:06 PST

We apologize for the delay, and are continuing to work to bring these regions back online as quickly as possible. We thank you for your patience! Please follow this blog for further updates. - 09:10 PST 

 

We are continuing to work on restoring the regions still offline from this morning's Rolling Restarts, and are making progress. Please watch this blog for updates. - 09:45 PST

 

Getting online, there was chatter about it in several groups I was in, including the Bellisserian chat. At one point, Abnor Mole spoke up, "The restarts this week have been going... poorly. The region your home is in it probably one of those affected, but it is not alone. The Lindens are well aware of the problem and working on it."

Eventually, the Lab finally fixed whatever the problem was, but decided to restart some of the sims again to make sure.

We believe this issue has been resolved but will be restarting some regions as part of the final process. Thank you for your continued patience! - 12:45 PST

Eventually, all was back to normal, with the usual lag, and the residents went on.
 
Bixyl Shuftan
 

Reader Submitted Commentary: "The Many Faces of Linden Lab"

 

I have been on Second Life for close to 15 years now.  I have been here through the deforming teleports, the iffy, sim crossings, the constant red clouds.  Over the years, the people at Linden Lab have worked on many of these issues and have reduced if not eliminated many of them.  Sim crossing, with vehicles, is still hazardous however. I have no idea why they can't seem to fix that issue, most other video games have no such problem.

I remember when they had actual Lindens hangout with the people that inhabited their world, listening to what we said about Second Life. What was good, bad, needed work, etc.  That kind of feedback in valuable. They stopped doing that to save money, they said, and service has suffered.

I can understand why Linden Lab stopped gambling, on Second Life years ago and just recently Gatchas. US and state gov'ts breathing down your neck, is great incentive to deal with it.  The issue I have was the way they dealt with it.  It was a major PR disaster in both cases.  With gambling, people lost so much money they had invested, in SL, they got really upset, and left SL, never to return.  The player base of SL has never recovered.

Their biggest fiasco has been Sansar.  No one asked for this, nor wanted it. Linden Lab failed to take into account that we would NOT be able to take with us years and thousands upon thousands of dollars worth of sims, avatars, clothes, no copy items, custom items, etc, to  their new grid. That was a non starter. No one was willing to toss away all their accumulated inventory, to move to a new sim. That would be madness.  LL really didn't think that one through. Instead, all that time and money could have been better spent on improving their existing grid, Second Life!

Another issue is the two-faced handling of content creators.  Linden Lab loves the lindens they get from players buying, the items these creators, Spending hours building items like avatars, furniture, homes, clothes, misc, whatever you heart desires. How many of these talented creators have left SL, because LL doesn't care that their creations get copybotted and sold by the offenders. Complain the LL, you get deaf ears.  Going after the copybotters is WORK!

Linden Lab, instead of trying to figure new ways to wring more money out of your existing player base, you might work at making it more attractive to actually WANT to play Second Life.  Build your player base.  The more new folks you can get to play the game, the more might actually like it, and start spending money..

My final bitch at Linden Lab, is a big one, and I can see ZERO EXCUSE for it.  If your Second Life account gets hacked, instead of LL going after the offender that HACKED you, YOU are treated as the criminal, get your account banned and they REFUSE to help you get your account back, no matter what info you supply. Then when you create a new account, they ban that one too, SACRILEGE!  LL wakeup and smell the coffee!  We play your game, we pay your bills, start treating us with respect!

Rita Mariner
 

Map Issues (Mostly) Fixed, New Sims Visible

After months of not working properly, the Second Life map is more or less back to normal. For months, one couldn't zoom out more than a little without the sims turning invisible. If a sim changed it's appearance, it wouldn't show on the map. And if a new sim appeared, such as the Chalet Linden Home ones that connected Bellisseria with the eastern mainland continents, it would be invisible except for it's name. Naturally, many residents complained, and even the Moles were saying they were inconvenienced. But for month after month, despite the efforts of Lindens working on the problem, including a greater effort after the move to the Cloud was completed in early January, the glitches remained. There were a couple resident-made maps that helped with the panning out problem, but the others remained.

Finally few days ago on April 15, Linden Lab announced on the Grid Status Report that they were on the verge of finally fixing it.

We know it's been a long time! Thank you for your continued patience. We finally have a fix in testing, and look forward to rolling it out soon! 

But it wasn't until yesterday, Monday April 19, in which they announced the work was done.

Hooray! We have a working system for generating map tiles again, along with a few other fixes to maps (zoom). We're still monitoring the results. Thank you for your patience during this long wait, and happy navigating. 

 It should be noted that sometimes the map still glitches a little when used. A time or two when I was trying it recently, sims did turn invisible when panning out or zooming in. But more often, they would appear normally, occasionally taking a little time to rezz. And there are other glitch issues, such as the group chat issues which have made communications between numbers of people a bit difficult at times.

So this means that any sims with big changes, like the sign over Raglan Shire, are now visible. And any new sims, like Bellisseria's Chalet sims on it's northeast, can now be seen. So now planning that boat trip across the Blake Sea or the Straits of Bellisseria will be much easier.
 
Bixyl Shuftan
 

Breaking News: Second Life's Cloud Servers Degrading, Lag And Other Issues To Get Significantly Worse

 Update: April Fools

In a stunning development, Linden Lab announced an unexpected and severe problem with the Amazon Cloud Servers that now house Second Life's data. The servers have been deteriorating and glitching, and the problem is going to get much worse.

"It turns out the servers that Amazon provided us with weren't meant to take the strain of a virtual world as active as Second Life is," the statement by the Lab explained, "Not only are we not getting the performance improvements we were hoping for, the lag and other issues such as teleport fails and inventory issues have been slowly getting worse over the past few months."

The statement went on to say that the servers are starting to deteriorate faster,  and so the issues will start to get even worse, and fast. It's estimated that within two weeks, viewers will need four to five times to load information around an avatar after TPing or logging on. Crashes an teleport fails will be several more times likely to happen as well, and items will start vanishing from inventories. Within three weeks, viewers will take ten times as long to process information after logging on or teleporting. That is if people can log on or teleport at all as fails will start to keep people from doing so for five to ten minutes. And within a month, Second Life will become unusable to those without newer and more powerful computers.

Linden Lab is working with Amazon to get better servers and transfer the data to them, but this will take time. For now, the Lab is offering a downloadable code that will allow the servers to better recognize users and their data and not suffer quite as much lag.

For more information and to download the code: Click Here. 

Bixyl Shuftan
 

EOTB: SL Destination Guide Still Being Worked On

Although Linden Lab completed the move of it's data from old servers to The Cloud a few weeks ago, the job isn't quite finished. A few days ago in the official blog, Strawberry Linden stated that the way the Second Life Destination Guide is updated developed bugs and glitches in the move that are still being worked on. There will be no new updates "for the foreseeable future."

On December 22nd I announced that the Second Life Destination Guide was being uplifted to the cloud. That process is still underway, however, we have run into some errors. For this reason, the web publishing tool that allows us to update the Destination Guide is still unavailable and will be for the foreseeable future until these errors can be corrected. I am hoping it won’t be much longer than another week or two. I will continue to update you on this status every Friday.

The submission form for the Destination Guide is also giving errors because of this, so please continue to email your submissions to editor@lindenlab.com and we will add them as soon as we have access again.

To read the blog post in full, Click Here.

"Thanksgiving Bakefail" - Grid Gliches Inspire Linden Humor

 

Despite that all the sims have been moved to the Amazon Cloud Servers, there's been no shortage of glitches on the grid. From longer avatar load times, to delayed teleports, to the map not showing landscapes at times, and more. There's been some minor grumblings from some residents (including a rumor or two). As for the Lindens and Moles, they've gone about fixing things the best they can. But it looks like the irritation of the residents hasn't gone unheard as there's been a display of some self-depreciating humor. On Thursday November 26, their Grid Status Report announced they were dealing with a problem causing people to stay clouded.

Our engineers are currently investigating an issue where some Residents may be unable to change outfits, always appearing as a cloud regardless of the outfit they put on. We have identified a potential cause of the problem and are working on reproducing and fixing the issue.

The title of the report was "Thanksgiving Bake Fail."

It wasn't until December 2 that Linden Lab announced they were well on their way to solving the issue. Other issues are still continuing to happen. But there hasn't been an opportunity for the Lab to show it's humor since, at least not on the Grid Status Report.

Bixyl Shuftan

Linden Lab: "We Apologize For Recent Service Disruptions"

Glitches are more or less a part of the Second Life experience. But if it seems they've been getting worse lately, it's not your imagination. And even Linden Lab is admitting things are buggier than usual. In the Tools and Technology blog on Friday October 16, Oz Linden stated in the "Uplift Update" that while there's been overall improvement in the Second Life infrastructure, "it’s the bumps in the road that are most noticeable to our residents. We apologize for recent service disruptions ..." He would point out several "rough spots."

Region Crossings
One of the first troubles we found was that region crossings were significantly worse between a cloud region and a datacenter region. We did a deep dive into the code for objects (boats, cars, planes, etc) and produced an improvement that made them significantly faster and more reliable even within the datacenter. This has been applied to all regions already and was a good step forward.

Group Chat stalls
Many users have reported that they are not able to get messages in some of their groups; we're very much aware of the problem. The start of those problems does coincide with when the chat service was uplifted; unfortunately the problems did not become clear until moving that service back to the datacenter was not an option. We haven’t been able to get that fixed as quickly as we would like, but the good news is that we have some changes nearly ready that we think may improve the service and will certainly provide us with better information to diagnose it if it isn't fixed. Those changes are live on the Beta grid now and should move to the main grid very soon.

Bake Failures
Wednesday and especially Thursday of this past week were bad days for avatar appearance, and we're very much aware of how important that is. The avatar bake service has actually been uplifted for some time - it wasn't moving it that caused the problem, but another change to a related service. The good news is that thanks to a great cross-team effort during those two days we were able to determine why an apparently unrelated simulator update triggered the problem and got a fix deployed Thursday night.

Increased Teleport Failures
We have seen a slight increase in the frequency of teleport failures. I know that if it's happened to you it probably doesn't feel like a "slight" problem, especially since it appears to be true that if it's happened to someone once, it tends to keep happening for a while. Measured over the entire grid, it's just under two percentage points, but even that is unacceptable. We're less sure of the specific causes for this (including whether or not it's Uplift related), but are improving our ability to collect data on it and are very much focused on finding and fixing the problem whatever it is.

Marketplace & Stipend Glitches
We've had some challenges related to uplift for both the Marketplace and the service that pays Premium Stipends. Marketplace had to be returned to the datacenter yesterday, but we'll correct the problems that required the rollback and get it done soon. The Stipends issues were both good and bad for users; there were some delays, but on the other hand we sent some users extra stipends (our fault, you win - we aren't taking them back); those problems are, we believe, solved now.

So why all of the glitches and bugs? Oz called the Uplift to move Second Life's data to cloud servers "a massive, complicated project that I've previously compared to converting a steam-driven railroad to a maglev monorail -- without ever stopping the train." He would later try to reasure the residents, "While this week in particular has seen some bumps in the road, it's actually going well overall. Lots of the infrastructure you don't interact with directly, and some you do, has been uplifted and has worked smoothly." He would go on to say almost all of the Beta Grid's sims were in the Cloud, and "we've uplifted around a hundred regions on the main grid. Performance of those regions has been very very good, and stability has been excellent." He would go on to say more sims would be uplifted over the weekend and that "Uplift of the Release Candidate regions, which will bring the count into the thousands, will begin soon."

Oz would say the Uplift to the Cloud should be completed, or at least be close to it, by the end of the year.

To read the blog entry in full, Click Here.

Bixyl Shuftan

Reader Submitted: The Cloud Move

Kἶէმƈεlἶმ Lմἶղƈօɾἶმlօէε Kεlνმɾl (Kitacelia Resident) recently sent a reader submitted commentary about what she feels has become quite an aggravating problem: increasing lag along with bugs and glitches due to Linden Lab moving data from it's current servers to cloud-based ones. The Lab's "good idea," from her point of view, is causing a lot of problems. So she took the time to explain the problem as she sees it.

Read Kitacelia's reader submission in Extra.

Tiny Empires Game Not Working With New Last Names

image from Liska Fuchs

Yesterday, Linden Lab announced that residents (with Premium accounts) could have their last names changed after paying a fee. But while the Lab has been trying to work out the bugs with how residents interact with objects and Marketplace, there are still problems with things outside the Lab's direct control. For instance, some resident-created games may not recognize the new names. In one group notice inworld, there was a warning to players of the game "Tiny Empires."

For Tiny Empires players:

LL has reinstated “last names” effective today as a new option for Second Life Premium Members. Premium account holders may change their first, last or both names.

However, active TE players should NOT, repeat, NOT do this at this time. There are reports that one GV changed her name and was demoted to a wanderer. The Emperor has been asked to confirm or deny this and if confirmed, fix the glitch that allowed it to happen.

Until that time, please do NOT change your name.

 This was also mentioned in the Second Life forums.

I just paid £42.97 to change my name to Eva Nova and was so pleased, until I logged back in to SL and put my games HUDS on to find all progress lost. I had a guild in Tiny Empires 3000 and also played the classic game, for many years. ...
 
Please everyone be careful before you rush in like I did because I have destroyed years of progress on my games.

Also, Oldesoul Eldemar notified the Newser of a notice he'd gotten from Caspertech:

We were prepared for the name change stuff coming well in advance. It will not affect the operation of CasperVend or CasperLet in any way.

However, if you change your name, you will need to continue using your old name to log in to CasperVend.
~Casper

So those in Second Life changing their names early will likely go through a number of inconveniences.

Bixyl Shuftan

Unscheduled Maintenance

Yesterday afternoon, people in some SL Discord groups were stating they were having trouble logging in. At 3:15 PM SL time, the Grid Status report stated there was some unscheduled maintenance going on.

We are currently performing unscheduled maintenance. At this time residents may experience trouble logging in, completing transactions, rezzing, and other in-world functions which may be unresponsive or slow. Please follow this blog for updates.

Before long, the issues were resolved, and people were able to log on without trouble.

Glitches on The Grid

On Friday January 17, Club Cutlass in Sunweaver Bay was having it's "Military" party when all of a sudden, everyone's viewer crashed. As people came back online, they saw the sim was down. It took several minutes, but eventually the area was back online and the party resumed.

But this wasn't the only glitch this weekend. On Sunday afternoon, January 19, people were saying in SL and Discord they were having a hard time logging on.  Checking the Grid Status Report, the Lab confirmed there were login issues that day, as well as trouble on the 17th.

We are aware that some Residents may be experiencing problems trying to log into and/or stay connected to Second Life. Our engineers are actively trying to alleviate the issue.

It took a few hours, but eventually the Lindens "banging on things" had some results, and people were able to login and log off without worry. At least most of us weren't worried. One friend of mine whom was both at Cutlass on Friday and had login trouble Sunday half-seriously wondered if somehow the Lindens were out to get her. They weren't of course, though it certainly felt like it.

Bixyl Shuftan

Rolling Restart Issues Trouble Second Life

In Second Life's early years, it was common for the entire Grid to be taken offline and the residents having to leave for a while so the Lindens could fix glitches. These days, things are much better with most residents having to face rolling restarts about once a week. But earlier this week, things began to go wrong. Starting on early Tuesday morning, people in some places found themselves unable to go back to some sims. This included Cyfir (Cyfiremmerich), whom couldn't get back to his home.

Read Cyfir's article in Design.

Glitches in the Grid

If you're wondering if Second Life seems more buggy than usual, you're not alone. Over the weekend and on Monday, I've heard from a number of people about various things on the Grid going wrong. Yours truly has experienced occasional lag, and sudden viewer crashes, and I've heard a complaint or two about sim crossings. But friends have told me some of their items just haven't been working right. A contest board at a club I frequent had trouble working for the manager. One friends permapetted horse got sick from hunger, and it was supposed to be free of the need to eat the company's food. Another friend was sailing, and even away from the sim borders the boat could barely steer.

So what was going on? On Thursday September 26, Linden Lab would announce in the Grid Status Report that an attempt to allow scripts to run better ended up breaking the ability of some to work with others.

Investigating - We believe we have identified the source of the scripting problems that rolled to the main channel this week (and so are present on the entire grid now). That simulator build had improvements to script scheduling that reduced overhead, allowing more scripts to be run per frame. Unfortunately, it appears that those changes also exacerbated a previously undetected problem with scripts communicating with other scripts.

The most obvious example of the problem concerns the timing of the object_rez event delivered to a script that has requested the creation of a new object. When a script creates a new object, the script that requests the creation gets an object_rez event to indicate that the new object is ready. Normally, the scripts in the new object should be running before the parent script gets the object_rez event. Similarly, scripts get an on_rez event when the object they are in is rezzed; the order in which those on_rez events occur is not defined, but previously all scripts in the object would get those events roughly at the same time. What we've discovered is that in rare circumstances, the on_rez and object_rez events could be delivered before the all scripts in the new object had been started. When that happened, any communication between the scripts would not work as expected.

This out-of-order event delivery was previously so rare that it had never come to our attention, but the script scheduling changes had the side effect of making it more likely that some scripts would get the rez events more quickly, increasing the chance that the events are delivered before the other scripts they expect to communicate with are ready.

We are working now on a fix; we hope to have the fix deployed on a set of test regions on the beta (Aditi) grid for the weekend so that creators can test with it (watch this space). The rolls for next week will revert the main channel to a version prior to the scheduling changes, and the RC channels will get a version with the improved scheduler but modified so that the rez events are never delivered to any script until after all scripts in the rezzed object have been started.

We know that this bug has affected many of you and your enjoyment of Second Life, and are profoundly sorry that it was not found before it was so widely deployed.

 The following day, the Lab announced on the Grid Status Report that there would be rolling restarts on the Grid.

Update - We will be doing a roll to address this issue on Friday, September 27th, at approximately 10:00 PM PDT. We apologize for the disruptive weekend roll and will try to minimize impact. Thank you so much for your patience during this frustrating time.

Early Saturday morning, September 28, Linden Lab would do rolling restarts. In the forums, Mazidox Linden would say:

"Yep, we are rolling the grid right now to revert the most recent update to LSL script improvements. This should prevent further issues with objects communicating. (But if you continue seeing issues with scripts, whether it's communication or something else we want to know about them!)"

On Monday, yesterday, the Lab declared the problem over.

Resolved - The scripting errors which resulted in many scripts, huds, and other items to not function properly should now be resolved.

However, friends of mine are still complaining about problems that are mostly likely due to script errors, "There is something wrong with (Linden Script Language) and (Linden Lab) broke it," one told me. She also told me as far as she knows, the Lab is still working on further fixes. So it looks like there are still glitches on the Grid.

Bixyl Shuftan