SL Poetry: "Oh TP Crash"

Oh tp crash,
Oh tp crash,
How I love the tp crash,

Oh tp crash,
Oh tp crash,
How I love the tp crash,

Just when I want to simply go home,
You lock me up and throw me to the gnomes,

Oh tp crash,
Oh tp crash,
How I love the tp crash,

Here is my hope for Linden wootmas gifts,
They only get smelly coal and malware bits.

Oh tp crash,
Oh tp crash,
How I love the tp crash,

The End , Thank you and Merry Wootmas 

by Caleb Kit, on the Raglan Shire Discord channel

Linden Lab: "We Apologize For Recent Service Disruptions"

Glitches are more or less a part of the Second Life experience. But if it seems they've been getting worse lately, it's not your imagination. And even Linden Lab is admitting things are buggier than usual. In the Tools and Technology blog on Friday October 16, Oz Linden stated in the "Uplift Update" that while there's been overall improvement in the Second Life infrastructure, "it’s the bumps in the road that are most noticeable to our residents. We apologize for recent service disruptions ..." He would point out several "rough spots."

Region Crossings
One of the first troubles we found was that region crossings were significantly worse between a cloud region and a datacenter region. We did a deep dive into the code for objects (boats, cars, planes, etc) and produced an improvement that made them significantly faster and more reliable even within the datacenter. This has been applied to all regions already and was a good step forward.

Group Chat stalls
Many users have reported that they are not able to get messages in some of their groups; we're very much aware of the problem. The start of those problems does coincide with when the chat service was uplifted; unfortunately the problems did not become clear until moving that service back to the datacenter was not an option. We haven’t been able to get that fixed as quickly as we would like, but the good news is that we have some changes nearly ready that we think may improve the service and will certainly provide us with better information to diagnose it if it isn't fixed. Those changes are live on the Beta grid now and should move to the main grid very soon.

Bake Failures
Wednesday and especially Thursday of this past week were bad days for avatar appearance, and we're very much aware of how important that is. The avatar bake service has actually been uplifted for some time - it wasn't moving it that caused the problem, but another change to a related service. The good news is that thanks to a great cross-team effort during those two days we were able to determine why an apparently unrelated simulator update triggered the problem and got a fix deployed Thursday night.

Increased Teleport Failures
We have seen a slight increase in the frequency of teleport failures. I know that if it's happened to you it probably doesn't feel like a "slight" problem, especially since it appears to be true that if it's happened to someone once, it tends to keep happening for a while. Measured over the entire grid, it's just under two percentage points, but even that is unacceptable. We're less sure of the specific causes for this (including whether or not it's Uplift related), but are improving our ability to collect data on it and are very much focused on finding and fixing the problem whatever it is.

Marketplace & Stipend Glitches
We've had some challenges related to uplift for both the Marketplace and the service that pays Premium Stipends. Marketplace had to be returned to the datacenter yesterday, but we'll correct the problems that required the rollback and get it done soon. The Stipends issues were both good and bad for users; there were some delays, but on the other hand we sent some users extra stipends (our fault, you win - we aren't taking them back); those problems are, we believe, solved now.

So why all of the glitches and bugs? Oz called the Uplift to move Second Life's data to cloud servers "a massive, complicated project that I've previously compared to converting a steam-driven railroad to a maglev monorail -- without ever stopping the train." He would later try to reasure the residents, "While this week in particular has seen some bumps in the road, it's actually going well overall. Lots of the infrastructure you don't interact with directly, and some you do, has been uplifted and has worked smoothly." He would go on to say almost all of the Beta Grid's sims were in the Cloud, and "we've uplifted around a hundred regions on the main grid. Performance of those regions has been very very good, and stability has been excellent." He would go on to say more sims would be uplifted over the weekend and that "Uplift of the Release Candidate regions, which will bring the count into the thousands, will begin soon."

Oz would say the Uplift to the Cloud should be completed, or at least be close to it, by the end of the year.

To read the blog entry in full, Click Here.

Bixyl Shuftan

Linden Lab Announces Improvements on Sim Crossings

One of the pet peeves a number of residents have is sim crossings, or "region crossings" as Linden Lab calls them. Sometimes there's a short pause. Occasionally there's a longer pause. And sometimes there's the "walk of lag" that results in you going walking on and on until you either "rubberband" back to the sim crossing, or the viewer crashes and you get knocked offline.

Last year, crashing after teleporting and sim crossing was getting to be worse of a problem. Though after a while, things began to improve. So how are things now? Yesterday on the "Tools and Technology" Linden blog, April Linden, announced that the programing on how sim crossings/teleports was changed and the results were that they were easier and less glitchy than ever.

The process of crossing from one region to another when you’re riding in a vehicle is pretty involved. The region you’re leaving needs to be able to tell the region where you’re arriving  everything it knows about the vehicle, and has to do it really quickly. That includes all of the scripts in the vehicle, everything that’s attached to it, the direction and how fast it’s going, and lots of other stuff. 

To make this happen quickly, early on in Second Life’s history we made some assumptions about our network, including things like how big a packet can be. Those assumptions generally worked okay on our own network, but not outside it. 

When you crossed from one region to another, the regions were putting a lot of information into large packets and sending them across our network. This was usually okay because our network was purposefully built to run Second Life. Then, as soon we tried to do this on someone else’s network (in the cloud), things didn’t work quite right. The problem was most noticeable when crossing from a region in our data center to one in the cloud. 

The first thing our engineering teams tried was breaking those large packets up into smaller ones, but that actually made the problem worse. Rather than send one big packet and wait for the other side to say it received the data, with smaller packets, it had to repeat that bunch of times for each packet. (Send, get an acknowledgement, send another piece, get an acknowledgement, etc.) It was still mostly okay across our network, but way worse when a region in our network was talking to one in the cloud. We now knew this code would never work well, so we needed a different approach. 

Next, our engineering team decided to use another way to send the data across the network, using the same protocol and method we use for other types of data. Most importantly, it is faster and more reliable. That did the trick! We’re still collecting statistics on the impact this change has, but things are looking very positive. 

Once this new code was written, the performance when going  from region to region got a lot better, and it worked between our data center and the cloud! The improvement was so dramatic that we decided to not make our Residents wait for uplifted simulators, and rolled the changes out right away. That code is what rolled out to the grid this week.

"It’s really exciting that the cloud migration is helping us find really old bugs and make Second Life better as we go," the "gridbun" would conclude.



Luca decided to test things out for herself. This video she did shows her testing several vehicles. Although some took longer than others, she concluded all showed some improvement, a few making the crossing in less than a second.

Have you found sim crossing and teleports easier? What about vehicle crossings? Please leave comments below.

Source: Linden blog, Luca 
Hat tip: Cynthia Farshore, Liffento 

Bixyl Shuftan

Update on the "Teleport Disconnects"

Earlier this month, Linden Lab came out and admitted what the residents had more or less figured out, Second Life was much buggier than usual, especially when it came to teleporting from sim to sim or crossing sim borders. So has there been improvement since then? Maybe some. Checking "Modem World," Inara Pey stated "There have been mixed reports on the outcome of the deployment made on Thursday, April 18th, 2019, with some indicating matters have improved, others reporting no real change. In particular, people visiting Fantasy Faire and travelling around the new Linden Homes continent appear to be faring a lot better than had been the case."

Simon Linden was quoted as saying at the Simulator User Group, " for server news, many of us are focused on the teleport issues … I hate to promise anything, but we all know a lot more about TP internal problems than we used to (smile) . We do understand they’re annoying and a frustrating problem – we definitely want to fix it and make it better."

Earlier, some residents wondered if the Lab moving data to Cloud servers was the cause of the glitches. Lately, there's been a new suspect, the EEP or Environmental Enhancement Project. Simon did admit that the problems appearing just after the release of EEP was "suspicious" and they were "trying to sort out is if somehow the environmental work caused the TP problems." He didn't think the glitches were related to avatar complecity, though stated "that said, the more complex your AV is, the more work it needs to change regions. It’s always been better to have fewer scripts and data for teleports and region crossings."

On a personal note, I myself seem to be having fewer problems with teleport disconnects, though there are still other glitches such as the viewer freezing for a bit. Some friends of mine are still crashing when teleporting around, so it seems the problem is still around.

Source: Modem World, SL Wiki

Bixyl Shuftan

Reader Submitted: "Hilarious TP Error"

Teleporting around is a pretty convenient way to get around Second Life, and when getting between places not connected, sometimes the only way. But sometimes, things go wrong. When Henning Lionheart tried going to Fairelands Junction, he ended up inside one of the mirrors.

From the outside, only his head, and the basket he was holding, showed. "This would be an annoying TP error if it hadn't been so hilariously perfect."

"Darn Teleport Disconnects," Lab Admits Grid Trouble

If you're been having trouble teleporting across Second Life lately, you're not alone. Lately, a number of people in group chats I've listened in on, as well as friends and associates in private messages, have been complaining about various glitches on the grid. Besides the lag, probably the most frequent complaint I've heard has been crashing while trying to teleport from one spot to another.

Well, it seems things have gotten to the point that the Lab has to acknowledge there's a problem. In the "Tools and Technology" part of the official Second Life blog on Monday April 9, they posted that they're aware of the problem.

Many Residents have noted that in the last few weeks we have had an increase in disconnects during a teleport. These occur when an avatar attempts to teleport to a new Region (or cross a Region boundary, which is handled similarly internally) and the teleport or Region crossing takes longer than usual.  Instead of arriving at the expected destination, the viewer disconnects ...

The Lab believes that the cause is "a timing-sensitive failure in the hand-off" between sims when a teleport takes place, and there doesn't seem to be a way to eliminate the chance of a crash. They have people trying to get to the root of the problem and get rid of it. "Quite unfortunately, this problem dodged our usual monitors of the behavior of simulators in the Release Channels, and as a result we're also enhancing those monitors to prevent similar problems getting past us in the future. We're sorry about this - we empathize with how disruptive it has been."

A few residents I've talked to wonder if Linden Lab's moving information of it's data centers to cloud servers has something to do with recent bugs. When Patch and Grumpity Linden spoke at the VWBPE last Thursday, Grumpity stated the Lab was "very carefully moving things and not disturbing the way things are set up." So if one takes the Lab's word for it, the problem isn't The Cloud.

In the meantime, it looks like residents will be dealing with the possibility of their teleports ending up as viewer crashes, and having to restart the viewers and slowly log back in.

Bixyl Shuftan

Classic SLN Cartoons

From 2009, presumably at a moment when my avatar somehow ended up at 0,0,0 when trying to port somewhere.

Bixyl Shuftan

Unscheduled Maintenance

Yesterday in the afternoon, things weren't going quite right for some Second Life residents. Some couldn't log in. Others couldn't teleport or ended up in places like the hub in the Adult sim of Arapama. Ending up there, the only way I could escape was to log off and relog at my Home location. So what was going on? A look at the Grid Status revealed a problem.

Unscheduled Maintenance
Posted by Status Desk on October 27th, 2014 at 03:24 pm PDT
[Posted 3:24 PM PDT, 27 October 2014] We are currently undergoing unscheduled maintenance to several systems. Some Residents may experience difficulties with multiple services, including logins, Web dashboards, and searches. Please keep an eye on this blog for details.

Eventually people were able to log in and port as they were before, though for a while, things were frustrating.