TeamSpeak Down

Anything and everything to do with Teamspeak goes in here.

Moderators: Board of Directors, Command

Paddy
Posts: 1386
Joined: Wed Sep 16, 2015 10:47 pm

Re: TeamSpeak Down

Post by Paddy » Sun Jan 31, 2016 11:36 pm

Following on from today's several outages, including a rather inconvenient one during them mission, depending on the response you get back you might do well to remind them that there are other providers you can equally move your business too if they're not interestedor unable to provide a stable service.
Cheers,

Paddy
Image
Group Captain John 'Paddy' Hemingway, we salute you

User avatar
Dickie
Group Captain
Group Captain
ACG Board
contributor
Posts: 13837
Joined: Sat Jul 14, 2012 12:15 pm
Location: Gloucestershire, England
Contact:

Re: TeamSpeak Down

Post by Dickie » Mon Feb 01, 2016 11:57 am

Yup, I'm far from happy with it. I shall be posting up the ticket response here so you guys can read it for yourselves.

Vranac
Posts: 1281
Joined: Sat Nov 17, 2012 9:59 pm

Re: TeamSpeak Down

Post by Vranac » Mon Feb 01, 2016 5:17 pm

What I was thinking of, is there any possibility to set up some reserve TS server for the cases like yesterday?
Does anyone have some machine with a static IP adress?

I mean, it isn't a big problem because that have happened only a few times but it would be good to have alternative TS in a case of emergency.
Image

Geordie
Posts: 1555
Joined: Fri Jan 23, 2015 10:25 pm
Location: Durham, United Kingdom

Re: TeamSpeak Down

Post by Geordie » Mon Feb 01, 2016 5:58 pm

Vranac wrote:What I was thinking of, is there any possibility to set up some reserve TS server for the cases like yesterday?
Does anyone have some machine with a static IP adress?

I mean, it isn't a big problem because that have happened only a few times but it would be good to have alternative TS in a case of emergency.
Could we host our own TS server on the same server as our mission server? I don't know how much bandwidth VoiP on TS uses though. Just a thought.
I mean, I had fast motor cars and fast motor bikes, and when I wasn't crashing airplanes, I was crashing motor bikes. It's all part of the game — Sir Harry Broadhurst

User avatar
Dickie
Group Captain
Group Captain
ACG Board
contributor
Posts: 13837
Joined: Sat Jul 14, 2012 12:15 pm
Location: Gloucestershire, England
Contact:

Re: TeamSpeak Down

Post by Dickie » Mon Feb 01, 2016 6:06 pm

Vranac wrote:What I was thinking of, is there any possibility to set up some reserve TS server for the cases like yesterday?
Does anyone have some machine with a static IP adress?

I mean, it isn't a big problem because that have happened only a few times but it would be good to have alternative TS in a case of emergency.

Totally, and this is one of the things I said to them. I warned them of a potential issue at 6pm, they said "yeah, it'll be ok" and then nothing else. Then it went tits up and the response was an explanation of what DDOS is which I could've got from WIkipedia - I didn't even read it.

I suggested that they should have a backup TS for their customers - this isn't over. :veryangry:


Geordie, I could probably host but I think that 'free' TS is limited numbers - I think there's no setup for the 150 seats we have.

Vranac
Posts: 1281
Joined: Sat Nov 17, 2012 9:59 pm

Re: TeamSpeak Down

Post by Vranac » Mon Feb 01, 2016 6:42 pm

Geordie wrote:Could we host our own TS server on the same server as our mission server? I don't know how much bandwidth VoiP on TS uses though. Just a thought.
No, I came to the same idea yesterday and asked Pitti about that. We should pay more for the server if we want TS server on it.

Kuntz, we were running 'free' TS server for SEOW but that was for 60-70 pilots.
Image

Paddy
Posts: 1386
Joined: Wed Sep 16, 2015 10:47 pm

Re: TeamSpeak Down

Post by Paddy » Mon Feb 01, 2016 7:36 pm

The point for me is that there are ways to mitigate DDoS attacks and these fellas sound like they know nothing of it. I remember hearing that this is not the first time this has happened so measures should have been put in place by them. I would still like to know if it was our virtual server that was hit or the baremetal. I would also like to know what countermeasure they are going to put in place, if any.
Cheers,

Paddy
Image
Group Captain John 'Paddy' Hemingway, we salute you

User avatar
Dickie
Group Captain
Group Captain
ACG Board
contributor
Posts: 13837
Joined: Sat Jul 14, 2012 12:15 pm
Location: Gloucestershire, England
Contact:

Re: TeamSpeak Down

Post by Dickie » Tue Feb 02, 2016 10:23 am

#2916453

TS server down

It's down, and then pops up for a fleeting moment, then drops out again. "Failed to connect to server" messages. Are you being DDOS'd??

We have a 130 person event tonight.



Hello,

I've performed some checks on your service and have found that your server has been taken offline for a while, judging from our monitoring and logs this was the result of a group of people attacking individual servers on the machine you're hosted on with a form of packet spamming known as a DDoS attack. It's extremely difficult to prevent/block these "DDoS attacks" before they start due to the nature of the attacks. The way they work is by flooding the server with traffic and reaching the limit of the network, this then stops legitimate traffic from getting through.

I've just checked to make sure and it appears that your server has now been brought back online and is fully functioning :)

If your issue is resolved, please do not reply to this ticket. Simply mark it as resolved and please rate my support when you receive the survey.
You can also email customerservice@multiplay.co.uk and let them how how I'm doing.

I sincerely apologise for any inconvenience you may have experienced. If you have any further queries, please do not hesitate to contact us again.



I know what a DDOS attack is, but what safeguards do you have? We just had the server go down again mid-way through our gaming night tonight.

Is it from one location or multiple? Why can't you block the location??

Also, why can't you just give us another server, a temporary one, to use if we're getting such issues? I reported this before we started, but the support help is just not helpful.



Hello

I'm terribly sorry you're having issues :(

We are aware some servers are currently experiencing DDOS attacks, it is important that you understand why this problem is occurring and the widescale effects that are ongoing as a result of the current climate of attacks.

DDoS attacks these days are done using an exploit known as "Reflection".

Reflection attacks work by sending a packet with a spoofed origin address to a server requesting data, while this packet is relatively small the amount of data the server replies with is much larger - amplifying the attack. The server replies to the "origin" of the packet (the spoofed part) it points these servers to the attack target.

They send this packet to many hundreds of servers, and what would ordinarily be a very small DDoS becomes an unreasonably large attack with very little effort, for a visual representation of what I just described see this diagram: http://puu.sh/c6BfN.jpg.

Since this exploit was discovered, some of the largest DDoS attacks in history are being recorded, some of these attacks are reaching highs of 400Gb per second. Unfortunately there isn't a single transit provider or even an ISP on the planet that can push that kind of data without suffering significant network-wide problems.

The scale of the attack simply takes down the transit provider/ISP itself, and renders the server uncontactable regardless.

As a result, there isn't any kind of protection that can be offered against these attacks. There isn't a single system that we could install that would react fast enough, or offer any realistic or even potential protection against this style of attack. We mitigate the attacks as best as we can by implementing ACLs and null routes in real time to deal with the traffic. If you experience an attack you can guarantee that our network engineers know about it and are running around as though their heads were on fire working to fight against it. But when a network gets that heavily saturated, there will be lag, there will be disconnections, regardless of how hard they work against them, they are simply too large for any system.

We try as hard as we can to minimise the extent and duration of these symptoms, but it is a losing battle until the backbone of servers around the world being used as amplifiers that are susceptible to the Reflection exploit have their systems patched. Unfortunately that is out of ours or anybody's hands - it's down to the owners of those servers being exploited to actually patch their equipment to stop them being used in attacks.

It is vital you understand that it doesn't really matter where you go, which provider you go with or what network they use - if they get attacked on these scales, they will have problems.

What I CAN do for you is move you to another machine on our network that isn't currently receiving attacks but this will result in voice comms settings being reset, and your IP and port changing.

Please let me know if you'd like us to go through with this. Again - I'm very sorry you're experiencing problems!



Thanks for your response Michael. I wouldn't want to change address permanently since we've made a lot of configuration to our home, but is it possible to set up a server somewhere temporarily in such situations? ie, if you know which servers are getting hammered or does it simply affect all of them?

Unfortunately not while the attack is under way, We've had issues in the past where moving a sever while the attack is ongoing would result in two ghost servers.

Generally while the attack is ongoing it is best to wait it out I'm afraid.

Generally it is not a specific server being attacked and often just the machine.


Bunny
Posts: 5431
Joined: Tue Jul 17, 2012 2:56 pm

Re: TeamSpeak Down

Post by Bunny » Tue Feb 02, 2016 10:59 am

They should shoot spammers in any form. No questions asked.

“It is a mistake to think you can solve any major problems just with potatoes I-16s.” - Douglas Adams


Image

Post Reply