Dear community,
This is a full report of what happened yesterday night and lasted until this morning.
The Problem
We detected an overload in one of our serves, and we immediately proceeded to upgrade it. After this action, which left the main website (zignaly.com) down for a few minutes (not the app, the API, or the internal processes), the server immediately became overloaded again, so we understood that we were suffering a DDoS attack.
We started to mitigate the problem, but it was a distributed attack, making it challenging to block IPs (there were thousands of IPs involved). After several unsuccessful attempts, we decided to implement a third-party solution, Cloudflare, to act as a shield.
With Cloudflare, we stopped the attack. To clarify, that nothing was compromised in the platform, your assets, API keys, nothing were exposed at any time. The only problem was that our servers were overloaded.
Collateral Damage
However, installing this DNS required a DNS change, and even when our nameservers were configured to be checked each hour, this is something that not everybody uses, and some services take longer, or there could be even caches in browsers and routers. Some people were still accessing the old IP (which was blocked for traffic outside of Cloudflare networks). Additionally, the firewall blocked TradingView and algo-bots, because it was asking for a challenge before continuing the connection, and because they are legitimate bots, they didn’t pass it.
At this point, these network’s collateral effects are solved, and everybody should access any part of Zignaly without any problem (feel free to open a ticket if you still experience any issue).
Unfortunately, the timing couldn’t have been worse. The market crash made it bad; internal triggers from Zignaly worked, but people who were trying to operate manually, who had DNS problems, or from TradingView and algo-bots, weren’t able to handle their positions. At this point, we have recovered all the positions that were closed because of this malfunction.
Actions Taken
We have recovered these positions, and they should be accessible again from your dashboard.
For those of you following profit-sharing services, you may have experienced problems with disconnections or withdraws. We are handling these problems now, but this will take a while because it requires individual attention.
Due to this issue, some profit-sharing futures accounts were liquidated. We are looking into each one individually, and we plan to cover the loss, but this requires a deep investigation, so we ask you to be patient, please. Once the services are restored, the disconnections and withdraws will work without any problem.
Learnings
Just one last thing. From here we get two learnings:
- We are implementing better systems to avoid such situation in the future
- We will improve the way we communicate across channels (Telegram, Twitter, Discord, etc.)
Final
Thanks for your understanding and patience. We want to assure you that we have been working on this issue since we detected it without rest. It will take a few days to fix all the accountings, but the platform is operating without any other issues since this morning.
