~ This event isn't fiction. Here follows the description of these events as they unfolded~
Paul had been working for this client for 15 years. He built him all of his sites!
Paul even wrote one of the Ecommerce platforms in PHP. Can you imagine? From scratch!— A la mano. He set up the servers, the synchronization scripts with the warehouses to manage the stock. He knows the business by heart, has all the settings in his head. *sigh* What would we do without Paul, and that's why we pay him so well.
But this morning, end of September 2018, Paul died. Boom. 40 years old. Heart attack. -As if it wasn't tragic enough the main server failed 2 days later. No sound, no image. Just a ping.
…. Shit ….
Luckily, a few months ago, this client had asked us to give Paul a hand on some Prestashop. We didn't have root administration access, but just limited developer access. Basically we were completely screwed.
On the phone I tell the client : Don't worry bro, I'm putting Pavel and Artem on this.You have no idea how much damage 2 senior Ukrainian sys-admins can inflict on a badly kept up server in very little time. (all within the confines of the Law of course). I'm used to it now, I've been working with my team for 11 years now, but to see so much experience going away at fixing a problem that's so complex amazes me every time.
- Within 12 hours all the sites were online on a new server with reinforced security, we had root access and the client was in business again.
- Within 24 hours, we had done a full audit and created technical documentation for all the sites, with a working plan of what we thought would be indispensable for the operation of the business.