Follow

looks like cybre.space might be having intermittent problems with duplicating new posts made by users on here, looking into it now. (in the meantime, if someone sends you multiples of the same reply, it's probably not their fault!)

· · Web · 3 · 4 · 10

cybre.space users: this appears to be due to the cybre.space web server taking too long to respond and returning a 504 / timeout error. if you see this happen, it might mean the server is just processing slowly and will eventually submit the post! (looking into why it's taking so long, but this should help you avoid accidentally submitting duplicates in the meantime)

server slowness appears to be due to degraded disk performance on the host, following up with them to see if there's anything to be done about it

and then suddenly its fine?? didnt get a response to my support ticket but i'll take it

@chr if you use this as an excuse to load up my mentions again chr I swear there will be heck to pay

@chr I see this same behavior when memory is low or set to dynamic on the VM.

fwiw.

@TheGibson we seem to be good on memory and the sidekiq queues are empty, weirdly. current theory is it might be a slow db connection, possibly due to disk i/o

@chr yep, sounds like you’re probably right. IOPS are about all that’s left.

@chr cybre's hosting has repeatedly broken in various ways more frequently than seems normal for Cloud Hosting Providers and like, don't they have SLAs or something? i don't understand

@haskal digitalocean is definitely failing their SLA, but hetzner has been basically fine til now aside from an occasional short outage that kicked something of ours over. we are unfortunately getting what we're paying for, there, though.

@chr Very odd. Could it possibly be failing disks that switched to backups?

@chr did the issues coincide with a backup window of some kind? I used to get this kind of behaviour periodically, and it took me ages to put 2 and 2 together and realise what was going on.

@chr Duplicates shouldn't happen from the web client or any clients that use the Idempotency-Key header. Unless that's not working as intended...

@Gargron I've seen similar troubles with the crossposter on servers that are failing-ish, specially when the list contains pictures. I think the idempotency might have a corner case on those cases

Sign in to participate in the conversation
Cybrespace

cybrespace: the social hub of the information superhighway jack in to the mastodon fediverse today and surf the dataflow through our cybrepunk, slightly glitchy web portal support us on patreon or liberapay!